Aircraft pulse attitude control method based on incremental dual-heuristic dynamic programming

Abstract

The invention discloses an aircraft pulse attitude control method based on incremental dual-heuristic dynamic programming, which comprises the following steps: firstly, designing a state quantity to carry out state characterization on an aircraft according to attitude dynamic characteristics of the aircraft and characteristics of an attitude control execution mechanism, designing an action based on an input quantity required by the execution mechanism, and carrying out dynamic programming on the aircraft; designing a value function according to the aircraft attitude control target; then, according to the designed parameters, a self-adaptive dynamic planning network based on an actuator-evaluator network is constructed, dynamic parameters of the aircraft are identified by using an incremental model, and network weights are updated according to the identified parameters; the actuator-evaluator network parameters are updated through an online identification result and a forward propagation algorithm, and the incremental model is updated by using an iterative least square method; and finally, designing a pulse conversion function, and converting the action through a pulse width modulation technology to realize pulse attitude control of the aircraft.

Description

Technical Field

[0001] The present invention belongs to the field of aircraft attitude intelligent control, and in particular, relates to an aircraft pulse attitude control method based on incremental dual heuristic dynamic programming. Background Art

[0002] At present, with the continuous development of aerospace science and technology, the importance of aircraft control technology is becoming increasingly prominent. Attitude control technology plays a vital role in ensuring the smooth operation of aircraft. For aircraft within the atmosphere, the aircraft attitude affects the aerodynamic force and air resistance it is subjected to, and these forces increase with the increase of speed. It is particularly important to maintain a specific flight attitude. For aircraft outside the atmosphere, aerodynamic control is no longer effective, and the pulse attitude control technology that replaces it is of great significance. Specifically, attitude and orbit thrust control is usually used outside the atmosphere, and the thrust vector direction is related to the attitude of the aircraft, so the orbit control performance will be affected by attitude instability. At the same time, aircraft usually have autonomous detection capabilities, and the calculation of the seeker's line of sight and relative information also requires aircraft attitude information. Attitude instability will also affect the detection accuracy. Therefore, aircraft pulse attitude control technology has important research value and engineering significance.

[0003] One of the most typical and widely used methods in the traditional attitude control of aircraft is the proportional-integral-differential control method. This type of method has the characteristics of wide adaptability, simple method and easy engineering implementation, but when faced with more complex models, working conditions and physical characteristics with characteristics such as parameter uncertainty, the control performance and control accuracy are reduced; based on modern control theory, intelligent optimal control methods such as nonlinear intelligent predictive control and optimal learning control can overcome the above problems to a certain extent, and have been applied to some aircraft. However, the existing methods still have problems such as large amount of calculation and long time consumption, and it is difficult to meet the real-time requirements of the flight control system. In particular, how to design an intelligent online attitude control method for pulsed attitude control aircraft has strong engineering application and theoretical research significance. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of existing methods in aircraft pulse attitude control technology and design a pulse attitude control method based on an intelligent method. The design method of the present invention is an attitude control method based on incremental dual heuristic dynamic programming, which has the characteristics of fast online adaptation, can dynamically estimate system parameter changes and use them to update network parameters, thereby achieving timely tracking of the desired attitude.

[0005] To achieve the above technical objectives, the technical solution of the present invention is as follows:

[0006] A method for aircraft pulse attitude control based on incremental dual heuristic dynamic programming includes the following steps:

[0007] S1: Considering the attitude dynamics of the aircraft and the characteristics of the attitude control actuator, design state quantities and actions to characterize and control the aircraft state, design actions based on the input quantities required by the actuator, and design value functions according to the attitude control objectives of the aircraft;

[0008] S2: Based on the state quantity and action designed in step S1, an adaptive dynamic programming network based on the actuator-evaluator network is constructed, the dynamic parameters of the aircraft are identified using the incremental model, and the network parameters are updated according to the dynamic parameters obtained by the identification;

[0009] S3: The parameters of the actuator-evaluator network are updated by online identification parameters combined with the forward propagation algorithm, and the incremental model is updated using the least squares method;

[0010] S4: Design a pulse conversion function to transform the action through pulse width modulation technology to achieve pulse attitude control of the aircraft;

[0011] The beneficial effects of the present invention are:

[0012] (1) The aircraft pulse attitude control method of the present invention is applicable to various aircraft that use pulse engines for attitude control, and can quickly adapt different aircraft systems online to achieve the desired attitude control.

[0013] (2) The present invention adopts an incremental dual-heuristic dynamic programming method, which does not require modeling of a complete aircraft model, nor does it need to rely on a pre-trained network parameter model. It has the characteristics of online adaptability and good robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 It is a flow chart of the aircraft pulse attitude control method based on incremental dual heuristic dynamic programming of the present invention;

[0016] Figure 2 It is a schematic diagram of the inference characteristic curve of attitude control pulse thrust output;

[0017] Figure 3 It is a schematic diagram of the principle of the incremental dual heuristic dynamic programming network;

[0018] Figure 4This is a schematic diagram of attitude control simulation. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above-mentioned purpose, the present invention adopts the following technical scheme.

[0020] like Figure 1 As shown, the steps of an aircraft pulse attitude control method based on incremental dual heuristic dynamic programming of the present invention are as follows: first, considering the attitude dynamic characteristics of the aircraft and the characteristics of the attitude control actuator, design state quantities and actions to characterize and control the state of the aircraft, design actions based on the input quantities required by the actuators, and design value functions according to the aircraft attitude control objectives; then, based on the state quantities and actions designed in step S1, construct an adaptive dynamic programming network based on an actuator-evaluator network, use an incremental model to identify the dynamic parameters of the aircraft, and update the network parameters according to the dynamic parameters obtained by the identification; wherein the actuator-evaluator network parameters are updated by combining online identification parameters with a forward propagation algorithm, and the incremental model is updated by using the least squares method; finally, design a pulse conversion function, transform the actions through pulse width modulation technology, and realize pulse attitude control of the aircraft.

[0021] It can be seen from the above steps that this method mainly realizes the processing of aircraft attitude information and the generation of control signals through the combined action of the adaptive dynamic programming network and the incremental model. The incremental network is mainly used to identify system parameters to obtain more accurate information about the aircraft, and then update the parameters of the adaptive dynamic programming network. On this basis, the control method of the present invention can realize adaptive adjustment of attitude changes to ensure the achievement of the desired attitude.

[0022] It should be understood that in the present invention, whenever a character is marked with a “·”, it is the first-order derivative of the variable represented by the character.

[0023] The specific implementation steps of the aircraft pulse attitude control method based on incremental dual heuristic dynamic programming of the present invention are as follows:

[0024] S1: Considering the attitude dynamics of the aircraft and the characteristics of the attitude control actuator, design state quantities and actions to characterize and control the aircraft state, design actions based on the input required by the actuator, and design value functions according to the attitude control target of the aircraft. The specific implementation details are as follows:

[0025] First consider the following aircraft attitude dynamics equation:

[0026] (1)

[0027] in, , and are the three-axis angular velocities of the aircraft, , and are the moments of inertia of the aircraft around the three axes, , and They are the Euler angles of the aircraft. It can be seen that the attitude dynamics of the aircraft is a nonlinear second-order system. At least two state quantities of a single channel should be selected, generally angle and angular velocity. Combined with the control requirements, that is, the pitch angle of the aircraft follows the height angle of the line of sight, the yaw angle follows the azimuth angle of the line of sight, and the roll angle is kept at 0. The design state is as follows:

[0028] (2)

[0029] in, and They are the altitude angle and azimuth angle of the aircraft's sight respectively.

[0030] The attitude action actuator, that is, the pulse engine has only two states: on or off. The on state outputs a rated thrust, and the off state does not output thrust. The output curve of the attitude action actuator is as follows: Figure 2 As shown, In the figure, the output thrust in the open state is shown. , Represent the opening and closing time respectively, and the delay time is For attitude control engine groups, the engines are usually combined in a specific way to generate attitude control torque, and the forward and reverse directions are generally of the same magnitude. Therefore, it can be assumed that the maximum torques of the three attitude channels of the aircraft are , then for any channel torque, such as rolling torque Only possible values , or .

[0031] Based on this, the actions can be designed as follows:

[0032] (3)

[0033] in, are the attitude control moments of the longitudinal axis, vertical axis and lateral axis of the aircraft respectively.

[0034] Finally, for the design of the value function, in order to facilitate the optimization process, it is designed in the form of a quadratic form as follows:

[0035] (4)

[0036] in, As the reference input, take the system state at the previous moment; is the value allocation matrix.

[0037] The cumulative discount value function is defined as follows:

[0038] (5)

[0039] in, is the discount factor. And the partial derivative of the value function with respect to the state quantity is recorded as:

[0040] (6)

[0041] S2: Based on the state and action designed in step S1, an adaptive dynamic programming network based on the actuator-evaluator network is constructed, and the dynamic parameters of the aircraft are identified using the incremental model, and the network parameters are updated according to the dynamic parameters obtained by identification. The specific implementation details are as follows:

[0042] like Figure 3 As shown in the figure, the solid line is the signal flow of the model network, and the dotted line is the network parameter update flow. The aircraft state is updated in real time according to the system dynamics and transmitted to the evaluator, actuator and incremental model. Subsequently, the actuator network gets the action And output to the system and incremental model, the evaluator obtains an estimate of the partial derivative of the value function , incremental model update and output of model parameter estimates and , combined with the evaluator output to update the parameters of the actuator and evaluator networks.

[0043] S3: The parameters of the actuator-evaluator network are updated through online identification parameters and forward propagation algorithm, and the incremental model is updated using the least squares method; the parameter update steps in step S2 are explained as follows:

[0044] The original model dynamic equation of the incremental model is:

[0045] (7)

[0046] in, is the state quantity, For action, is a first-order differentiable system state transfer function, for the equation at time Taylor expansion is performed:

[0047] (8)

[0048] in,

[0049] (9)

[0050] (10)

[0051] are the system transfer matrix and control gain matrix respectively, for arrive The system state increment at time, for arrive The increment of the control input at any time.

[0052] The above system correlation matrix is ​​needed to update the network parameters. However, the exact value of this matrix cannot be directly obtained. Therefore, the least squares method is used to estimate the parameters online.

[0053] remember ,parameter , then the following state estimation equation can be obtained:

[0054] (11)

[0055] The parameters can be adjusted by historical system status increments and control input increments. Perform least squares estimation. For the incremental model here, recursive least squares can be further used to iterate at each step to obtain the estimation result. First, define the estimation error:

[0056] (12)

[0057] ,

[0058] Based on the estimated error, the iterative calculation formula of the parameters can be further obtained:

[0059] (13)

[0060] in, is the estimated coefficient, is the estimated covariance matrix. The iterative update formula can be used to achieve the parameter , that is, the system parameters and Online identification.

[0061] The evaluator is optimized by minimizing the following error function:

[0062] (14)

[0063] In the formula, the timing difference error The definition is as follows:

[0064] (15)

[0065] In the formula, and The evaluator can be used for real-time estimation. This can be derived based on the definition of the discount value and the status value of the previous step. is the partial derivative of the system state with respect to the state at the previous moment, given by Figure 3 It can be seen that arrive There are two update paths: one is The update within the system is determined by the characteristics of the system itself, which can be estimated by the aforementioned incremental model. to replace; the second is After passing through the actuator, the action is obtained and then enters the system for update. Therefore, the following calculations can be made according to the two paths:

[0066] (16)

[0067] Using incremental modeling and Approximation:

[0068] (17)

[0069] in, It can be calculated by the actuator model when calculating the action value.

[0070] The parameters of the evaluator network are , the learning rate of the evaluator is recorded as , the parameter update method is as follows:

[0071] (18)

[0072] (19)

[0073] The actuator is implemented by minimizing the value function:

[0074] (20)

[0075] The weights are updated using the gradient descent method. The actuator network parameters are , define the executor learning rate , then:

[0076] (twenty one)

[0077] (twenty two)

[0078] In the formula,

[0079] (twenty three)

[0080] The two terms on the right side of the above formula have been obtained in the discussion of the evaluator; Use the evaluator estimate instead; From the incremental model expression, we can know , so it can be used Replacement; The gradient of the network calculation result with respect to the parameter can be obtained by storing the node value when forward calculating the action value. So the calculation expression of the actuator weight update can be obtained:

[0081] (twenty four)

[0082] In step S4, a pulse conversion function is designed to transform the action through pulse width modulation to achieve pulse attitude control of the aircraft. The pulse conversion function is as follows:

[0083] (25)

[0084] in, is a set of attitude control engine output torque magnitude, is the number of pulses contained in a single output cycle, is the design parameter, quantity conversion function The design is as follows:

[0085] (26)

[0086] in, is the Gaussian rounding function, is the number of pulses contained in a single output cycle, For input.

[0087] The following takes the tracking process of a certain exoatmospheric aircraft to a target as an example to illustrate the effectiveness of the method proposed in the present invention. Among them, the initial attitude angle of the aircraft is a random value within -5°~5°, the simulation time is set to 100s, the step size is 1ms, and the torque generated by the pulse attitude control engine group is set to , simulations were performed separately in scenarios where the expected sight elevation angle and azimuth angle were both 3°. Figure 4 The simulation results of attitude control using the method described in this application in the corresponding simulation scenario. Among them, Figure a is the pitch tracking situation, Figure b is the yaw tracking situation, Figure c is the roll tracking situation, Figure d is the roll moment output curve, Figure e is the yaw moment output curve, Figure h is the pitch moment output curve, Figure g is the model pitch increment, Figure h is the model yaw increment, Figure i is the model roll increment, Figure j is the model pitch rate increment, Figure k is the model yaw rate increment, and Figure l is the model roll rate increment.

[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for aircraft pulse attitude control based on incremental dual heuristic dynamic programming, characterized in that: The following steps are involved: S1: Design state quantities and actions to characterize and control the state of the aircraft based on the attitude dynamics characteristics of the aircraft and the characteristics of the attitude control actuator. Design actions based on the input quantities required by the actuator. Design value functions based on the attitude control objectives of the aircraft. S2: Based on the state and action designed in step S1, an adaptive dynamic programming network based on the actuator-evaluator network is constructed, the dynamic parameters of the aircraft are identified using an incremental model, and the network parameters are updated according to the identified dynamic parameters; S3: The parameters of the actuator-evaluator network are updated through online identification parameters and forward propagation algorithm, and the incremental model is updated using the least squares method; S4: Design a pulse conversion function to transform the action through pulse width modulation to achieve pulse attitude control of the aircraft.

2. The control method according to claim 1, characterized in that: In step S1, the aircraft attitude dynamics equation is as follows: ， in, , , are the three-axis angular velocities of the aircraft, , , is the moment of inertia of the aircraft around the three axes, , , They are the pitch, yaw and roll angles of the aircraft respectively. The symbol with "·" above the character represents the first-order derivative of the variable represented by the character.

3. The control method according to claim 2, characterized in that: In step S1, the state quantity is designed as follows: ， in, and It is the altitude angle and azimuth angle of the aircraft’s sight.

4. The control method according to claim 3, characterized in that: In step S1, the action is designed as follows: ， in, , , They are the attitude control moments of the aircraft's transverse axis, lateral axis, and longitudinal axis respectively.

5. The control method according to claim 4, characterized in that: In step S1, the value function is designed based on the desired control target of the attitude, which is to make the pitch angle of the aircraft follow the line of sight altitude angle, the yaw angle follow the line of sight azimuth angle, and keep the roll angle at 0; the designed value function is as follows: ， in, As the reference input, take the system state at the previous moment; It is a value distribution matrix, which reflects the importance of different states during iteration and is a positive definite symmetric matrix.

6. The control method according to claim 5, characterized in that: In step S2, the actuator input is the state quantity, and the output action is calculated through the network .

7. The control method according to claim 6, characterized in that: In step S2, the evaluator inputs the state quantity, and the network calculates and outputs the partial derivative of the value function with respect to the state. The estimated value of .

8. The control method according to claim 7, characterized in that: In step S2, the incremental model is an incremental description of the original model equation, and the original model dynamic equation is as follows: ， in, is a first-order differentiable system state transfer function, is the system state quantity, The input of the designed incremental model is state quantity and action, and the system transfer matrix is ​​calculated by the least square method. and Make an estimate.

9. The control method according to claim 8, characterized in that: In step S3, in the update algorithm, the evaluator parameters are obtained by minimizing the following evaluator error function: ， In the formula, the timing difference error The definition is as follows: ， in, is the cumulative discounted value function, is the discount factor, Indicates the current step number.

10. The control method according to claim 9, characterized in that: In step S3, in the update algorithm, the actuator parameters are updated by minimizing the cost function To update: 。 11. The control method according to claim 10, characterized in that: In step S3, in the update algorithm, the incremental model updates the estimated covariance matrix as follows by recursive least squares method: ， in, To estimate the coefficients, the system output and actions at each step are used to dynamically update the estimated values ​​of the system parameters: ， Among them, the estimation error The definition is as follows: ， is the estimated state matrix, is the parameter estimation matrix; Respectively represent the increment of state quantity and action single step, namely: ， 。 12. The control method according to claim 11, characterized in that: In step S4, the pulse conversion function is as follows: ， in, is a set of attitude control engine output torque magnitude, is the number of pulses contained in a single output cycle, is the design parameter, quantity conversion function The design is as follows: ， in, is the Gaussian rounding function, is the number of pulses contained in a single output cycle, For input.

Images