Spin aircraft attitude reinforcement learning control method and system under saturation nonlinearity

Abstract

The invention provides a spin aircraft attitude reinforcement learning control method and system under saturation nonlinearity. The method comprises the following steps: acquiring instruction values of mathematical rudder deflection angles of a yaw channel and a pitching channel; acquiring instruction values of physical rudder deflection angles of the yaw channel and the pitching channel based on the instruction value of the mathematical rudder deflection angle; based on the instruction values of the physical rudder deflection angles, the physical rudder deflection angles of the yaw channel and the pitching channel are obtained; based on the physical rudder deflection angles of the yaw channel and the pitching channel, mathematical rudder deflection angles of the yaw channel and the pitching channel are generated; and based on the instruction value of the quasi attack angle, the instruction value of the quasi sideslip angle and the output state at the current moment, generating the state of the controlled object, and sending the state to the trained reinforcement learning network model. According to the method, saturation nonlinearity is considered when the deflection angle of the physical rudder is obtained, the attitude control parameters can be automatically adjusted, the obtaining efficiency and accuracy of the attitude control parameters can be improved, and higher control precision can be achieved.

Description

Technical Field

[0001] The present invention relates to the field of aircraft attitude control, and in particular to a method and system for attitude reinforcement learning control of a spinning aircraft under saturated nonlinearity. Background Art

[0002] For a spinning vehicle, the roll channel keeps rotating at a high speed and is uncontrolled. It is necessary to perform attitude or overload control on the pitch and yaw channels to achieve the desired flight trajectory, such as controlling the angle of attack of the pitch channel and the sideslip angle of the yaw channel, or controlling the overload of the pitch and yaw channels. The control of a spinning vehicle mostly uses a servo system, and the deflection of the rudder surface will change the flight attitude. Unlike non-rotating vehicles, the high-speed roll of a spinning vehicle leads to a complex coupling effect between the dynamics of the pitch and yaw channels, mainly including inertial coupling caused by the gyro effect, aerodynamic coupling caused by the Magnus effect, and control coupling caused by the servo response delay. In addition, the high-speed roll makes the frequency of the servo command very high. The saturation nonlinearity inside the servo will cause large errors when tracking high-frequency commands, resulting in severe attitude limit cycle oscillation or even instability under the action of the outer loop attitude control system.

[0003] At present, in terms of the control of autopilots, the main solution is to model the servo as a first-order or second-order inertial link. However, this solution ignores the saturation nonlinearity inside the servo. However, servo saturation may cause severe attitude limit cycle oscillation or even instability. Therefore, servo saturation must be considered in controller design. Although there are control schemes that consider the gap nonlinearity inside the servo, only stability analysis is performed, and it is not solved through controller design and parameter optimization. In addition, the control parameters of the autopilot are manually adjusted and fixed. Manual parameter adjustment mainly relies on personal experience and trial and error, which is a cumbersome process and a long cycle. In addition, the existing parameter optimization methods rely on object linear models, such as pole configuration and loop shaping in overload autopilots. When there is servo nonlinearity in the controlled object, these methods are not applicable. Summary of the invention

[0004] In view of the above technical problems, the technical solution adopted by the present invention is:

[0005] According to a first aspect of the present invention, a method for attitude reinforcement learning control of a spinning aircraft under saturated nonlinearity is provided, the method comprising the following steps:

[0006] S100, based on the current attitude control parameters, the output state of the controlled object and the command values ​​of the quasi-angle of attack and the quasi-sideslip angle of the controlled object received at the current moment, obtain the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel; wherein the controlled object is a dynamic model of the spinning aircraft, and the output state of the controlled object includes the quasi-angle of attack, the quasi-sideslip angle, the yaw angular velocity and the pitch angular velocity.

[0007] S200, respectively decompose the instruction value of the mathematical rudder angle of the current yaw channel and the instruction value of the mathematical rudder angle of the current pitch channel to obtain the instruction value of the physical rudder angle of the current yaw channel and the instruction value of the physical rudder angle of the current pitch channel.

[0008] S300, obtaining the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel based on the instruction value of the physical rudder deflection angle of the current yaw channel and the instruction value of the physical rudder deflection angle of the current pitch channel.

[0009] S400, based on the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel, generate the mathematical rudder deflection angle of the current yaw channel and the mathematical rudder deflection angle of the current pitch channel, and send them to the controlled object to obtain the output state of the controlled object at the current moment.

[0010] S500, based on the command value of the quasi-angle of attack, the command value of the quasi-sideslip angle and the output state at the current moment, generate the state of the controlled object at the current moment as the current state of the controlled object; the state of the controlled object includes the control error of the quasi-angle of attack, the control error of the quasi-yaw angle, the yaw angular velocity and the pitch angular velocity, the control error of the quasi-angle of attack is the difference between the command value of the quasi-angle of attack and the quasi-angle of attack, and the control error of the quasi-sideslip angle is the difference between the command value of the quasi-sideslip angle and the quasi-sideslip angle.

[0011] S600, sending the current state of the controlled object to the trained reinforcement learning network model to obtain the current posture control parameters; executing S100.

[0012] According to the second aspect of the present invention, there is provided a control system for attitude reinforcement learning of a spinning aircraft under saturated nonlinearity, the system comprising: a control parameter optimization module, a mathematical rudder angle command value generation module, a mathematical rudder angle command value decomposition module, a physical servo module, a physical rudder angle synthesis module, a controlled object and a state generation module; the controlled object is a dynamic model of the spinning aircraft. The mathematical rudder angle command value generation module is used to obtain the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel based on the current attitude control parameters, the output state of the controlled object and the command values ​​of the quasi-attack angle and quasi-sideslip angle of the controlled object received at the current moment, the output state includes the quasi-attack angle, the quasi-sideslip angle, the yaw angular velocity and the pitch angular velocity; the mathematical rudder angle command value decomposition module is used to decompose the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel. The physical rudder angle synthesis module is used to obtain the physical rudder angle of the current yaw channel and the physical rudder angle of the current pitch channel based on the command value of the physical rudder angle of the current yaw channel and the command value of the physical rudder angle of the current pitch channel, and send them to the physical rudder angle synthesis module; the physical rudder angle synthesis module is used to obtain the physical rudder angle of the current yaw channel and the physical rudder angle of the current pitch channel based on the command value of the physical rudder angle of the current yaw channel and the command value of the physical rudder angle of the current pitch channel, and send them to the physical rudder angle synthesis module; the physical rudder angle synthesis module is used to obtain the physical rudder angle of the current yaw channel and the physical rudder angle of the current pitch channel based on the command value of the physical rudder angle of the current yaw channel and the command value of the physical rudder angle of the current pitch channel. The rudder angle is used to generate the mathematical rudder angle of the current yaw channel and the mathematical rudder angle of the current pitch channel, and send them to the controlled object; the controlled object is used to obtain the output state at the current moment based on the received mathematical rudder angle of the current yaw channel and the mathematical rudder angle of the current pitch channel, and send them to the state generation module and the mathematical rudder angle command value generation module; the state generation module is used to generate the state of the controlled object at the current moment based on the command value of the quasi-angle of attack, the command value of the quasi-sideslip angle and the output state at the current moment, as the current state of the controlled object, and send it to the control parameter optimization module; the state of the controlled object includes the control error of the quasi-angle of attack, the control error of the quasi-yaw angle, the yaw angular velocity and the pitch angular velocity, the control error of the quasi-angle of attack is the difference between the command value of the quasi-angle of attack and the quasi-angle of attack, and the control error of the quasi-sideslip angle is the difference between the command value of the quasi-sideslip angle and the quasi-sideslip angle; the control parameter optimization module is used to generate the current attitude control parameter based on the received current state of the controlled object, and send it to the mathematical rudder angle command value generation module.

[0013] The present invention has at least the following beneficial effects:

[0014] The embodiment of the present invention provides a method for attitude reinforcement learning control of a spinning aircraft under saturated nonlinearity, comprising: obtaining the instruction value of the mathematical rudder deflection angle of the current yaw channel and the instruction value of the mathematical rudder deflection angle of the current pitch channel based on the current attitude control parameters, the output state of the controlled object and the instruction values ​​of the quasi-attack angle and the quasi-sideslip angle of the controlled object received at the current moment, wherein the current attitude control parameters are obtained by inputting the current state of the controlled object into a trained reinforcement learning network model; respectively decomposing the instruction value of the mathematical rudder deflection angle of the current yaw channel and the instruction value of the mathematical rudder deflection angle of the current pitch channel to obtain the instruction value of the physical rudder deflection angle of the current yaw channel and the instruction value of the mathematical rudder deflection angle of the current pitch channel. The physical rudder angle command value of the current yaw channel is obtained based on the command value of the physical rudder angle of the current yaw channel and the command value of the physical rudder angle of the current pitch channel; the mathematical rudder angle of the current yaw channel and the mathematical rudder angle of the current pitch channel are generated based on the physical rudder angle of the current yaw channel and the physical rudder angle of the current pitch channel, and sent to the controlled object to obtain the output state of the controlled object at the current moment; based on the command value of the quasi-angle of attack at the current moment, the command value of the quasi-sideslip angle and the output state at the current moment, the current state of the controlled object is generated and sent to the trained reinforcement learning network model. The present invention takes saturation nonlinearity into account when obtaining the physical rudder angle, can automatically adjust the attitude control parameters, can improve the acquisition efficiency and accuracy of the attitude control parameters, and can achieve higher control accuracy.

[0015] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flow chart of a method for strengthening learning and controlling the attitude of a spinning aircraft under saturated nonlinearity provided by an embodiment of the present invention;

[0018] Figure 2 A structural block diagram of a saturated nonlinear attitude reinforcement learning control system for a spinning aircraft provided by an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the evaluation network structure;

[0020] Figure 4 It is a structural diagram of the action network;

[0021] Figures 5 to 10 Schematic diagram of the experimental results. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0024] It should be noted that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0025] The embodiment of the present invention provides a method for strengthening learning of attitude control of a spinning aircraft under saturated nonlinearity, and the method may include: Figure 1 The following steps are shown:

[0026] S100, based on the current attitude control parameters, the output state of the controlled object and the command values ​​of the quasi-angle of attack and the quasi-sideslip angle of the controlled object received at the current moment, obtain the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel; wherein the controlled object is a dynamic model of the spinning aircraft, and the output state of the controlled object includes the quasi-angle of attack, the quasi-sideslip angle, the yaw angular velocity and the pitch angular velocity.

[0027] In the embodiment of the present invention, the dynamic model of the spinning aircraft satisfies the following conditions:

[0028]

[0029]

[0030]

[0031]

[0032] Among them, m, V, and ψ V are mass, velocity, ballistic inclination and ballistic deviation, respectively. and They are V, and ψ V The first-order derivative of , γ, ψ and θ represent the roll angle, yaw angle and pitch angle respectively; α, β and γ V are the quasi-attack angle, quasi-sideslip angle and quasi-speed roll angle respectively. is the first-order derivative of θ, is the first-order derivative of γ; ω x ,ω y and ω z are the roll angular velocity, yaw angular velocity and pitch angular velocity respectively, and They are ω x ,ω y and ω z The first derivative of x , J y and J z are the moments of inertia of the roll channel, yaw channel and pitch channel respectively; P, X, Y, Z are thrust, drag, lift and lateral force respectively, M x 、M y and M z They are rolling moment, yaw moment and pitch moment respectively, q is the flight pressure, q = ρV 2 / 2, ρ is the atmospheric density. S is the reference area of ​​the spinning vehicle; c1 and c2 are the longitudinal and lateral reference lengths of the spinning vehicle respectively; c x (γ), c y (γ) and c z (γ) are the drag coefficient, lift coefficient and lateral force coefficient respectively. x (γ), c y (γ) and c z (γ) is also referred to as c x 、c y and c z . m x (γ), m y (γ) and m z (γ) are the rolling moment coefficient, yaw moment coefficient and pitch moment coefficient respectively, below, m x (γ), m y (γ) and m z (γ) is also referred to as mx 、m y and m z The variable corresponding to the “·” in the coefficient brackets indicates the variable that affects the coefficient. Ma is the Mach number, Ma=V / V s , V s is the speed of sound. yaw and δ pitch They are the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel respectively.

[0033] In the embodiment of the present invention, since the autogyroscope has no power when performing attitude control, P=0.

[0034] Based on equations (1)-(4), the small perturbation model of the pitch and yaw directions of the spinning aircraft can be derived as follows:

[0035]

[0036] in, d α ,d β Respectively indicate the impact The error factor of the other parameters is expressed as:

[0037]

[0038] in, represents the partial derivative of the aerodynamic coefficient γ with respect to the variable *.

[0039] In an embodiment of the present invention, the attitude control parameters may include a control gain of a quasi-attack angle, a control gain of a quasi-sideslip angle, a pitch angular velocity feedback gain, and a yaw angular velocity feedback gain.

[0040] Based on equation (5), the kinetic equations of α and β can be rewritten as:

[0041]

[0042] in:

[0043]

[0044] Similarly, based on formula (5), we can deduce ω z and ω y The kinetic equation is:

[0045]

[0046] in, and The control instructions for the mathematical rudder angles of the pitch channel and the yaw channel are represented by the command values, respectively, z and f yare the total disturbance factors of the pitch channel and the yaw channel respectively, and their expressions are:

[0047]

[0048] In the embodiment of the present invention, the total disturbance factor includes coupling effects such as inertial coupling, aerodynamic coupling caused by the Magnus effect, and control errors of the mathematical rudder angle. Combining equations (9) and (10), the derivative of equation (7) with respect to time can be obtained:

[0049]

[0050] Because ω z and ω y It can be directly measured by the rate gyro, so it is used for inner loop feedback to increase the damping coefficient. and Can be designed as:

[0051]

[0052] in, and are the pitch angular velocity feedback gain and the yaw angular velocity feedback gain, and is the virtual control signal that needs further design. At this time, equation (11) becomes:

[0053]

[0054] After the damping is enhanced, the virtual control signal ( and ) to α and β can be approximated as a first-order object, so Equation (13) can be written as:

[0055]

[0056] in, and represents the input gain after the model is reduced in order, that is, is the input gain of the pitch channel, is the input gain of the yaw channel. α and t β is the total disturbance factor including model uncertainty, coupling effect and model reduction, which are the total disturbance factor of α and the total disturbance factor of β respectively. and The calculation process is the same as Take as an example to illustrate. Ignoring the coupling effect, error factor and control error of mathematical rudder angle in equation (5), the small disturbance model in the pitch direction can be obtained as:

[0057]

[0058] Substituting equation (12) into the above equation and performing Laplace transformation, we can obtain Transfer function to α for:

[0059]

[0060] Where e is a complex variable. According to formula (15), we can calculate The two extreme points p1 and p2 (p1<p2) of In order to compensate for the total disturbance factor, the following nonlinear extended state observer is designed based on equation (14):

[0061]

[0062] Among them, z α,1 and z α,2 are for α and t respectively α The estimate of z β,1 and z β,2 are β and t respectively. β The estimate of α,o and l β,o (o=1,2) represents the gain of the observer, which can be designed as:

[0063] l α,1 =2ω α ,l α,2 =ω α 2 ,l β,1 =2ω β ,l β,2 =ω β 2

[0064] Among them, ω α and ω β Represents the bandwidth of the observer, which can be an empirical value. The purpose of introducing the function fal(γ) is to achieve "large error with small gain, small error with large gain", so as to avoid saturation of the control signal. Its specific form is:

[0065]

[0066]

[0067] Among them, λ α and λ β are the switching thresholds of the quasi-angle of attack and quasi-sideslip angle estimation errors, which can be empirical values. α and δ βare the exponents of the estimation errors of the quasi-angle of attack and quasi-sideslip angle, respectively, and can be empirical values. Then, the virtual control signal can be written as follows:

[0068]

[0069] Among them, α c and β c is the control command of α and β, that is, the command value of α and β, K α and K β are the control gains of α and β respectively. Combining equations (12) and (17), the final mathematical command of the rudder angle can be obtained as:

[0070]

[0071] Among them, K α , K β and The optimization is performed using a reinforcement learning algorithm. That is, in S100, the command values ​​of the mathematical steering angle of the yaw channel and the mathematical steering angle of the pitch channel satisfy equation (18).

[0072] In an embodiment of the present invention, the extended observer estimates and compensates for the total disturbance including the saturation nonlinearity of the servo, and when the control parameters are optimized, the attitude limit cycle suppression is reflected in the objective function of the model, so that the optimized control parameters can suppress the attitude limit cycle to the greatest extent.

[0073] S200, respectively decompose the instruction value of the mathematical rudder angle of the current yaw channel and the instruction value of the mathematical rudder angle of the current pitch channel to obtain the instruction value of the physical rudder angle of the current yaw channel and the instruction value of the physical rudder angle of the current pitch channel.

[0074] In S200, the command values ​​of the physical rudder deflection angle of the yaw channel and the physical rudder deflection angle of the pitch channel meet the following conditions:

[0075]

[0076] in, is the command value of the physical rudder deflection angle of the yaw channel, is the command value of the physical rudder deflection angle of the pitch channel, σ is the intermediate value,

[0077] Furthermore, in the embodiment of the present invention, if the amplitude of the command value of the physical rudder deflection angle is Exceeding the maximum amplitude u of the servo δ , then the command value of the mathematical rudder deflection angle needs to be readjusted as follows:

[0078]

[0079] S300, obtaining the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel based on the instruction value of the physical rudder deflection angle of the current yaw channel and the instruction value of the physical rudder deflection angle of the current pitch channel.

[0080] Furthermore, S300 specifically includes:

[0081] S301, obtain rudder angle tracking error And obtain the rudder angle tracking error of the pitch channel δ y is the physical rudder deflection angle of the yaw channel, δ z It is the physical rudder deflection angle of the pitch channel.

[0082] In the embodiment of the present invention, the physical rudder deflection angle for calculating the tracking error in S301 is the physical rudder deflection angle obtained at the previous moment.

[0083] S302, based on and the maximum speed of the servo Get the rate limit output value u i , where i is x or y, u i The following conditions must be met:

[0084] That is, if set up if set up if set up

[0085] S303, based on u i and linear models Get the linear output value r i ,in, and r i The first and second derivatives of i is the time constant, b i The time constant and steady-state gain can be obtained by conducting experiments on the servo, measuring the input and output data of the servo, and fitting the data.

[0086] S304, based on r i and the maximum amplitude u of the servo δ , get δ i , δ i The following conditions must be met:

[0087]

[0088] S400, based on the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel, generate the mathematical rudder deflection angle of the current yaw channel and the mathematical rudder deflection angle of the current pitch channel, and send them to the controlled object to obtain the output state of the controlled object at the current moment.

[0089] Further, in S400, the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel meet the following conditions:

[0090] δ yaw =δ y cosγ-δ z sinγ,

[0091] δ pitch =δ y sinγ+δ z cosγ.

[0092] It should be noted that for actual flight tests, the autopilot and the physical servo are both actual hardware devices. The physical servo drives the rudder surface to deflect, which will directly change the attitude of the aircraft, so there is no need to perform the physical rudder deflection angle synthesis process at this time. For mathematical simulation, the autopilot and the physical servo are both represented by mathematical models. Because the input of the autopilot dynamics model is the mathematical rudder deflection angle, the physical rudder deflection angle needs to be synthesized according to the above formula.

[0093] S500, based on the command value of the quasi-angle of attack, the command value of the quasi-sideslip angle and the output state at the current moment, generate the state of the controlled object at the current moment as the current state of the controlled object; the state of the controlled object includes the control error of the quasi-angle of attack, the control error of the quasi-yaw angle, the yaw angular velocity and the pitch angular velocity, the control error of the quasi-angle of attack is the difference between the command value of the quasi-angle of attack and the quasi-angle of attack, and the control error of the quasi-sideslip angle is the difference between the command value of the quasi-sideslip angle and the quasi-sideslip angle.

[0094] S600, sending the current state of the controlled object to the trained reinforcement learning network model to obtain the current posture control parameters; executing S100.

[0095] In an embodiment of the present invention, when the trained reinforcement learning network model receives the current state of the controlled object, it obtains the corresponding action, that is, the posture control parameter, through the action network.

[0096] Furthermore, in an embodiment of the present invention, the reinforcement learning network model may include a first evaluation network, a second evaluation network, a first target evaluation network, a second target evaluation network, an action network, and a target action network.

[0097] Furthermore, the trained reinforcement learning network model can be obtained through the following steps:

[0098] S1, initialize the parameters of the first evaluation network, the second evaluation network, the first target evaluation network, the second target evaluation network, the action network and the target action network, and initialize the experience replay pool; set the training round counter p=1; the experience replay pool stores a plurality of sample data, each sample data is a four-tuple information (s, a, r, s′), where s represents the state, s=(e α ,e β ,ω z ,ω y ), e α is the control error of the quasi-angle of attack, e α =α c -α, α c is the quasi-angle of attack command value, α is the quasi-angle of attack, e β is the control error of the quasi-sideslip angle, e β =β c -β,β c is the quasi-sideslip angle command value, β is the quasi-sideslip angle, ω y is the yaw angular velocity, ω z is the pitch angular velocity, a represents the action, a=(K,K d ), K and K d They are all posture control parameters, r represents the reward obtained by selecting action a, and s′ represents the state obtained after taking action a at time s.

[0099] In the embodiment of the present invention, the network parameters and the experience replay pool may be generated in a random manner.

[0100] In the embodiment of the present invention, since the dynamics of the pitch channel and the yaw channel are similar, the control parameters of the two channels are set to be the same, that is, K α =K β =K, Setting the dimension of the action space of a to 2 can improve the convergence speed of the model.

[0101] In the embodiment of the present invention, based on the following considerations: (1) the control accuracy of the angle of attack and the sideslip angle is reflected by the angle control error; (2) the limit cycle oscillation of the attitude angle is suppressed, which is reflected by the angular velocity; (3) the control signal saturation is avoided, which is reflected by the control command of the mathematical rudder deflection angle, i.e., the command value, and the set return r satisfies the following conditions:

[0102]

[0103]

[0104]

[0105] Among them, c e 、cω and c u are the coefficients of return, is the mathematical rudder angle command value of the yaw channel, is the mathematical rudder angle command value of the pitch channel, tanh() is the hyperbolic tangent function, R1 is the penalty value when the control error is greater than the first set error threshold e1, and R2 is the reward value when the control error is less than the second set error threshold e2. The smaller the attitude angle control error, the smaller the angular velocity, and the smaller the mathematical rudder angle command value, the greater the reward. In the embodiment of the present invention, e1 and e2 can be empirical values.

[0106] S2, if p>p0 or the average value of the rewards of p1 consecutive training rounds in p training rounds R>R0, take the current reinforcement learning network model as the trained reinforcement learning network model; if p≤p0 and R≤R0, set the training step counter t=1 and execute S3; p0 is the preset training round threshold, R0 is the preset reward threshold, p0 and R0 can be set based on actual needs.

[0107] In the embodiment of the present invention, the last round number of p1 consecutive training rounds is p, that is, the p1 consecutive training rounds are the p-p1+1th round, the p-p1+2th round, ..., the pth round. p1 can be an experience value.

[0108] S3, if t≤t0, execute S4, otherwise, set p=p+1 and execute S2; t0 is a preset time step threshold, which can be an empirical value.

[0109] In the embodiment of the present invention, the time period of each time step may be the same, which is ΔT, and may be an empirical value.

[0110] S4, the current state of the controlled object s c Input into the current action network to get the selected action a c .

[0111] S5, based on a c Obtain the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel, and send them to the controlled object to obtain the output state of the controlled object, and obtain the corresponding feedback r based on the output state of the controlled object c and the new state s c ′, forming the corresponding four-tuple information (s c , a c , r c ,s c ′) and added to the current experience replay pool.

[0112] S6, sample N from the current experience replay pool using the priority experience replay method sFour-tuple information is used as training sample data. Each training sample data (s m ,a m ,r m ,s′ m ) is input into the current target action network to obtain the corresponding optimal action a' m , m ranges from 1 to N s .

[0113] In the embodiment of the present invention, a′ m The following conditions must be met:

[0114]

[0115] Among them, ε′ represents random noise, which obeys normal distribution, has a mean of 0 and a variance of σ′. clip() is the clip function used to convert the input value π φ′ (s′ m )+ε′ is limited to the interval [a min ,a max ], a min and a max Represent the lower and upper bounds of the action, π φ′ (s′ m ) means to convert s′ m Input the output result obtained in the target network.

[0116] S7, the state s of the current node in each training sample data m and the corresponding action a m Input them into the current first evaluation network respectively, obtain the corresponding first evaluation network output result and second evaluation network output result, and convert the s′ corresponding to each quadruple information m and a′ m The first target evaluation network and the second target evaluation network are input into the current network respectively to obtain the corresponding output results of the first target evaluation network and the second target evaluation network.

[0117] S8, based on the output results of the first evaluation network, the output results of the second evaluation network, the output results of the first target evaluation network and the output results of the second target evaluation network of each training sample data, update the parameters of the current first evaluation network and the second evaluation network; if (t / d) is an integer, use the parameters of the current first evaluation network to update the parameters of the current action network, and use the parameters of the current evaluation network to update the parameters of the current target evaluation network and use the parameters of the updated action network to update the parameters of the current target action network, and execute S9; if (t / d) is not an integer, execute S9.

[0118] Since accurate estimation of the value function is a prerequisite for reasonable updating of control parameters, updating the action network after the evaluation network is fully trained can improve model stability.

[0119] S9, set t=t+1, and execute S3.

[0120] Furthermore, in the embodiment of the present invention, the parameters of the evaluation network j are updated as follows:

[0121]

[0122] Where j = 1 or 2, θ j (k+1) is the parameter of evaluation network j at the k+1th iteration, θ j (k) is the parameter of the evaluation network at the kth iteration, and the evaluation network j is the first evaluation network or the second evaluation network; H k The Hessian matrix corresponding to the kth iteration is specifically p θ ×p θ The Hessian matrix, p θ is the size of the parameters of the evaluation network. m is the intermediate variable, γ d is the discount factor, is the output result of the first evaluation network, is the output result of the first target evaluation network, is the output result of the second evaluation network, is the output result of the second target evaluation network, J c To evaluate the objective function of the network, For J c Regarding the parameter θ of the evaluation network j j The gradient of , λ is the damping factor. Among them, for About θ i The gradient of It can be calculated by existing techniques such as back propagation.

[0123] The parameters of the target evaluation network j are updated as follows:

[0124] θ j ′=τθ j +(1-τ)θ j ′;

[0125] θ j ′ is the parameter of the target evaluation network j, the target evaluation network j is the first target evaluation network or the second target evaluation network, and τ is the weighting coefficient.

[0126] Furthermore, in an embodiment of the present invention, the parameters of the action network are updated as follows:

[0127] φ=φ+χ a ▽ φ J a ;

[0128] Among them, φ is the parameter of the action network, χ a represents the learning rate of the action network, ▽ φ J a Denotes the objective function J of the action network a With respect to the gradient of φ, is the output result of the first evaluation network.

[0129] in, express With respect to the gradient of a, ▽ φ π φ (s m ) represents π φ (s m )The gradient of φ can be calculated using existing techniques such as back propagation.

[0130] In this embodiment of the present invention, the parameters of the target action network are updated as follows:

[0131] φ′=τφ+(1-τ)φ′;

[0132] Among them, φ′ is the parameter of the target action network and τ is the weight coefficient.

[0133] Based on the same inventive concept, an embodiment of the present invention provides a saturated nonlinear spinning vehicle attitude reinforcement learning control system, such as Figure 2 As shown, the system includes: a mathematical rudder deflection angle command value generation module 1, a mathematical rudder deflection angle command value decomposition module 2, a physical rudder module 3, a physical rudder deflection angle synthesis module 4, a controlled object 5, a state generation module 6 and a control parameter optimization module 7. Among them, the controlled object 5 is a dynamic model of the spinning aircraft.

[0134] In an embodiment of the present invention, the mathematical rudder angle command value generation module 1 is used to obtain the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel based on the current attitude control parameters, the output state of the controlled object, and the command values ​​of the quasi-attack angle and the quasi-sideslip angle of the controlled object received at the current moment.

[0135] In the embodiment of the present invention, the instruction value of the mathematical rudder angle of the current yaw channel and the instruction value of the mathematical rudder angle of the current pitch channel can be obtained by the aforementioned formula (18).

[0136] The mathematical rudder angle command value decomposition module 2 is used to decompose the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel respectively, obtain the command value of the physical rudder angle of the current yaw channel and the command value of the physical rudder angle of the current pitch channel, and send them to the physical servo module.

[0137] In the embodiment of the present invention, the command values ​​of the physical rudder deflection angle of the yaw channel and the physical rudder deflection angle of the pitch channel meet the following conditions:

[0138]

[0139] in, is the command value of the physical rudder deflection angle of the yaw channel, is the command value of the physical rudder deflection angle of the pitch channel, σ is the intermediate value,

[0140] Furthermore, in the embodiment of the present invention, if the amplitude of the command value of the physical rudder deflection angle is Exceeding the maximum amplitude u of the servo δ , then the command value of the mathematical rudder deflection angle needs to be readjusted as follows:

[0141]

[0142] The physical servo module 3 is used to obtain the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel based on the command value of the physical rudder deflection angle of the current yaw channel and the command value of the physical rudder deflection angle of the current pitch channel, and send them to the physical rudder deflection angle synthesis module.

[0143] In the embodiment of the present invention, the physical servo module 3 may include a tracking error calculation module, a rate saturation module, a linear module, and an amplitude saturation module, wherein the tracking error calculation module is used to calculate the rudder angle tracking error based on the currently received command value of the physical rudder angle and the physical rudder angle, and send it to the rate saturation module, that is, to implement the content defined in S301. The rate saturation module is used to limit the rudder angle tracking error based on the rudder angle tracking error and the maximum rate of the servo, obtain the corresponding rate limit output value, and send it to the linear module, that is, to implement the content defined in S302. The linear module is used to perform linear processing on the received rate limit output value, obtain the corresponding linear output value, and send it to the amplitude saturation module, that is, to implement the content defined in S303. The amplitude saturation module is used to limit the linear output value based on the received linear output value and the maximum amplitude of the servo, obtain the corresponding amplitude limit output value as the physical rudder angle, and send it to the tracking error calculation module and the physical rudder angle synthesis module 4.

[0144] The physical rudder angle synthesis module 4 is used to generate the mathematical rudder angle of the current yaw channel and the mathematical rudder angle of the current pitch channel based on the physical rudder angle of the current yaw channel and the physical rudder angle of the current pitch channel, and send them to the controlled object.

[0145] In the embodiment of the present invention, the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel meet the following conditions:

[0146] δ yaw =δ y cosγ-δ z sinγ,

[0147] δ pitch =δ y sinγ+δ z cosγ.

[0148] The controlled object 5 is used to obtain the output state at the current moment based on the received mathematical rudder deflection angle of the current yaw channel and the mathematical rudder deflection angle of the current pitch channel, and send it to the state generation module and the mathematical rudder deflection angle command value generation module; the output state includes the quasi-angle of attack, the quasi-sideslip angle, the yaw angular velocity and the pitch angular velocity.

[0149] By inputting the mathematical rudder deflection angle of the current yaw channel and the mathematical rudder deflection angle of the current pitch channel into the controlled object, the corresponding output state can be obtained through calculation.

[0150] The state generation module 6 is used to generate the state of the controlled object at the current moment based on the command value of the quasi-angle of attack, the command value of the quasi-sideslip angle and the output state at the current moment, as the current state of the controlled object, and send it to the control parameter optimization module 7.

[0151] Among them, the state of the controlled object includes the control error of the quasi-attack angle, the control error of the quasi-yaw angle, the yaw angular velocity and the pitch angular velocity. The control error of the quasi-attack angle is the difference between the command value of the quasi-attack angle and the quasi-attack angle, and the control error of the quasi-sideslip angle is the difference between the command value of the quasi-sideslip angle and the quasi-sideslip angle.

[0152] The control parameter optimization module 7 is used to generate current attitude control parameters based on the received current state of the controlled object, and send the parameters to the mathematical rudder deflection angle command value generation module.

[0153] In the embodiment of the present invention, the control parameter optimization module 7 is provided with a trained reinforcement learning network model. In the embodiment of the present invention, when the trained reinforcement learning network model receives the current state of the controlled object, it obtains the corresponding action, i.e., the posture control parameter, through the action network. The structure and training process of the reinforcement learning network model can refer to the contents described in the above embodiment.

[0154] (Example)

[0155] 1. Model and algorithm parameters

[0156] (1) Aircraft model parameters:

[0157]

[0158]

[0159]

[0160] (2) Obtain relevant parameters of the physical rudder deflection angle:

[0161] u eδ =5,u δ =20, a z =a y =8882.6, b z =b y =180.

[0162] (3) Control parameters that do not need to be optimized in the attitude controller:

[0163] ω α =ω β =5,δ α =δ β =0.2,λ α =λ β =0.8,δ max =20,α c =1,β c =0.

[0164] (4) Reinforcement learning network model parameters:

[0165] In the embodiment of the present invention, the structures of the four evaluation networks are the same, such as Figure 3 As shown, the structures of the two action networks are the same, such as Figure 4 shown. Figure 3 and Figure 4 For detailed explanation of the parameters in , please refer to Table 1 and Table 2 below. Each evaluation network has 2369 learning parameters and each action network has 142 parameters. All parameters of the reinforcement learning algorithm are as follows:

[0166] c e =100, c ω =10, c u =10, e1=1, e2=0.5, a min =[0,0], a max =[50,10],Ns =128,σ′=[1,0.5],

[0167] γ d =0.997,λ=0.99,χ a =0.0001, d=3, τ=0.005, p0=3000, p1=20, R0=2000, ΔT=0.1, t0=200, σ=[1,0.5].

[0168] Table 1 Detailed information of the evaluation network structure

[0169]

[0170] Table 2 Detailed information of the action network structure

[0171]

[0172]

[0173] (II) Algorithm Results

[0174] The rewards during training are as follows Figure 5 As shown in Figure 2, the average return tends to stabilize after the 250th training round, indicating that the algorithm has achieved rapid convergence. Subsequently, the trained action network is used to control the attitude of the spinning aircraft. The simulation results are shown in Figure 2. Figures 6 to 10 shown. Figure 6 The optimized control parameters during the simulation are displayed. It can be seen that the control parameters can quickly jump to a stable value, ensuring the rapid response of the attitude angle and stability in steady state. Figure 7 and Figure 8 The command values ​​and actual response values ​​of the mathematical rudder deflection angle and the physical rudder deflection angle are shown. It can be seen from the figure that both the mathematical rudder deflection angle and the physical rudder deflection angle have large response errors, which are mainly caused by the saturation nonlinearity inside the servo. Figure 8 As shown in Figure 2, the control parameters optimized by reinforcement learning can still effectively track the command values ​​of the angle of attack and sideslip angle, and no attitude limit cycle phenomenon occurs. Fig. 9 As shown, the pitch and yaw angular rates fluctuate rapidly only in the first second, and then quickly stabilize.

[0175] An embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method described in the embodiment of the present invention.

[0176] The embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer instructions are used to execute the method described in the embodiment of the present invention.

[0177] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0178] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for attitude reinforcement learning control of a spinning aircraft under saturated nonlinearity, characterized in that: The method comprises the following steps: S100, based on the current attitude control parameter, the output state of the controlled object, and the command values ​​of the quasi-angle of attack and the quasi-sideslip angle of the controlled object received at the current moment, obtaining the command value of the mathematical rudder deflection angle of the current yaw channel and the command value of the mathematical rudder deflection angle of the current pitch channel; wherein the controlled object is a dynamic model of the autogyro, and the output state of the controlled object includes the quasi-angle of attack, the quasi-sideslip angle, the yaw angular velocity, and the pitch angular velocity; S200, respectively decomposing the instruction value of the mathematical rudder deflection angle of the current yaw channel and the instruction value of the mathematical rudder deflection angle of the current pitch channel to obtain the instruction value of the physical rudder deflection angle of the current yaw channel and the instruction value of the physical rudder deflection angle of the current pitch channel; S300, acquiring the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel based on the instruction value of the physical rudder deflection angle of the current yaw channel and the instruction value of the physical rudder deflection angle of the current pitch channel; S400, generating a mathematical rudder deflection angle of the current yaw channel and a mathematical rudder deflection angle of the current pitch channel based on the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel, and sending them to the controlled object to obtain an output state of the controlled object at the current moment; S500, based on the command value of the quasi-angle of attack, the command value of the quasi-sideslip angle and the output state at the current moment, generating a state of the controlled object at the current moment as the current state of the controlled object; the state of the controlled object includes a control error of the quasi-angle of attack, a control error of the quasi-yaw angle, a yaw angular velocity and a pitch angular velocity, the control error of the quasi-angle of attack being the difference between the command value of the quasi-angle of attack and the quasi-angle of attack, and the control error of the quasi-sideslip angle being the difference between the command value of the quasi-sideslip angle and the quasi-sideslip angle; S600, sending the current state of the controlled object to the trained reinforcement learning network model to obtain the current posture control parameters; executing S100.

2. The method according to claim 1, characterized in that The reinforcement learning network model includes a first evaluation network, a second evaluation network, a first target evaluation network, a second target evaluation network, an action network and a target action network; wherein the trained reinforcement learning network model is obtained by the following steps: S1, initialize the parameters of the first evaluation network, the second evaluation network, the first target evaluation network, the second target evaluation network, the action network and the target action network, and initialize the experience replay pool; set the training round counter p=1; the experience replay pool stores a plurality of sample data, each sample data is a four-tuple information (s, a, r, s′), where s represents the state, s=(e α ,e β ,ω z ,ω y ), e α is the control error of the quasi-angle of attack, e α =α c -α, α c is the quasi-angle of attack command value, α is the quasi-angle of attack, e β is the control error of the quasi-sideslip angle, e β =β c -β,β c is the quasi-sideslip angle command value, β is the quasi-sideslip angle, ω y is the yaw angular velocity, ω z is the pitch angular velocity, a represents the action, a=(K,K d ), K and K d are posture control parameters, r represents the reward obtained by selecting action a, and s′ represents the state obtained after taking action a at time s; S2, if p>p0 or the average value of the rewards of p1 consecutive training rounds in p training rounds R>R0, take the current reinforcement learning network model as the trained reinforcement learning network model; if p≤p0 and R≤R0, set the training step counter t=1 and execute S3; p0 is the preset training round threshold, R0 is the preset reward threshold; the last round number in p1 consecutive training rounds is p; S3, if t≤t0, execute S4, otherwise, set p=p+1 and execute S2; t0 is the preset time step threshold; S4, the current state of the controlled object s c Input into the current action network to get the selected action a c ; S5, based on a c Obtain the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel, and send them to the controlled object to obtain the output state of the controlled object, and obtain the corresponding feedback r based on the output state of the controlled object c and the new state s c ′, forming the corresponding four-tuple information (s c , a c , r c ,s c ′) and added to the current experience replay pool; S6, sample N from the current experience replay pool using the priority experience replay method s Four-tuple information is used as training sample data. Each training sample data (s m ,a m ,r m ,s′ m ) is the state s′ of the next node in m Input into the current target action network to obtain the corresponding optimal action a′ m , m ranges from 1 to N s ; S7, the state s of the current node in each training sample data m and the corresponding action a m Input them into the current first and second evaluation networks respectively, obtain the corresponding output results of the first evaluation network and the second evaluation network, and convert the s′ corresponding to each quadruple information into m and a′ m Input them into the current first target evaluation network and the second target evaluation network respectively, and obtain the corresponding output results of the first target evaluation network and the second target evaluation network; S8, based on the output result of the first evaluation network, the output result of the second evaluation network, the output result of the first target evaluation network and the output result of the second target evaluation network of each training sample data, update the parameters of the current first evaluation network and the second evaluation network; if (t / d) is an integer, update the parameters of the current action network using the parameters of the current first evaluation network, and update the parameters of the current target evaluation network using the parameters of the current evaluation network and update the parameters of the current target action network using the parameters of the updated action network, and execute S9; if (t / d) is not an integer, execute S9; S9, set t=t+1, and execute S3.

3. The method according to claim 2, characterized in that The return r satisfies the following conditions: Among them, c e 、c ω and c u are the coefficients of return, is the mathematical rudder angle command value of the yaw channel, is the mathematical rudder angle command value of the pitch channel, tanh() is the hyperbolic tangent function, R1 is the penalty value when the control error is greater than the first set error threshold e1, and R2 is the reward value when the control error is less than the second set error threshold e2.

4. The method according to claim 2, characterized in that: The parameters of the evaluation network j are updated as follows: Where j = 1 or 2, θ j (k+1) is the parameter of evaluation network j at the k+1th iteration, θ j (k) is the parameter of the evaluation network at the kth iteration, and the evaluation network j is the first evaluation network or the second evaluation network; H k is the Hessian matrix corresponding to the kth iteration, ψ m is the intermediate variable, γ d is the discount factor, is the output result of the first evaluation network, is the output result of the first target evaluation network, is the output result of the second evaluation network, is the output result of the second target evaluation network, J c To evaluate the objective function of the network, For J c Regarding the parameter θ of the evaluation network j j The gradient of , λ is the damping factor; The parameters of the target evaluation network j are updated as follows: i j ′=τθ j +(1-τ)θ j ′; θ j ′ is the parameter of the target evaluation network j, the target evaluation network j is the first target evaluation network or the second target evaluation network, and τ is the weighting coefficient.

5. The method according to claim 2, characterized in that: The parameters of the action network are updated as follows: Among them, φ is the parameter of the action network, χ a represents the learning rate of the action network, Represents the objective function J of the action network a With respect to the gradient of φ, is the output result of the first evaluation network; The parameters of the target action network are updated as follows: φ′=τφ+(1-τ)φ′; Among them, φ′ is the parameter of the target action network and τ is the weight coefficient.

6. The method according to claim 1, characterized in that The dynamic model of the spinning aircraft meets the following conditions: Among them, m, V, and ψ V are mass, velocity, ballistic inclination and ballistic deviation, respectively. and They are V, and ψ V The first-order derivatives of , γ, ψ and Represents the roll angle, yaw angle and pitch angle; α, β and γ V are the quasi-attack angle, quasi-sideslip angle and quasi-speed roll angle respectively. for The first derivative of is the first-order derivative of ψ, is the first-order derivative of γ; ω x ,ω y and ω z are the roll angular velocity, yaw angular velocity and pitch angular velocity respectively, and They are ω x ,ω y and ω z The first derivative of x , J y and J z are the moments of inertia of the roll channel, yaw channel and pitch channel respectively; P, X, Y, Z are thrust, drag, lift and lateral force respectively, M x 、M y and M z are the rolling moment, yaw moment and pitch moment respectively, q is the flight pressure; S is the reference area of ​​the spinning vehicle; c1 and c2 are the longitudinal and lateral reference lengths of the spinning vehicle respectively; c x (γ), c y (γ) and c z (γ) are drag coefficient, lift coefficient and lateral force coefficient, m x (γ), m y (γ) and m z (γ) are the rolling moment coefficient, yaw moment coefficient and pitch moment coefficient respectively, Ma is the Mach number, δ yaw and δ pitch They are the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel respectively.

7. The method according to claim 6, characterized in that The attitude control parameters include a control gain of a quasi-angle of attack, a control gain of a quasi-sideslip angle, a pitch angular velocity feedback gain, and a yaw angular velocity feedback gain; In S100, the command values ​​of the mathematical rudder angle of the yaw channel and the mathematical rudder angle of the pitch channel meet the following conditions: in, K is the command value of the mathematical rudder deflection angle of the pitch channel, α is the control gain of α, α c is the command value of α, z α,2 is the total disturbance factor t on α α The estimate, is the pitch angular velocity feedback gain, is the input gain of the pitch channel, K is the command value of the mathematical rudder angle of the yaw channel, β is the control gain of β, β c is the command value of β, z β,2 is the total disturbance factor t on β β The estimate, is the yaw rate feedback gain, is the input gain of the yaw channel.

8. The method according to claim 7, characterized in that In S200, the command values ​​of the physical rudder deflection angle of the yaw channel and the physical rudder deflection angle of the pitch channel meet the following conditions: in, is the command value of the physical rudder deflection angle of the yaw channel, is the command value of the physical rudder deflection angle of the pitch channel, σ is the intermediate value, S300 specifically includes: S301, obtaining the physical rudder angle tracking error of the yaw channel And obtain the physical rudder angle tracking error of the pitch channel δ y is the physical rudder deflection angle of the yaw channel, δ z is the physical rudder deflection angle of the pitch channel; S302, based on and the maximum speed of the servo Get the rate limit output value u i , where i is x or y, u i The following conditions must be met: S303, based on u i and linear models Get the linear output value r i ,in, and r i The first and second derivatives of i is the time constant, b i is the steady-state gain; S304, based on r i and the maximum amplitude u of the servo δ , get δ i , δ i The following conditions must be met:

9. The method according to claim 7, characterized in that: In S400, the mathematical rudder deflection angle of the yaw channel and the mathematical rudder deflection angle of the pitch channel meet the following conditions: d yaw =d y cosγ-δ z sing, d pitch =d y sinγ+δ z cosγ.

10. A saturated nonlinear attitude reinforcement learning control system for a spinning aircraft, characterized in that: The system comprises: a control parameter optimization module, a mathematical rudder deflection angle command value generation module, a mathematical rudder deflection angle command value decomposition module, a physical servo module, a physical rudder deflection angle synthesis module, a controlled object and a state generation module; the controlled object is a dynamic model of a spinning aircraft; The mathematical rudder angle command value generation module is used to obtain the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel based on the current attitude control parameters, the output state of the controlled object, and the command values ​​of the quasi-attack angle and the quasi-sideslip angle of the controlled object received at the current moment; the output state includes the quasi-attack angle, the quasi-sideslip angle, the yaw angular velocity, and the pitch angular velocity; The mathematical rudder angle command value decomposition module is used to decompose the command value of the mathematical rudder angle of the current yaw channel and the command value of the mathematical rudder angle of the current pitch channel respectively, obtain the command value of the physical rudder angle of the current yaw channel and the command value of the physical rudder angle of the current pitch channel, and send them to the physical steering gear module; The physical servo module is used to obtain the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel based on the instruction value of the physical rudder deflection angle of the current yaw channel and the instruction value of the physical rudder deflection angle of the current pitch channel, and send them to the physical rudder deflection angle synthesis module; The physical rudder deflection angle synthesis module is used to generate a mathematical rudder deflection angle of the current yaw channel and a mathematical rudder deflection angle of the current pitch channel based on the physical rudder deflection angle of the current yaw channel and the physical rudder deflection angle of the current pitch channel, and send them to the controlled object; The controlled object is used to obtain the output state at the current moment based on the received mathematical rudder deflection angle of the current yaw channel and the mathematical rudder deflection angle of the current pitch channel, and send the output state to the state generation module and the mathematical rudder deflection angle command value generation module; The state generating module is used to generate the state of the controlled object at the current moment based on the command value of the quasi-angle of attack, the command value of the quasi-sideslip angle and the output state at the current moment, as the current state of the controlled object, and send it to the control parameter optimizing module; the state of the controlled object includes the control error of the quasi-angle of attack, the control error of the quasi-yaw angle, the yaw angular velocity and the pitch angular velocity, the control error of the quasi-angle of attack is the difference between the command value of the quasi-angle of attack and the quasi-angle of attack, and the control error of the quasi-sideslip angle is the difference between the command value of the quasi-sideslip angle and the quasi-sideslip angle; The control parameter optimization module is used to generate current attitude control parameters based on the received current state of the controlled object, and send the parameters to the mathematical rudder deflection angle command value generation module.

Images