Aircraft engine sliding mode rotating speed control method based on reinforcement learning optimization

Abstract

The invention discloses an aero-engine sliding mode rotating speed control method based on reinforcement learning optimization. The method comprises the following steps of S1, collecting original data of an aero-engine; s2, carrying out denoising smoothing processing on the collected original data; s3, performing system identification based on the processed fuel flow and the processed rotor speed to obtain a second-order discrete state space model of the aero-engine, and S4, designing a sliding mode controller based on the second-order discrete state space model of the aero-engine; s5, building a reinforcement learning environment according to a sliding mode control process of the aero-engine; s6, training by adopting an SAC algorithm in deep reinforcement learning, and finally obtaining an optimal sliding mode controller; and S7, the optimal sliding mode controller is used for controlling the rotating speed of the aero-engine needing to be controlled. The problem that selection of control parameters of an existing sliding mode controller depends on engineering experience and a cut-and-try method, and global optimum is difficult to achieve is solved.

Description

[Technical field]

[0001] The invention belongs to the technical field of aircraft engine control, and in particular relates to an aircraft engine sliding mode speed control method based on reinforcement learning optimization. [Background technology]

[0002] With the rapid development of the aviation industry, the requirements for aircraft engine performance are increasing day by day, especially in speed control, where high precision, high stability and fast response have become key indicators. Although traditional aircraft engine speed control methods, such as proportional-integral-derivative (PID) control, can meet basic needs to a certain extent, they often seem to be unable to cope with complex and changeable flight environments and strict performance indicators. In recent years, sliding mode control (SMC), as a robust nonlinear control method, has attracted widespread attention in the field of aircraft engine control. SMC achieves precise control of the system state by designing the sliding surface so that the system state reaches and remains on the sliding surface within a limited time. However, the performance of SMC depends largely on the selection of its control parameters, and the optimization of these parameters often relies on engineering experience and trial and error, which makes it difficult to achieve global optimization.

[0003] Reinforcement learning, as a machine learning paradigm that learns optimal strategies by interacting with the environment, has shown great potential in complex system control. In particular, the Soft Actor-Critic (SAC) algorithm, as an advanced algorithm in the field of reinforcement learning, can maintain the diversity and exploratory nature of strategies while maximizing cumulative rewards by introducing entropy regularization terms and automatically adjusting temperature parameters. It is very suitable for dealing with the complexity and uncertainty in aircraft engine speed control.

[0004] The performance of existing sliding mode controllers depends largely on the selection of their control parameters, and the optimization of these parameters often relies on engineering experience and trial and error, making it difficult to achieve global optimality. [Summary of the invention]

[0005] The purpose of the present invention is to provide an aircraft engine sliding mode speed control method based on reinforcement learning optimization to solve the problem that the selection of control parameters of the existing sliding mode controller depends on engineering experience and trial and error, and it is difficult to achieve the global optimum.

[0006] The present invention adopts the following technical solution: a sliding mode speed control method of an aircraft engine based on reinforcement learning optimization, comprising the following steps:

[0007] Step S1, performing an open-loop experiment on an aircraft engine that needs to be speed-controlled, and collecting raw data of the aircraft engine, wherein the raw data includes an input fuel flow rate as a control quantity and an output rotor raw speed as a controlled quantity;

[0008] Step S2, performing denoising and smoothing processing on the collected raw data to obtain a processed fuel flow rate and a processed rotor speed;

[0009] Step S3: performing system identification based on the processed fuel flow rate and the processed rotor speed to obtain a second-order discrete state space model of the aircraft engine, specifically:

[0010]

[0011] Wherein, u(k) is the input fuel flow at the current moment, y(k) is the output rotor speed at the current moment, x(k) is the state quantity of the second-order discrete state space model at the current moment, x(k+1) is the state quantity of the second-order discrete state space model at the next moment, A, B, C, and D are the system matrix, input matrix, output matrix, and direct transfer matrix of the second-order discrete state space model respectively;

[0012] Verify the accuracy of the second-order discrete state space model again. If the accuracy meets the requirement, proceed to step S4. If the accuracy does not meet the requirement, increase the amount of raw data collected and return to continue to execute steps S1 to S3 until the accuracy meets the requirement.

[0013] Step S4, designing a sliding mode controller based on a second-order discrete state space model of the aircraft engine;

[0014] Step S5, building a reinforcement learning environment according to the aircraft engine sliding mode control process, that is, defining the action space, state space and reward function in the reinforcement learning environment, and realizing the interactive simulation control process of the sliding mode controller and the second-order discrete state space model;

[0015] Step S6: Based on the reinforcement learning environment built in step S5, the SAC algorithm in deep reinforcement learning is used for training, and the parameters of the sliding mode controller are continuously optimized during the training process to finally obtain the optimal sliding mode controller;

[0016] Step S7: Use the optimal sliding mode controller to control the speed of the aircraft engine to be controlled.

[0017] Furthermore, the action space, state space and reward function of the reinforcement learning environment constructed in step S5 are specifically set as:

[0018] Action Space A t=[c, q, ε]; c, q and ε are parameters of the sliding mode controller; c is a sliding surface parameter, which affects the speed at which the system state approaches the sliding surface and the dynamic performance of the system sliding along the sliding surface; ε and q are positive constants, which are used to adjust the approach speed and the smoothness of the approach process, respectively, and both are greater than 0;

[0019] State space S t =[y_cur, error_cur, error_cumulation], where y_cur is the current aircraft engine rotor speed, error_cur is the deviation between the current aircraft engine rotor speed and the target speed, and error_cumulation is the cumulative speed deviation at the current moment;

[0020] Reward function R t =e -0.1|error_cur| , reward R t It is inversely proportional to the speed deviation error_cur at the current moment, that is, the smaller the speed deviation, the greater the reward.

[0021] Furthermore, the interactive simulation control process of the sliding mode controller and the second-order discrete state space model in the reinforcement learning environment constructed in step S5 is as follows:

[0022] Step S51, directly assign the values ​​of the system matrix, input matrix, output matrix, direct transfer matrix and initial state to corresponding variables; then obtain the action output by the strategy, and the output action is the parameter of the sliding mode controller in this control process;

[0023] Step S52, at each moment of the interactive simulation control process, the output at the current moment is obtained according to the sliding mode controller, and then the output rotor speed at the current moment and the state quantity of the second-order discrete state space model at the next moment are obtained according to the second-order discrete state space model;

[0024] Step S53, loop the process of step S52 until a complete interactive simulation control process is completed.

[0025] Furthermore, in step S6, there are five neural networks in the SAC algorithm using deep reinforcement learning, which are Actor network Critic1 Network Critic2 Network Target Critic1 Network TargetCritic2 Network

[0026] Furthermore, the training process of the SAC algorithm is:

[0027] S61, initializing five neural networks and the experience replay pool M of the SAC algorithm;

[0028] S62, initializing the reinforcement learning environment and the initial state s;

[0029] S63. Observe the current status and use Collect state-action-reward samples, i.e., {s, a, r, s'}, where s is the current state, a is the random action at the current moment, r is the reward at the current moment, and s' is the state at the next moment; after the collection is completed, save this sample to the experience replay pool D;

[0030] S64. Collect a batch of samples from the experience replay pool D. i ,a i ,r i ,s′ i}, there are N in this batch, and the actions here are based on The probability distribution is obtained by random sampling;

[0031] where s i ,a i ,r i ,s′ i The subscript i of refers to the i-th sample among these N samples;

[0032] S65, Use and Calculate the target value as shown in the following formula:

[0033]

[0034] Where γ is the discount factor, α is the entropy temperature coefficient, where j = 1 and 2, Target Critic1 network Target Critic2 Network

[0035] S66, Calculation and The network's loss function is updated using gradient descent and The parameters of the network, θ1 and θ2, are shown in the following formula:

[0036]

[0037] S67. Calculate the loss function of the Actor network and update the Actor network using the gradient descent method use or As shown in the following formula:

[0038]

[0039] S68. Calculate the entropy loss function and update the entropy using the gradient descent method, as shown in the following formula:

[0040] Where H0 is the target entropy set before training;

[0041] S69, update target Q network and As shown in the following formula:

[0042] θ′ i =ρθ i +(1-ρ)θ′ i , where ρ is the soft update coefficient;

[0043] S610: Return to S63 until the training is completed.

[0044] The beneficial effects of the present invention are as follows: the present invention uses the SAC algorithm in deep reinforcement learning to interactively train with the second-order discrete state space model of the aircraft engine. During the interactive training process, the parameters of the sliding mode controller are trained and optimized, and finally the optimal sliding mode controller is obtained; the optimal sliding mode controller obtained by training is used to control the speed of the aircraft engine, thereby improving the control accuracy and response speed. The present invention optimizes the cumbersome parameter adjustment process of the sliding mode control. The experimental results show that the aircraft engine speed sliding mode controller obtained through SAC algorithm training and optimization has a fast response speed and high control accuracy. The SAC algorithm is used to realize the intelligent optimization and verification of the control strategy, ensure the effective implementation and high robustness of the control method, and promote the intelligent development of aircraft engine control technology.

Brief Description of the Drawings

[0045] Figure 1 This is a flow chart of an aircraft engine sliding mode speed control method based on reinforcement learning optimization according to the present invention;

[0046] Figure 2 This is a comparison chart of the output of the original JT9D engine and the second-order discrete state space model obtained by identification;

[0047] Figure 3 It is the interactive simulation flow chart of sliding mode controller and second-order discrete state space model;

[0048] Figure 4 This is the structure diagram of the Actor neural network of the SAC algorithm;

[0049] Figure 5 This is the structure diagram of the SAC algorithm Critic neural network;

[0050] Figure 6 This is the cumulative reward distribution diagram for SAC training rounds;

[0051] Figure 7 This is the control effect diagram of the original JT9D aircraft engine by the optimal sliding mode controller obtained through training. [Specific implementation method]

[0052] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0053] A sliding mode speed control method for aircraft engines based on reinforcement learning optimization, such as Figure 1 As shown, the following steps are included:

[0054] Step S1, performing an open-loop experiment on an aircraft engine that needs to be speed-controlled, and collecting raw data of the aircraft engine, wherein the raw data includes an input fuel flow rate as a control quantity and an output rotor raw speed as a controlled quantity;

[0055] Step S2, performing denoising and smoothing processing on the collected raw data to obtain a processed fuel flow rate and a processed rotor speed;

[0056] Step S3: performing system identification based on the processed fuel flow rate and the processed rotor speed to obtain a second-order discrete state space model of the aircraft engine, specifically:

[0057]

[0058] Wherein, u(k) is the input fuel flow at the current moment, y(k) is the output rotor speed at the current moment, x(k) is the state quantity of the second-order discrete state space model at the current moment, x(k+1) is the state quantity of the second-order discrete state space model at the next moment, A, B, C, and D are the system matrix, input matrix, output matrix, and direct transfer matrix of the second-order discrete state space model respectively, where matrix A is a 2*2 matrix, matrix B is a 2*1 matrix, matrix C is a 2*2 matrix, and matrix D is a 2*1 matrix;

[0059] like Figure 2 As shown, the accuracy of the second-order discrete state space model is verified again. If the accuracy meets the requirements, step S4 is performed; if the accuracy does not meet the requirements, the amount of raw data collected is increased, and steps S1 to S3 are returned to continue to be performed until the accuracy meets the requirements. That is, after the accuracy does not meet the requirements, the newly collected raw data is subjected to denoising and smoothing processing and system identification is performed.

[0060] Among them, the accuracy of the second-order discrete state space model is verified, that is, the same fuel flow is input into the second-order discrete state space model of the aircraft engine obtained by system identification and the original aircraft engine, and the degree of fit of the output rotor speeds of the two is observed. If the degree of fit reaches 80%, the accuracy meets the requirements.

[0061] Step S4, designing a sliding mode controller based on a second-order discrete state space model of the aircraft engine;

[0062] Step S5: On the basis of step S3 and step S4, a reinforcement learning environment is built according to the aircraft engine sliding mode control process, that is, the action space A is defined in the reinforcement learning environment. t , state space S t , reward function R t , and realize the interactive simulation control process of the sliding mode controller and the second-order discrete state space model, such as Figure 3 As shown;

[0063] Step S6: Based on the reinforcement learning environment built in step S5, the SAC algorithm in deep reinforcement learning is used for training, and the parameters of the sliding mode controller are continuously optimized during the training process to finally obtain the optimal sliding mode controller;

[0064] Step S7: Use the optimal sliding mode controller to control the speed of the aircraft engine to be controlled.

[0065] In some embodiments, the action space A of the reinforcement learning environment constructed in step S5 t , state space S t and the reward function R t The specific settings are:

[0066] Action Space A t =[c, q, ε], which represents the action output by the reinforcement learning strategy; c is the sliding surface parameter, which affects the speed at which the system state approaches the sliding surface and the dynamic performance of the system sliding along the sliding surface; ε and q are positive constants, which are used to adjust the approach speed and the smoothness of the approach process, respectively, and both are greater than 0;

[0067] State space S t =[y_cur, error_cur, error_cumulation], where y_cur is the current aircraft engine rotor speed, error_cur is the deviation between the current aircraft engine rotor speed and the target speed, and error_cumulation is the cumulative speed deviation at the current moment;

[0068] Reward function R t =e -0.1*|error_cur| , reward R tIt is inversely proportional to the speed deviation error_cur at the current moment, that is, the smaller the speed deviation, the greater the reward.

[0069] In some embodiments, the interactive simulation control process of the sliding mode controller and the second-order discrete state space model in the reinforcement learning environment constructed in step S5 is specifically as follows:

[0070] Step S51, directly assign the values ​​of the system matrix, input matrix, output matrix, direct transfer matrix and initial state to corresponding variables; then obtain the action output by the strategy, which is the parameters c, q and ε of the sliding mode controller in this control process;

[0071] Step S52, at each moment of the interactive simulation control process, the output u(k) at the current moment k is obtained according to the sliding mode controller, and then the output rotor speed y(k) at the current moment k and the state quantity x(k+1) of the second-order discrete state space model at the next moment are obtained according to the second-order discrete state space model;

[0072] Step S53, loop the process of step S52 until a complete interactive simulation control process is completed.

[0073] In some embodiments, the sliding mode controller design in step S4 includes the following sub-steps:

[0074] S41. Assume that the target speed signal is r(k), and its rate of change is dr(k). Use the linear extrapolation method to predict r(k+1) and its rate of change dr(k+1) at time k+1:

[0075]

[0076] Define R(k) = [r(k); dr(k)];

[0077] S42. Let the sliding mode function be s(k), and its formula is as follows:

[0078] s(k)=G*(R(k)-x(k)),

[0079] At this time, s(k+1) can be expressed by the following formula:

[0080]

[0081] S43, select the sliding mode exponential reaching law. Different reaching laws of sliding mode control mainly affect the approaching speed of the system state on the sliding surface and the system's ability to suppress chattering and noise. The exponential reaching law not only ensures the rapidity of the approaching process, but also reduces the speed when reaching the sliding surface, which helps to suppress chattering. Its formula is as follows:

[0082]

[0083] Where s is the sliding surface function, is its derivative, which indicates the rate of change of the sliding surface function; ε and q are positive constants, which are used to adjust the approach speed and the smoothness of the approach process, respectively, and both are greater than 0; sgn1(s) is a function built based on the symbolic function, which is defined as:

[0084]

[0085] S44: According to the continuous sliding mode exponential reaching law of S43, the corresponding discrete reaching law is obtained, as shown in the following formula:

[0086]

[0087] Where T is the sampling time;

[0088] S45: The two s(k+1) formulas obtained from S42 and S44 are combined to obtain the sliding mode controller formula:

[0089] u(k)=(GB) -1 [GR(k+1)-GAx(k)-[T*[-ε*sgn1(s(k))-q*s(k)]+s(k)]],

[0090] Wherein, G is the parameter of the sliding mode function s(k) in S42, and G=[c, 1].

[0091] In some embodiments, the SAC algorithm in deep reinforcement learning in step S6 has five neural networks, which are Actor network Critic1 Network Critic2 Network Target Critic1 Network Target Critic2 Network The training process of the SAC algorithm is:

[0092] S61. Initialize the 5 neural networks and experience replay pool M of the SAC algorithm.

[0093] S62. Initialize the reinforcement learning environment and initial state s.

[0094] S63. Observe the current status and use Collect state-action-reward samples, that is, {s, a, r, s'}, where s is the current state, a is the random action at the current moment, r is the reward at the current moment, and s' is the state at the next moment. After the collection is completed, save this sample to the experience replay pool D.

[0095] S64. Collect a batch of samples from the experience replay pool D. i ,a i ,r i ,s′ i}, there are N in this batch, and the actions here are based on The probability distribution is randomly drawn. i ,a i ,r i ,s′ i The subscript i refers to the i-th sample among these N samples.

[0096] S65, Use and Calculate the target value as shown in the following formula:

[0097]

[0098] Where γ is the discount factor, which is used to calculate the discount when accumulating rewards to ensure that the sum of accumulated rewards is limited. α is the entropy temperature coefficient, which determines the degree of entropy maximization. This is the Target Critic1 network mentioned above Target Critic2 Network The formula is to find the minimum value of these two, and the subscript j is 1 and 2.

[0099] S66, Calculation and The network's loss function is updated using gradient descent and The parameters of the network, θ1 and θ2, are shown in the following formula:

[0100]

[0101] S67. Calculate the loss function of the Actor network and update the Actor network using the gradient descent method use or As shown in the following formula:

[0102]

[0103] S68. Calculate the entropy loss function and update the entropy using the gradient descent method. As shown in the following formula, where H0 is the target entropy set before training.

[0104]

[0105] S69, update target Q network and As shown in the following formula:

[0106] θ′ i =ρθ i +(1-ρ)θ′ i , where ρ is the soft update coefficient, which is used to control the update speed of the target network weights.

[0107] S610: Return to S63 until the training is completed.

[0108] Example

[0109] This embodiment uses the JT9D aircraft engine model based on T-MATS as the control object, and uses the aircraft engine sliding mode speed control method proposed in the present invention to achieve precise control of the low-pressure rotor speed of the JT9D aircraft engine. The specific implementation steps are as follows:

[0110] SS1. Conduct an open-loop experiment on the JT9D aircraft engine, set the original low-pressure rotor speed of the JT9D aircraft engine to 3667rpm, set its input fuel flow rate to a step signal, the initial value is 0.015kg / s, and the step rises to 0.020kg / s at 20s. The simulation time is 50s in total, and the simulation step is 0.001s. Collect the original data of the JT9D aircraft engine, that is, the input fuel flow rate and the original output rotor speed, and use the fuel flow rate as the control quantity and the original rotor speed as the controlled quantity;

[0111] SS2, denoising and smoothing the collected raw data to obtain the processed fuel flow and rotor speed of the JT9D aircraft engine;

[0112] SS3. Based on the processed fuel flow and rotor speed, the system is identified to obtain the second-order discrete state space model of the JT9D aircraft engine:

[0113]

[0114] After system identification, we get:

[0115] A=[0.999855555415616,-0.000402443549474426;-0.0000924334128923676,0.99977559432944 9], B=[0.001551116; 0.000918850718750047], C=[42903.7563216226,-4.61690338784296], D=0.

[0116] The initial state x(0)=[0.0855; 0.0431].

[0117] The step fuel flow signal in SS1 is input into the original JT9D aircraft engine model and the identified JT9D aircraft engine second-order discrete state space model, and the output rotor speed comparison diagram is shown in the attached figure. Figure 2 As shown. Figure 2 It can be seen that the second-order discrete state space model of the JT9D aircraft engine obtained by system identification has a good fit with the original JT9D aircraft engine model of 81.22%, which means that the model obtained by system identification can be used for subsequent sliding mode controller design.

[0118] SS4. Based on the second-order discrete state space model of the JT9D aircraft engine, the sliding mode controller is designed as shown in the following formula:

[0119] u(k)=(GB) -1 [GR(k+1)-GAx(k)-[T*[-ε*sgn1(s(k))-q*s(k)]+s(k)]],

[0120] Among them, G is the parameter of the sliding mode function in S42, and G=[c,1].

[0121] SS5. Use the gym library to build the reinforcement learning environment for the sliding mode control of the JT9D aircraft engine, set the action space, state space, and reward function in turn, and implement the interactive simulation control process of the sliding mode controller and the second-order discrete state space model.

[0122] Among them, the action space represents the action output by the reinforcement learning strategy, which is the three parameters of the sliding mode controller in SS4. In the state space, y_cur is the low-pressure rotor speed of the JT9D aircraft engine at the current moment, error_cur is the deviation between the low-pressure rotor speed of the JT9D aircraft engine at the current moment and the target low-pressure rotor speed, and error_cumulation is the cumulative low-pressure rotor speed deviation at the current moment. The reward R in the reward function t It is inversely proportional to the low-pressure rotor speed deviation error_cur at the current moment, that is, the smaller the speed deviation, the greater the reward.

[0123] In the interactive simulation control process of the sliding mode controller and the second-order discrete state space model in the reinforcement learning environment built in this example, the total time of an interactive simulation is set to 20s, the simulation step is 0.001s, and the maximum reward for each simulation step is 1. The maximum total reward for completing an interactive simulation is 20,000.

[0124] SS6, based on the environment built in SS5, uses the SAC algorithm in deep reinforcement learning based on Stable-Baselines3 for training. The Actor neural network structure of the SAC algorithm is set as follows: Figure 4As shown, the structure of the Critic neural network is as follows Figure 5 As shown in the figure. The learning rate is set to 3e-4, and the other hyperparameters are the default values ​​of Stable-Baselines3. The number of training rounds is set to 300, and the optimal model is saved according to the maximum cumulative reward of the round. During the training process, the parameters of the sliding mode controller are continuously optimized, and finally the optimal sliding mode controller is obtained.

[0125] The cumulative reward distribution diagram of SAC training rounds is as follows Figure 6 As shown, from Figure 6 It can be seen that the model reaches the optimal state when training for 229 rounds. At this time, the maximum round cumulative reward is 18962.28432, that is, the sliding mode controller parameters given by SAC are the optimal parameters. The optimal parameters are:

[0126] c=37178.88671875, q=72.9747314453125, ε=57.26795196533203

[0127] SS7. Use the optimal sliding mode controller to control the original JT9D aircraft engine, set the target rotor speed as a step signal, and the control effect is as follows Figure 7 The control effect of the optimal sliding mode controller obtained by training on the original JT9D aircraft engine is shown in the figure. Figure 7 It can be seen that when the target low-pressure rotor speed changes, the control method of the present invention can quickly control the low-pressure rotor speed of the JT9D aircraft engine to reach the target low-pressure rotor speed, that is, the adjustment time is very short, there is no overshoot, and the steady-state error is also very small. This shows that the sliding mode controller designed according to the method proposed in the present invention has excellent fast tracking ability and control accuracy.

[0128] In traditional sliding mode control methods, the performance of the control system is highly dependent on a series of carefully designed control parameters. The adjustment of these parameters usually requires engineers to have a deep control theory background and rich practical experience, and to find the optimal or near-optimal parameter combination through repeated trial and error and experimental verification. This process is not only time-consuming and labor-intensive, but also often difficult to ensure the robustness and stability of the final control system, especially when faced with complex and changeable system dynamics. Traditional sliding mode control methods often require complex parameter adjustment and trial and error processes. In contrast, the present invention introduces the SAC algorithm, an advanced reinforcement learning algorithm, to automatically complete the complex parameter adjustment process in traditional sliding mode control. The core of the present invention is to construct a reinforcement learning environment corresponding to the sliding mode control system, which can simulate the dynamic behavior of the system and allow the SAC algorithm to optimize the sliding mode controller parameters through trial and error learning, which greatly simplifies the design of the control system. Engineers only need to focus on the training and verification of the SAC algorithm to obtain the optimal control parameters, reducing the difficulty and cost of developing the control system.

[0129] Through deep learning and optimization of the SAC algorithm, the present invention can find the optimal SMC control parameters for a specific aircraft engine model, thereby achieving high-precision control of the rotation speed.

[0130] The reinforcement learning SAC algorithm used in the present invention has its own action noise. Specifically, the strategy of the SAC algorithm adds a certain amount of Gaussian noise to the output action when giving the action, that is, the three parameters of the sliding mode controller. The addition of this noise enables the algorithm to give a wider range of actions during the training process. This allows the sliding mode controller trained by the SAC algorithm to undergo extensive exploration during the training process and learn how to make reasonable responses under different conditions, so it has stronger robustness. Even if unforeseen disturbances are encountered during the actual operation, the control system can maintain stable performance and ensure the safe operation of the aircraft engine.

Claims

1. A sliding mode speed control method for an aircraft engine based on reinforcement learning optimization, characterized in that: The following steps are involved: Step S1, performing an open-loop experiment on an aircraft engine that needs to be speed-controlled, and collecting raw data of the aircraft engine, wherein the raw data includes an input fuel flow rate as a control quantity and an output rotor raw speed as a controlled quantity; Step S2, performing denoising and smoothing processing on the collected raw data to obtain a processed fuel flow rate and a processed rotor speed; Step S3: performing system identification based on the processed fuel flow rate and the processed rotor speed to obtain a second-order discrete state space model of the aircraft engine, specifically: Wherein, u(k) is the input fuel flow at the current moment, y(k) is the output rotor speed at the current moment, x(k) is the state quantity of the second-order discrete state space model at the current moment, x(k+1) is the state quantity of the second-order discrete state space model at the next moment, A, B, C, and D are the system matrix, input matrix, output matrix, and direct transfer matrix of the second-order discrete state space model respectively; Verify the accuracy of the second-order discrete state space model again. If the accuracy meets the requirement, proceed to step S4. If the accuracy does not meet the requirement, increase the amount of raw data collected and return to continue to execute steps S1 to S3 until the accuracy meets the requirement. Step S4, designing a sliding mode controller based on a second-order discrete state space model of the aircraft engine; Step S5, building a reinforcement learning environment according to the aircraft engine sliding mode control process, that is, defining the action space, state space and reward function in the reinforcement learning environment, and realizing the interactive simulation control process of the sliding mode controller and the second-order discrete state space model; Step S6: Based on the reinforcement learning environment built in step S5, the SAC algorithm in deep reinforcement learning is used for training, and the parameters of the sliding mode controller are continuously optimized during the training process to finally obtain the optimal sliding mode controller; Step S7: Use the optimal sliding mode controller to control the speed of the aircraft engine to be controlled.

2. The method for aircraft engine sliding mode speed control based on reinforcement learning optimization according to claim 1, characterized in that: The action space, state space and reward function of the reinforcement learning environment constructed in step S5 are specifically set as: Action Space A t =[c, q, ε]; c, q and E are parameters of the sliding mode controller; c is the sliding surface parameter, which affects the speed at which the system state approaches the sliding surface and the dynamic performance of the system sliding along the sliding surface; ε and q are positive constants, which are used to adjust the approach speed and the smoothness of the approach process, respectively, and both are greater than 0; State space S t =[y_cur,error_cur,error_cumulation], where y_cur is the current aircraft engine rotor speed, error_cur is the deviation between the current aircraft engine rotor speed and the target speed, and error_cumulation is the cumulative speed deviation at the current moment; Reward function R t =e -0.1*|error_cur| , reward R t It is inversely proportional to the speed deviation error_cur at the current moment, that is, the smaller the speed deviation, the greater the reward.

3. The method for aircraft engine sliding mode speed control based on reinforcement learning optimization according to claim 2 is characterized in that: The interactive simulation control process of the sliding mode controller and the second-order discrete state space model in the reinforcement learning environment constructed in step S5 is specifically as follows: Step S51, directly assign the values ​​of the system matrix, input matrix, output matrix, direct transfer matrix and initial state to corresponding variables; then obtain the action output by the strategy, and the output action is the parameter of the sliding mode controller in this control process; Step S52, at each moment of the interactive simulation control process, the output at the current moment is obtained according to the sliding mode controller, and then the output rotor speed at the current moment and the state quantity of the second-order discrete state space model at the next moment are obtained according to the second-order discrete state space model; Step S53, loop the process of step S52 until a complete interactive simulation control process is completed.

4. The method for aircraft engine sliding mode speed control based on reinforcement learning optimization according to claim 3 is characterized in that: The SAC algorithm in the deep reinforcement learning used in step S6 has five neural networks, which are Actor networks. Criticl Network Critic2 Network Target Critic1 Network Target Critic2 Network 5. The method for aircraft engine sliding mode speed control based on reinforcement learning optimization according to claim 4, characterized in that: The training process of the SAC algorithm is: S61, initializing five neural networks and the experience replay pool M of the SAC algorithm; S62, initializing the reinforcement learning environment and the initial state s; S63. Observe the current status and use Collect state-action-reward samples, i.e., {s, a, r, s′}, where s is the current state, a is the random action at the current moment, r is the reward at the current moment, and s′ is the state at the next moment; after the collection is completed, save this sample to the experience replay pool D; S64. Collect a batch of samples from the experience replay pool D. i , a i , r i , s′ i }, there are N in this batch, and the actions here are based on The probability distribution is obtained by random sampling; where s i , a i , r i , s′ i The subscript i of refers to the i-th sample among these N samples; S65, Use and Calculate the target value as shown in the following formula: Where γ is the discount factor, α is the entropy temperature coefficient, where j = 1 and 2, Target Critic1 network Target Critic2 Network S66, Calculation and The network's loss function is updated using gradient descent and The parameters of the network, θ1 and θ2, are shown in the following formula: S67. Calculate the loss function of the Actor network and update the Actor network using the gradient descent method use or As shown in the following formula: S68. Calculate the entropy loss function and update the entropy using the gradient descent method, as shown in the following formula: Where H0 is the target entropy set before training; S69, update target Q network and As shown in the following formula: θ′ i =ρθ i +(1-ρ)θ′ i , where ρ is the soft update coefficient; S610: Return to S63 until the training is completed.

Images