A diesel engine air system model predictive control method based on reinforcement learning

Abstract

A diesel engine air system self-learning model predictive control method based on disturbance observation, comprising the following steps: step 1, establishing a TVA-VGT-EGR diesel engine air system model, and calculating an intake manifold pressure p 22 , a pre-vortex pressure p3 and an EGR rate X EGR in real time; step 2, designing an extended state observer ESO, and taking f1, f2 and f3 output by the extended state observer ESO as error compensation items of a prediction model in step 3; step 3, designing a model predictive controller with a disturbance observer, namely an ESO-MPC controller; and step 4, adjusting a weight coefficient in a cost function J MPC in step 3 by using reinforcement learning Q-learning, and realizing a self-learning function of the algorithm. The application can realize better decoupling control and smaller calibration workload.

Description

Technical Field

[0001] This invention relates to the field of diesel engine air system control technology, and in particular to a model predictive control method for diesel engine air systems based on reinforcement learning. Background Technology

[0002] Diesel engine air system control is one of the most important functions of engine control. The quality of air system control directly determines the fuel consumption and emissions performance of a diesel engine. Currently, the most advanced diesel engines are equipped with intake throttle valves (TVA), variable geometry turbochargers (VGT), and exhaust gas recirculation (EGR) systems, making the air system highly nonlinear, with various parameters cross-coupled and actuators influencing each other. Precisely decoupling and controlling each actuator to ensure that each air path parameter reaches the expected target, thereby improving engine emissions performance and reducing fuel consumption, has become a key research challenge in the industry.

[0003] To address the control problem of diesel engine air systems, numerous scholars both domestically and internationally have conducted research on advanced control algorithms. In industry, the PID algorithm is commonly used, employing piecewise PID to control each parameter individually, which suffers from a complex calibration process. Research based on PID includes fuzzy PID and combinations of PID and feedforward algorithms (see: Y.Li, X.Zhou, Y.Hu and H.Chen, "Air path system control of turbocharged gasoline engine based on fuzzy PID," The 27th Chinese Control and Decision Conference (2015 CCDC), Qingdao, China, 2015, pp.941-946, doi:10.1109 / CCDC.2015.7162054). However, this method still cannot overcome the problem of a large calibration workload. Model-based control has become a widely studied solution in academia over the past decade. For example, Li Dinggen, He Chunmeng, and others proposed a model reference sliding mode control method for the VGT-EGR-TVA system, which improves robustness and transient performance compared to the traditional PID method. See the literature: Li Dinggen, He Chunmeng. Research on diesel engine air path control based on model reference sliding mode control [J]. Journal of Wuhan University of Technology, 2015, 37(10):90-97. However, the control accuracy of this method is highly dependent on the accuracy of the model, resulting in a large amount of computation. The modeling is very sophisticated, and the adaptability cannot be guaranteed. Data-driven control is an emerging control method in recent years. Its application in diesel engine air system control mainly utilizes machine learning to bypass the complex mechanism model construction process. For example, researchers at Honda Research Center used neural networks to design a high-precision nonlinear predictive model for diesel engine air systems. This method significantly improved the tracking performance of intake manifold pressure and EGR rate (see: Gokul S. Sankar, Rohan C. Shekhar, Chris Manzie, Takeshi Sano, Hayato Nakada, "Model predictive controller with averageemissions constraints for diesel airpath," Control Engineering Practice, Volume 90, 2019, Pages 182-189, ISSN 09670661, https: / / doi.org / 10.1016 / j.conengprac.2019.06.021). However, it requires extensive data collection in the early stages, and the control effect on other engine models cannot be guaranteed; its universality needs to be verified.

[0004] In summary, to improve the control quality of diesel engine air systems, a control algorithm that is simple to calibrate, requires less computation, and has a certain degree of self-adaptation and self-learning capabilities is needed. Summary of the Invention

[0005] The purpose of this invention is to address the problems of large workload, poor decoupling effect of the three valves, and poor adaptability in the existing technology of diesel engine air system TVA-VGT-EGR control, and to provide a model predictive control method for diesel engine air system based on reinforcement learning.

[0006] The technical solution adopted to achieve the purpose of this invention is:

[0007] A predictive control method for a diesel engine air system based on a self-learning model, comprising the following steps:

[0008] Step 1: Establish a model of the TVA-VGT-EGR diesel engine air system and calculate the intake manifold pressure p in real time. 22 vortex inlet pressure p3 and EGR rate X EGR ;

[0009] Step 2: Based on the dynamic equations of intake manifold pressure, turbocharger turbine inlet pressure, and EGR rate, an expansion state observer (ESO) is designed. The expansion state observer (ESO) includes a manifold pressure observer, a turbine inlet pressure observer, and an EGR rate observer. The ESO outputs f1, f2, and f3 as error compensation terms for the prediction model in Step 3.

[0010] Step 3: Combining the TVA-VGT-EGR diesel engine air system model and the Extended State Observer (ESO), design a model predictive controller with a disturbance observer, namely the ESO-MPC controller. The ESO-MPC controller includes: a predictive model and a cost function J. MPC The rolling optimization solver uses a predictive model consisting of a TVA-VGT-EGR diesel engine air system model and disturbance compensation (f1, f2, f3) from an extended state observer (ESO). The predictive model yields predictive iterative equations for future multiple steps, which are then substituted into the cost function for optimization. The first step of the control sequence is calculated, and the rolling optimization solver continuously performs rolling optimization, ultimately integrating the ESO-MPC output control quantity to obtain the control output A. TVA A VGT and A EGR Finally, the actuator opening information is calculated and output to the controlled object.

[0011] Step 4: Using reinforcement learning Q-learning, adjust the cost function J in step 3.MPC The weight coefficients in the algorithm enable the self-learning function of the algorithm. The Q-learning algorithm consists of four main components: state, action, reward, and state-action value function. Based on the current state of the system, the corresponding action is selected according to the value of the state-action value function, the corresponding reward is calculated, and the state-action value function is updated accordingly.

[0012] In the above technical solution, the TVA-VGT-EGR diesel engine air system model in step 1 is as follows:

[0013]

[0014] In equation (9):

[0015]

[0016]

[0017]

[0018] η vol For inflation efficiency, V d For cylinder volume, N Eng V is the engine speed. 22 p is the intake manifold volume. 21 T is the pressure before the TVA valve. 22 T represents the intake air temperature. 21 The temperature before the TVA valve is σ1, which is the coefficient to be calibrated, and p 22 σ2 is the manifold pressure, σ3 is the coefficient to be calibrated, p3 is the pressure before the vortex, R is the ideal gas constant, T3 is the temperature before the vortex, V3 is the exhaust manifold volume, and p4 is the pressure after the vortex. For fuel mass flow rate, A TVA A is the effective flow cross-sectional area of ​​the TVA valve. EGR Where A is the effective flow cross-sectional area of ​​the EGR valve, T3 is the inlet temperature, p3 is the inlet pressure, and A is the outlet pressure. VGT ρ is the effective flow cross-sectional area of ​​the VGT valve, β is the coefficient to be calibrated, and ρ2 is the density of the intake gas in the cylinder.

[0019] In the above technical solution, in step 2, the manifold pressure observer:

[0020]

[0021] Vortex-front pressure observer:

[0022]

[0023] EGR rate observer:

[0024]

[0025] In the formula These are the ESO estimates for boost pressure, turbine inlet pressure, and EGR rate, respectively. These represent the rates of change of ESO estimates for boost pressure, inlet pressure, and EGR rate, respectively. p 22 p3 and X EGR The ESO estimate of the channel perturbation, the observed estimate is used as the perturbation of the model. β1, β2, β3, β4, β5, and β6 are the observer gains; ω0 is the observer bandwidth.

[0026] In the above technical solution, in step 3, the prediction model is:

[0027]

[0028] In the formula: F is intended to compensate for errors caused by system simplification. For the rate of change of the TVA valve, For the rate of change of the VGT valve, The change rate of the EGR valve.

[0029] In the above technical solution, the cost function in step 4 is:

[0030] J MPC =(X k -X r ) T Q(X k -X r )+U k T RU k

[0031] Where X k X is a sequence of state variables, obtained through a prediction model; r The target state sequence; U k The control state sequence is the variable to be determined; Q is the tracking weight matrix; R is the actuator action weight matrix, including R1, R2, and R3, where R1 is the control weight of the TVA valve, R2 is the action weight coefficient of the VGT valve, R3 is the control weight of the EGR valve, T represents the transpose of the vector, and R is adjusted by reinforcement learning Q-learning in step 4.

[0032] In the above technical solution, in step 4, the state-action value function is:

[0033]

[0034] In the formula S t Represents the current state, S t+1 Represents the state at the next moment, A t In this study, R² (VGT valve action weight) represents the current action, α represents the learning efficiency, and y represents the discount factor. This indicates the highest-scoring action currently being performed.

[0035] In the above technical solution, in step 4,

[0036] J MPC =U k T (C T QC+R)U k +2X(k) T M T QCU k +2F k T D T QCU k -2X r T QCU k

[0037] Minimum J MPC The control sequence within the prediction domain can then be obtained by taking only the output of the first step of the control law, thus obtaining the vector U. k The first component is then subjected to rolling optimization calculations, and the final control quantity U output by ESO-MPC is... k The first component integral yields A TVA A VGT and A EGR Finally, the actuator opening information is calculated and output to the controlled object.

[0038] In the above technical solution, in step 4,

[0039] The state action value function is:

[0040] Q(S t A t )=Q(S t A t )+α[R t+1 +γmax a Q(S t+1 ,a)-Q(S t A t )]

[0041] In the formula S t Representing the current state, i.e., p3, S t+1 Represents the state at the next moment, A tThe current action (vgt) represents the valve action weight coefficient, α represents the learning efficiency, and R represents the learning efficiency. t+1 Represents the state reward value, γ represents the discount factor, This indicates the highest-scoring action currently being performed.

[0042] In the above technical solution, the state division rule in step 4 is as follows:

[0043]

[0044] In the formula, n is the state number, x is the current state, i.e., p3, x min x is the minimum value of the state within the current control cycle. max This represents the maximum value of the state within the current control cycle.

[0045] In the above technical solution, the action selection rules in step 4 are as follows:

[0046]

[0047] In the formula, η is a randomly generated number between 0 and 1, A is the action set; s represents the state, and a represents the action;

[0048]

[0049] In the formula, ∈ represents the exploration rate, and eps represents the current learning iteration number.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] 1. Compared with the PID algorithm commonly used in engineering, this algorithm can achieve better decoupled control in the control of diesel engine air system and requires less calibration work.

[0052] 2. This algorithm has a simple prediction model, employing a simplified ESO observation disturbance compensation model, which achieves basic control effects while requiring less computation than traditional MPC. It can be deployed on the ECU.

[0053] 3. By using reinforcement learning to update time-varying weight coefficients, the weight coefficients can be adjusted in real time to find the optimal parameters under different operating conditions, thus improving control performance. This method is more adaptable than the constant weight coefficients used in traditional MPC.

[0054] 4. By using reinforcement learning to learn the weight coefficients, the controller's adaptability can be improved to cope with changes in the system characteristics of the diesel engine air system caused by aging and other reasons. Attached Figure Description

[0055] Figure 1 It is the overall framework of the algorithm.

[0056] Figure 2 This is the Q-learning iterative process. Ref represents the target value, and episode represents the number of iterations, which are 1, 10, 20, 30, and 50 times respectively.

[0057] Figure 3 This is a comparison of the control effects of time-varying weight coefficients and constant weight coefficients in reinforcement learning. R² represents the learned weight coefficients, R² = RL represents the weight coefficients learned through reinforcement learning, and R² = 0.5 and 0.3 represent two fixed parameters. Detailed Implementation

[0058] The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0059] Example 1

[0060] A predictive control method for a diesel engine air system based on a self-learning model, comprising the following steps:

[0061] Step 1: Establish a model of the TVA-VGT-EGR diesel engine air system and calculate the intake manifold pressure p in real time. 22 vortex inlet pressure p3 and EGR rate X EGR ;

[0062] Step 2: Based on the dynamic equations of intake manifold pressure, turbocharger turbine inlet pressure, and EGR rate, an Extended State Observer (ESO) is designed. The Extended State Observer (ESO) includes a manifold pressure observer, a turbine inlet pressure observer, and an EGR rate observer. The ESO outputs f1, f2, and f3 as error compensation terms for the prediction model in Step 3.

[0063] Step 3: Combining the TVA-VGT-EGR diesel engine air system model and the Extended State Observer (ESO), design a model predictive controller with a disturbance observer, namely the ESO-MPC controller. The ESO-MPC controller includes: a predictive model and a cost function J. MPC The rolling optimization solver uses a predictive model consisting of a TVA-VGT-EGR diesel engine air system model and disturbance compensation (f1, f2, f3) from an extended state observer (ESO). The predictive model yields predictive iterative equations for future steps, which are then substituted into a cost function for optimization. The control sequence is calculated, and the first step control output is taken. This rolling optimization is continuously performed, ultimately integrating the ESO-MPC output control quantity to obtain the control output A. TVA A VGT and A EGRFinally, the actuator opening information is calculated and output to the controlled object;

[0064] Step 4: Using reinforcement learning Q-learning, adjust the cost function J in step 3. MPC The weight coefficient (R) in the algorithm enables the self-learning function of the algorithm. The Q-learning algorithm includes four main components: state, action, reward and state-action value function. Based on the current state of the system, the corresponding action is selected according to the value of the state-action value function, the corresponding reward is calculated, and the state-action value function is updated accordingly.

[0065] Example 2

[0066] In step 1, based on the various parts of the diesel engine air system, including the intake manifold, exhaust manifold, and turbocharger, the control volume is determined, and a dynamic model of the diesel engine air system is established according to the fundamental laws of thermodynamics.

[0067] The dynamic equation for the intake manifold pressure is:

[0068]

[0069] In the formula, p 22 This refers to the intake manifold pressure. Intake manifold pressure change rate; R is the ideal gas constant; T 22 V represents the intake air temperature. 22 This refers to the intake manifold volume; These are the compressor mass flow rate, EGR mass flow rate, and engine intake mass flow rate, respectively.

[0070] The dynamic equation for the turbine inlet pressure of the turbocharger is:

[0071]

[0072] In the formula, p3 is the pressure in front of the vortex. The rate of change of pressure in front of the vortex; R is the ideal gas constant; T3 is the temperature in front of the vortex; V3 is the exhaust manifold volume; Fuel mass flow rate; VGT mass flow rate; This refers to the EGR mass flow rate.

[0073] The flow rates in equations (1) and (2) are calculated as follows:

[0074]

[0075] In the formula, A is the compressor mass flow rate. TVA p is the effective flow cross-sectional area of ​​the TVA valve. 21 The pressure before the TVA valve; p22 R is the intake manifold pressure; R is the ideal gas constant; T is the intake manifold pressure. 21 σ1 is the temperature before the TVA valve; σ1 is the coefficient to be calibrated.

[0076]

[0077] In the formula, For EGR mass flow rate, A EGR p3 is the effective flow cross-sectional area of ​​the EGR valve; p3 is the pressure before the vortex; R is the ideal gas constant; T3 is the temperature before the vortex; p 22 σ is the intake manifold pressure; σ2 is the coefficient to be calibrated.

[0078]

[0079] In the formula, For VGT mass flow rate, A VGT ρ is the effective flow cross-sectional area of ​​the VGT valve; p3 is the pressure before the vortex; R is the ideal gas constant; T3 is the temperature before the vortex; p4 is the pressure after the vortex; and β is the coefficient to be calibrated.

[0080]

[0081] In the formula, η is the engine intake air mass flow rate. vol For inflation efficiency; p 22 V is the intake manifold pressure; d Cylinder volume; N Eng R is the engine speed; T is the ideal gas constant; 22 This refers to the intake manifold temperature.

[0082] The equation for the EGR rate is:

[0083]

[0084] In the formula, X EGR For EGR rate, m EGR For EGR quality, m Eng The mass of air entering the engine; m Eng =ρ2V 22 ρ2 is the density of the intake gas in the cylinder, V 22 This refers to the intake manifold volume.

[0085] Based on the equation for the EGR rate (7), differentiating both sides yields the dynamic equation for the EGR rate:

[0086]

[0087] The derivative of the EGR rate, This refers to the engine intake air mass flow rate. X is the EGR mass flow rate. EGR For EGR rate, m Eng The quality of air entering the engine.

[0088] Based on the above formula, the intake manifold pressure p 22 vortex inlet pressure p3 and EGR rate X EGR The simplified model of the TVA-VGT-EGR diesel engine air system is as follows:

[0089]

[0090] In equation (9):

[0091]

[0092]

[0093]

[0094] η vol For inflation efficiency, V d For cylinder volume, N Eng V is the engine speed. 22 p is the intake manifold volume. 21 T is the pressure before the TVA valve. 22 T represents the intake air temperature. 21 The temperature before the TVA valve is σ1, which is the coefficient to be calibrated, and p 22 σ2 is the manifold pressure, σ3 is the coefficient to be calibrated, p3 is the pressure before the vortex, R is the ideal gas constant, T3 is the temperature before the vortex, V3 is the exhaust manifold volume, and p4 is the pressure after the vortex. For fuel mass flow rate, A TVA A is the effective flow cross-sectional area of ​​the TVA valve. EGR Where A is the effective flow cross-sectional area of ​​the EGR valve, T3 is the inlet temperature, p3 is the inlet pressure, and A is the outlet pressure. VGT β is the effective flow cross-sectional area of ​​the VGT valve, p2 is the coefficient to be calibrated, and p2 is the density of the intake gas in the cylinder.

[0095] In step 2, the intake manifold pressure observer:

[0096]

[0097] Vortex-front pressure observer:

[0098]

[0099] EGR rate observer:

[0100]

[0101] In the formula These are the ESO estimates for boost pressure, turbine inlet pressure, and EGR rate, respectively. These represent the rates of change of ESO estimates for boost pressure, inlet pressure, and EGR rate, respectively. p 22 p3 and X EGR The ESO estimate of the channel perturbation, the observed estimate is used as the perturbation of the model. β1, β2, β3, β4, β5, and β6 are the observer gains; ω0 is the observer bandwidth.

[0102] In step 3, the prediction model:

[0103]

[0104] In the formula: F is intended to compensate for errors caused by system simplification. For the rate of change of the TVA valve, For the rate of change of the VGT valve, The change rate of the EGR valve.

[0105] The cost function is:

[0106] J MPC =(X k -X r ) T Q(X k -X r )+U k T RU k (14)

[0107] Where X k X is a sequence of state variables, obtained through a prediction model; r The target state sequence; U k The control state sequence is the variable to be determined; Q is the tracking weight matrix; R is the actuator action weight matrix, including R1, R2, and R3, where R1 is the control weight of the TVA valve, R2 is the action weight coefficient of the VGT valve, R3 is the control weight of the EGR valve, and T represents the transpose of the vector. R is adjusted through the enhanced chemical Q-learning in step 4.

[0108] Solving model predictive control requires discretizing the continuous state-space equations, i.e., discretizing the predictive model, to obtain:

[0109]

[0110] X(k+1)=(I+TA)X(k)+TBU(k)+TKF(k) (16)

[0111] Where T is the discrete time step, U(k) is the control quantity at time k, F(k) is the disturbance at time k, X(k) represents the state at time k, X(k+1) represents the state at time k+1, and so on; thus, the relationship between the state at time k+1 and the state at time k is obtained. For ease of calculation, let A... T = (I+TA), B T =TB,C T =TK, equation (16) is transformed into (17):

[0112] X(k+1)=A T X(k)+B T U(k)+C T F(k) (17)

[0113] Starting from time k, define X(k+1|k) as the state predicted at time k+1, and so on. Then define the state sequence matrix within the prediction step size: Control sequence matrix: Observation perturbation sequence matrix: N is the prediction step size, and X(k|k) = X(k).

[0114] Based on the relationship between time k+1 and time k in equation (17), we can obtain (18) by iterating within the control region.

[0115] X k =MX(k)+CU k +DF k (18)

[0116] in

[0117] Substituting equation (18) into the cost function (14) and discarding the constant term, we get:

[0118] J MPC =U k T (C T QC+R)U k +2X(k) T M T QCU k +2F k T D T QCU k -2X r T QCU k (19)

[0119] Minimum J MPC The control sequence within the prediction domain can then be obtained by taking only the output of the first step of the control law, thus obtaining the vector U. k The first component is then used for rolling optimization calculations. The final control quantity U output by ESO-MPC is... k The first component integral yields A TVA A VGT and A EGR Finally, the actuator opening information is calculated and output to the controlled object.

[0120] In step 4, the real-time feedback state variable p3 of the engine is discretized into 30 intervals. The action space is determined by selecting 10 weight values ​​based on debugging experience. This forms a 30*10 Q-table, and the state-action value function is:

[0121] Q(S t A t )=Q(S t A t )+α[R t+1 +γmax a Q(S t+1 ,a)-Q(S t A t (19)

[0122] In the formula S t Represents the current state, S t+1 Represents the state at the next moment, A t R² represents the current action in this study, i.e., the valve action weighting coefficient vgt; α represents the learning efficiency; R t+1 Represents the state reward value, γ represents the discount factor, This indicates the highest-scoring action currently being performed.

[0123] The state partitioning rules are as follows:

[0124]

[0125] In the formula, n is the state number, x is the current state, i.e., p3, x min x is the minimum value of the state within the current control cycle. max This represents the maximum value of the state within the current control cycle.

[0126] The action selection rules are as follows:

[0127]

[0128] Where s represents the state, a represents the action, η is a randomly generated number between 0 and 1, and A is the set of actions.

[0129]

[0130] In the formula, ∈ represents the exploration rate, and eps represents the current learning iteration number. Formula (22) can be understood as exploring more actions and possibilities in the early stage of learning, and increasing the greedy rate and reducing randomness in the later stage of learning, in order to select the optimal action as much as possible.

[0131] The reward function is set by assigning different reward values ​​to each discrete state by comparing it with the ideal control curve. A state closer to the ideal control curve receives a higher score, while a state further away receives a lower score. The ideal control curve is a state change curve formed by arranging the transient process based on the dynamic setting time constant of the system and the change in the target value using this time constant.

[0132]

[0133] In equation (23), Reward is the reward and err is the error. When the system runs to the learning region, it calculates the state region according to (20) and selects the action to be executed according to (21), updates the weight R2 at this time, updates the state of the controlled object, calculates the reward according to (23), and updates the Q table according to (19).

[0134] from Figure 2 It can be seen that the algorithm continuously updates the weight coefficients and iterates gradually to achieve a better control effect. Figure 3 It can be seen that the time-varying weight coefficients learned by the algorithm improve both overshoot and convergence time compared to the constant weight coefficients. This proves the feasibility of the proposed method.

[0135] The above description is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A self-learning model predictive control method for a diesel engine air system based on disturbance observation, characterized in that, Includes the following steps: Step 1: Establish a model of the TVA-VGT-EGR diesel engine air system and calculate the intake manifold pressure in real time. vortex inlet pressure and EGR rate ; Step 2: Based on the dynamic equations of the intake manifold pressure, the turbocharger turbine inlet pressure, and the EGR rate, an expansion state observer (ESO) is designed. The ESO includes an intake manifold pressure observer, a turbine inlet pressure observer, and an EGR rate observer. The ESO outputs... , , As an error compensation term for the prediction model in step 3; Step 3: Combining the TVA-VGT-EGR diesel engine air system model and the Extended State Observer (ESO), design a model predictive controller with a disturbance observer, namely the ESO-MPC controller. The ESO-MPC controller includes: a predictive model and a cost function. The rolling optimization solver predicts the model based on the output of the TVA-VGT-EGR diesel engine air system model and the extended state observer (ESO). , , The composition involves obtaining predictive iterative equations for future multiple steps through a predictive model, and then substituting these equations into the cost function. The optimization solution is performed, and the control sequence is calculated. The first step of the control output is taken, and the rolling optimization solution is continuously applied. Finally, the control output of the ESO-MPC controller is obtained by integrating the control output. , and Finally, the actuator opening information is calculated and output to the controlled object; Step 4: Use reinforcement learning Q-learning to adjust the cost function in step 3. The weight coefficients in the algorithm enable the self-learning function of the algorithm. The Q-learning algorithm consists of four main components: state, action, reward and state-action value function. Based on the current state of the system, the corresponding action is selected according to the value of the state-action value function, the corresponding reward is calculated, and the state-action value function is updated accordingly. In step 4, the cost function is: ,in The state variable sequence is obtained through a prediction model; The target state sequence; The control state sequence is the variable to be determined; Q is the tracking weight matrix; R is the actuator action weight matrix, including R1, R2, and R3, where R1 is... The control weights, R² is Valve action weighting coefficient, R3 is The control weight, Represents the transpose of a vector. The reinforcement learning Q-learning in step 4 is used for regulation; In step 4, the minimum The control state sequence within the prediction domain can then be obtained. By taking only the output of the first step of the control law, the vector can be obtained. The first component is then subjected to rolling optimization calculations, and finally the control quantity output by the ESO-MPC controller is determined. The first component integral is obtained , and Finally, the actuator opening information is calculated and output to the controlled object; In step 4, the state-action value function is: In the formula Represents the current state Represents the state at the next moment. This represents the current action, i.e., the VGT valve action weighting coefficient. , Represents learning efficiency, Indicates discount factor, This indicates the highest-scoring action currently being performed; In step 4, the state division rules are as follows: （20） In the formula, n is the state number. The current state is , This represents the minimum value of the state within the current control cycle. This represents the maximum value of the state within the current control cycle; In step 4, the action selection rules are as follows: ； In the formula It is a randomly generated number between 0 and 1. It is an action set; s represents state, a represents action; In the formula eps represents the exploration rate, and eps represents the current learning iteration count.

2. The self-learning model predictive control method for diesel engine air systems based on disturbance observation as described in claim 1, characterized in that, In step 1, the air system model of the TVA-VGT-EGR diesel engine is as follows: ； In the formula: ； ； ； ； ； ； ； ； This represents the rate of change of intake manifold pressure. The rate of change of pressure in front of the vortex. The derivative of the EGR rate, For inflation efficiency, For cylinder volume, Engine speed, For intake manifold volume, The pressure before the TVA valve. Intake air temperature, Temperature before the TVA valve. For the coefficients to be calibrated, For intake manifold pressure, For the coefficients to be calibrated, Let R be the pressure before the vortex, and R be the ideal gas constant. For exhaust manifold volume, The pressure behind the vortex. For fuel mass flow rate, The effective flow cross-sectional area of ​​the TVA valve. The effective flow cross-sectional area of ​​the EGR valve. The temperature at the vortex inlet. The effective flow cross-sectional area of ​​the VGT valve. For the coefficients to be calibrated, This refers to the density of the intake gas in the cylinder.

3. The predictive control method for a diesel engine air system based on disturbance observation as described in claim 2, characterized in that, In step 2, the intake manifold pressure observer: ； Vortex-front pressure observer: ； EGR rate observer: ； In the formula , These are the estimated ESO values ​​for intake manifold pressure, turbine inlet pressure, and EGR rate, respectively. , These represent the rates of change of ESO estimates for intake manifold pressure, turbine inlet pressure, and EGR rate, respectively. , They are respectively and ESO estimates of channel perturbations and The ESO estimate of the channel perturbation is used as the perturbation of the model. , , ; , , , , For observer gain; This represents the observer bandwidth.

4. The self-learning model predictive control method for diesel engine air systems based on disturbance observation as described in claim 3, characterized in that, In step 3, the prediction model: ； In the formula: , , , , , To compensate for the errors caused by system simplification, The effective flow cross-sectional area change rate of the TVA valve. The effective flow cross-sectional area change rate of the VGT valve. This represents the rate of change of the effective flow cross-sectional area of ​​the EGR valve.

Images