Multivariable cooperative control method for dual-fuel engine

Abstract

The invention relates to an intelligent control technology of an internal combustion engine, in particular to a multivariable cooperative control method of a dual-fuel engine, which realizes dynamic optimization matching of fuel injection and an air system through deep reinforcement learning and multivariable decoupling control technologies. According to the method, a dual-fuel engine multivariable cooperative control system based on deep reinforcement learning and dynamic decoupling is constructed, the dual-fuel engine multivariable cooperative control system is composed of an air system control layer, a multivariable decoupling layer and a fuel injection control layer, and the air system control layer controls the fuel injection control layer through a TD3-PID algorithm according to the vehicle speed, the vehicle speed and the vehicle speed. The rotating speed N, the torque T, the common rail pressure Prail, the fuel temperature Tfuel and the mixed fuel density rho mix are operated in real time to serve as state variables, data are collected and processed, and the dynamic adjustment amount of the electronic throttle valve is obtained; by means of a PSO-PID decoupling mechanism and a DDPG-PID intelligent control algorithm, decoupling regulation and control over the dual-fuel dynamic ratio, the EGR rate and the supercharging pressure according to working condition changes are achieved, the economical efficiency, the emission performance and the dynamic performance of the dual-fuel engine under the dynamic operation working condition are remarkably improved, and the method has the advantages of being high in self-adaptive capacity and high in robustness.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology for internal combustion engines. Specifically, it is a control method that integrates deep reinforcement learning technology and multivariable decoupling control technology to achieve dynamic optimization matching and multivariable decoupling coordinated control of the fuel injection system and the air system, and solve the problems of asynchronous response of the fuel and air systems under transient conditions of dual-fuel engines, high cost and long cycle of large-scale test calibration, and difficulty in achieving optimal engine performance across the entire operating range. Technical Background

[0002] Dual-fuel engines use two fuels, such as biodiesel and methanol, to convert chemical energy into mechanical energy through a rational fuel ratio and injection control, achieving both green, low-carbon, and energy-saving emission reduction characteristics. Dynamically adjusting the dual-fuel ratio based on operating conditions is an effective technical measure to improve engine power, economy, and emissions across the entire operating range. The fuel injection and air systems also require dynamic coordination during engine operation to address the issue of asynchronous responses between the fuel and air systems. However, existing dual-fuel engine air and injection system control relies on extensive testing and calibration, and the dual-fuel ratio cannot be changed during engine operation, resulting in high testing costs. Furthermore, a fixed fuel ratio fails to address the varying fuel requirements under high and low loads. At low loads, high combustion stability is required, so a high proportion of fuel with high autoignition properties is required. At high loads, high nitrogen oxide emissions are achieved, so the proportion of fuel with high latent heat of vaporization is increased to control combustion chamber temperature. A fixed fuel ratio makes it difficult to achieve optimal engine performance across the entire operating range. In particular, under complex and variable operating conditions, engine performance deteriorates dramatically, preventing the full potential of dual-fuel engines. Therefore, there is an urgent need to develop control technology that can dynamically match the ratio of dual fuels according to engine operating conditions, as well as multivariable decoupling collaborative control technology for dynamic matching of fuel and air systems, in order to solve problems such as uneven power output, deterioration of combustion and emission performance caused by the asynchronous response of the fuel and air systems under transient conditions, and to achieve optimization of power, economy and emission performance under dynamic engine conditions. Summary of the Invention

[0003] In response to the shortcomings and deficiencies in the existing technology, the present invention proposes a control method that integrates deep reinforcement learning and multivariable decoupling collaborative control to achieve dynamic optimization matching and multivariable collaborative control of the fuel and air system, so that the engine can achieve optimal performance in all aspects.

[0004] The present invention is achieved by the following measures:

[0005] A dynamic ratio dual-fuel engine multivariable cooperative control method is characterized by constructing a dual-fuel engine multivariable cooperative control system based on deep reinforcement learning and dynamic decoupling. The dual-fuel engine multivariable cooperative control system consists of an air system control layer, a multivariable decoupling layer and a fuel injection control layer. The air system control layer uses the TD3-PID algorithm to calculate the vehicle speed, real-time operating speed N, torque T, common rail pressure P, and the like. rail , fuel temperature T fuel and the mixed fuel density ρ mix As the state variable, data is collected and processed to calculate the dynamic adjustment amount of the electronic throttle;

[0006] The multivariable decoupling control layer uses the PSO algorithm to perform parameter decoupling tuning optimization on the PID controller parameters to achieve decoupling control of the EGR valve and the exhaust gas bypass valve.

[0007] The fuel injection control layer uses the DDPG-PID algorithm to calculate the engine fuel consumption m f , NOx emissions and in-cylinder gas content As a state variable, data is collected and processed, and a comprehensive reward function is calculated to obtain the optimal flow control valve opening for each fuel, and the ratio of the two fuels is accurately controlled through the actuator;

[0008] The present invention specifically comprises the following steps:

[0009] Step 1: Real-time collection of engine operating status data, including the fuel system's common rail pressure, fuel temperature, mixed fuel density, boost pressure, EGR rate, vehicle speed, rotational speed, and torque;

[0010] Step 2: The air system control layer adjusts the electronic throttle opening using the TD3-PID algorithm, and achieves dynamic decoupling control between the fuel system and the air system through the self-learning capability of the neural network;

[0011] Step 3: The multivariable decoupling control layer uses the PSO algorithm to perform parameter decoupling tuning and optimization calculations on the PID controller parameters of the EGR valve and the wastegate valve. The PID parameters are adjusted in real time based on the calculated values ​​to achieve multivariable decoupling control of the air system and optimize the boost pressure and EGR rate control.

[0012] Step 4: The fuel injection control layer uses the DDPG-PID intelligent control algorithm to dynamically optimize the dual-fuel injection ratio, synchronously match the intake volume, calculate the dual-fuel flow control valve adjustment amount, and convert it into an actual opening instruction;

[0013] Step 5: Integrate the control instructions of each subsystem based on the hierarchical collaborative architecture, and finally output a coordinated and consistent control signal to complete the entire process of multi-variable collaborative optimization control of the engine.

[0014] In step 2 of the present invention, the air system control layer uses the TD3-PID algorithm, and the specific steps are as follows:

[0015] Step 2-1: Collect real-time engine operating status data, including fuel consumption, NOx emissions, intake pressure, intake volume, EGR rate, and air-fuel ratio. Perform sliding average filtering and normalization on the collected data to provide accurate data support for subsequent control.

[0016] Step 2-2: Actor-Critic network learning: The actor network generates adjustments to the dual-fuel injection valves, while the critic network evaluates the value of each state-action pair to ensure synchronous matching of in-cylinder gas under instantaneous operating conditions.

[0017] Step 2-3: Solve the comprehensive reward function and fully optimize the performance of the system;

[0018] Step 2-4: Execute command conversion. The PID controller converts the adjustment value output by the Actor network into an actual valve opening command to achieve precise control of the dual-fuel injection process.

[0019] The present invention is directed to a biodiesel butanol dual-fuel engine, and step 2 specifically performs the following steps:

[0020] Step 2-1: State variable collection and preprocessing: collect vehicle speed, engine real-time speed N, torque T, common rail pressure P rail , fuel temperature T fuel and the mixed fuel density ρ mix , perform sliding average filtering on the collected signal. The number of collected signals is n, and the filtered signal x at time t is: Normalize the data after sliding average filtering: Step 2-2: Using the Actor network in the TD3 model, according to the current engine operating state s=[N,T,P rail ,T fuel ,ρ mix ], the output action a is the electronic throttle opening adjustment amount Δθ th , the mathematical expression of the Actor network is:

[0021] μ(s|θ μ )=[Δθ th ]=f3(W3·f2(W2·f1(W1s+b1)+b2)+b3),

[0022] Where W1, W2, W3 are the weight matrices of each layer; b1, b2, b3 are the bias terms of each layer; f1, f2, f3 are the activation functions of each layer; θμ is the parameter set of the Actor network, including all weights and biases; the TD3 model Critic network is used to evaluate the value function Q(s,a|θ) of a given state-action pair (s,a). Q ), where the TD3 model uses two independent Critic networks to reduce overestimation bias:

[0023] The first Critic network Q1 is:

[0024]

[0025] The second Critic network Q2 is:

[0026]

[0027] Where, [s,a]=[N,T,P rail ,T fuel ,ρ mix ,Δθ diesel ,Δθ butanol ]; is the parameter set of the two Critic networks,

[0028] The minimum value output by the two critic networks is selected as the final value estimate:

[0029]

[0030] Step 2-3: Set the reward function and decouple the reward function R of the cooperative control system t for:

[0031]

[0032] F(N actual )=f(N actual -N min )·f(N max -N actual ),

[0033]

[0034] ρ mix_target =g(N actual ,T actual ,λ,ρ diesel ,ρ mixture ,NOx,m air ),

[0035]

[0036] In the formula, F(N actual ) is the speed related item; G(Tactual ) is the torque-related term; f(x) is the sigmoid activation function; ρ mix_actual is the actual mixed fuel density; ρ mix_target is the target mixed fuel density; η(T fuel ) is the combustion efficiency related to fuel temperature; σ(P rail ) is the maximum burst pressure related to the common rail pressure; N min is the minimum permissible speed; N max is the maximum allowable speed; ρ diesel is the density of biodiesel; ρ mixture is the fuel density in the common rail; w1, w2, w3, and w4 are weight coefficients. The reward function maximizes the engine's power output and fuel economy by optimizing the speed range, ensuring smooth torque output, and improving the accuracy of the fuel mixture ratio. Steps 2-4: Lower-level control execution, using the PID controller to execute the electronic throttle target adjustment generated by the TD3 model:

[0037] Define the error e(t) as:

[0038] e(t)=a t -θ th (t-1), then the output θ of the PID controller is th (t) is:

[0039] Where K p is the proportional gain; K i is the integral gain; K d The PID controller converts the electronic throttle target adjustment amount generated by the TD3 algorithm into an actual valve opening command to ensure that the electronic throttle reaches the target position.

[0040] In step 3 of the present invention, the multivariable decoupling control layer adopts the PSO-PID algorithm for optimization control, and the specific steps are as follows:

[0041] Step 3-1: Collect key state variables, including boost pressure and EGR rate, to provide real-time data for control;

[0042] Step 3-2: Particle swarm optimization: Initialize the particle swarm. Each particle represents a set of PID controller parameters that will be used to adjust the EGR valve and wastegate valve. Then, the particle speed and position are updated through an iterative process. During this process, the particle swarm continuously explores to find the optimal PID parameter combination, thereby achieving precise optimization of valve control.

[0043] Step 3-3: The optimized PID parameters are transmitted to the controllers of the EGR valve and the exhaust bypass valve to achieve multivariable coordinated optimization control of the air system.

[0044] Furthermore, in step 3 of the present invention, the multivariable decoupling control layer performs the following steps:

[0045] Step 3-1: State variable collection: Collect the boost pressure P in the engine air system boost 、EGR rateEGR actual ;

[0046] Step 3-2: Use the particle swarm optimization algorithm PSO to optimize the parameters of the EGR valve and wastegate valve controllers:

[0047] First, the position and velocity of the particle swarm are initialized, the number of particles is set to 100, and the position of each particle represents a set of PID parameters [K p ,K i ,K d ], and continuously update the particle speed and position during each iteration:

[0048] v i =w·v i-1 +c1·rand1·(pbest i-1 -x i )+c2·rand2·(gbest-x i-1 ),

[0049] Where, v i is the velocity of the current particle; v i-1 is the velocity of the particle in the last iteration; w is the inertia weight; c1 and c2 are learning factors; rand1 and rand2 are random numbers between [0,1]; pbest i-1 is the optimal position of the particle itself; g best is the optimal position of the entire particle swarm; x i-1 is the position of the particle in the last iteration, and the particle position iteration update formula is:

[0050] x i =x i-1 +v i-1 ,

[0051] Where x i is the updated particle position,

[0052] When the maximum number of iterations is reached or the fitness value converges to a certain threshold, the iteration is stopped and the optimal PID parameter [K p ,K i ,K d ];

[0053] Step 3-3: Lower level control execution, EGR valve opening u EGR for:

[0054]

[0055] e EGR =EGR target -EGR actual ,

[0056] EGR target =f(N actual ,T actual ,λ,NOx),

[0057] Where, e EGR is the EGR rate difference; EGR target is the EGR rate target value; EGR actual is the actual value of EGR rate; N actual is the real-time speed; T actual is the real-time torque; λ is the air-fuel ratio; NOx is the emission sensor data. WG for:

[0058]

[0059] e WG =P boost_target -P boost ,

[0060] P boost_target =g(N actual ,T actual ,m air ),

[0061] Where, e WG is the opening error of the wastegate valve; P boost_target is the boost pressure target value; P boost is the actual value of boost pressure; m air is the intake air mass flow rate.

[0062] In step 4 of the present invention, the fuel injection control layer performs dynamic optimization calculation of the dual fuel injection ratio through the DDPG-PID intelligent control algorithm, which specifically includes the following steps:

[0063] Step 4-1: State variable collection and preprocessing, collecting engine fuel consumption m f , NOx emissions and in-cylinder gas content Perform sliding average filtering on the collected signals. The number of collected signals is n, and the filtered signal x at time t is: Normalize the data after sliding average filtering: Step 4-2: Establish the DDPG network structure. The goal of the Actor network under the DDPG framework is to calculate the current engine operating state s = [m f ,E NOx ], the output action a is the biodiesel injection valve adjustment amount Δθ diesel and the adjustment amount Δθ of the butanol injection valve butanol , the mathematical expression of the Actor network is:

[0064] μ(s|θ μ )=[Δθ diesel ,Δθ butanol ]=f3(W3·f2(W2·f1(W1s+b1)+b2)+b3),

[0065] Where W1, W2, W3 are the weight matrices of each layer; b1, b2, b3 are the bias terms of each layer; f1, f2, f3 are the activation functions of each layer; θ μ is the parameter set of the Actor network, including all weights and biases. The goal of the Critic network is to evaluate the value function Q(s,a|θ) for a given state-action pair (s,a). Q ), the critic network Q is:

[0066] Q(s,a|θ Q )=f3(W3·f2(W2·[s,a]+b2)+b3)

[0067] Where, θ Q is the parameter set of the Critic network;

[0068] Step 4-3: Set the reward function to define the agent's behavior goals and guide the optimization of the strategy through feedback signals. The reward function R t for:

[0069]

[0070] Where w1 and w2 are weight coefficients; m ref is the reference value of fuel consumption; E ref is the NOx emission reference value; Step 4-4: Lower-level control execution, DDPG algorithm output action a t =[Δθ diesel ,Δθ butanol ] is the adjustment amount of the dual fuel injection valve, that is, the change relative to the current valve opening. In order to convert the adjustment amount into the actual valve opening command, it is further processed by the PID controller and the error e(t) is defined as: e diesel (t) = a t -θ diesel(t-1),

[0071] e butanol (t) = a t -θ butanol (t-1), where e diesel (t) is the current opening error of the diesel flow control valve; θ diesel (t-1) is the opening of the diesel flow control valve at the previous moment; e butanol (t) is the current opening error of the butanol flow control valve; θ butanol (t-1) is the opening of the butanol flow control valve at the previous moment;

[0072] Then the output θ(t) of the PID controller is:

[0073]

[0074] Where θ diesel (t) is the opening of the diesel flow control valve; θ butanol (t) is the opening of the butanol flow control valve; K p is the proportional gain; K i is the integral gain; K d For differential gain, the PID controller converts the target adjustment generated by the DDPG algorithm into an actual flow control valve opening command, quickly matching the air volume in the cylinder to ensure that the valve can reach the target position quickly, smoothly and accurately.

[0075] The present invention is directed to a biodiesel and butanol dual-fuel engine equipped with a turbocharging system. Its air system is equipped with a turbocharger and an exhaust gas recirculation device. In addition, it also has components such as an electronic throttle, an air filter, a compressor, a turbine, an exhaust gas bypass valve, an EGR valve, a water-cooled EGR intercooler, and a three-way catalytic converter. The fuel system is equipped with high-precision density sensors, temperature sensors, pressure sensors, and flow control valves to monitor the dual-fuel ratio in real time and feed the data back to the control system. To address the existing problems of asynchronous fuel and air system responses under transient dual-fuel engine conditions, high costs and long cycles for extensive test calibration, and difficulty in achieving optimal engine performance, the present invention proposes an intelligent collaborative control system and method for dual-fuel engines based on deep reinforcement learning. Through deep reinforcement learning and multivariable decoupling control technology, dynamic optimization matching of fuel injection and the air system is achieved. The present invention dynamically regulates the air system's electronic throttle, EGR valve, and exhaust gas release valve based on real-time vehicle speed conditions. Taking into account the differences in oil and gas response characteristics, the dual-fuel injection ratio is dynamically adjusted to achieve optimal synchronization and coordination with the air system, thereby improving the dual-fuel engine's power, economy, and emissions. When the dual-fuel engine is running, the air system matches the power requirements of the vehicle speed through intelligent optimization algorithms and deep reinforcement learning algorithms. The fuel system dynamically adjusts the dual-fuel mass ratio based on the deep reinforcement learning algorithm to ensure synchronous matching with the air system, significantly improving the engine's dynamic response speed, operational stability, and overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Attachment Figure 1 Schematic diagram of the fuel and air system of the dual-fuel engine in the present invention.

[0077] Attachment Figure 2 Schematic diagram of the intelligent decision-making and control principle of the air system in the present invention.

[0078] Attachment Figure 3 Schematic diagram of the dual-fuel optimal ratio control principle of the fuel system in the present invention.

[0079] Attachment Figure 4 Schematic diagram of dynamic multivariable coordinated control of base oil and air system in the present invention. DETAILED DESCRIPTION

[0080] The present invention will be described in further detail below with reference to the accompanying drawings.

[0081] Example:

[0082] like Figure 1As shown, the dual-fuel engine system involved in this example includes a biodiesel fuel tank, a butanol fuel tank, a high-pressure fuel pump, a common rail pipe, a temperature sensor, a density sensor, a pressure sensor, a fuel injector, a common rail pipe, a cylinder electronic throttle, an air filter, an intake manifold, a cylinder, a compressor, a turbine, a wastegate valve, an EGR valve, a water-cooled EGR, an intercooler, and a three-way catalytic converter. The fuel supply path is as follows: biodiesel enters the high-pressure fuel pump from the biodiesel fuel tank through the biodiesel valve, and then enters the Enters the common rail pipe; butanol enters the high-pressure fuel pump from the butanol fuel tank through the butanol valve, enters the common rail pipe after pressurization and mixes with biodiesel, and the mixed fuel enters the cylinder through the injector to mix with the gas fuel; its air supply path: external air enters the compressor through the air filter, reaches the electronic throttle after cooling in the intercooler, and then enters the intake manifold; the combustion exhaust gas enters the wastegate valve through the drive turbine, part of the exhaust gas is discharged through the three-way catalytic converter, and part of the exhaust gas passes through the EGR valve and intercooler back to the intake manifold.

[0083] This example proposes a dual-fuel engine multivariable coordinated control method for the above-mentioned dual-fuel engine system, including:

[0084] Step S1: a multivariable coordinated intelligent control system for a biodiesel butanol dual-fuel engine;

[0085] Step S2: The air system control uses the TD3-PID controller to intelligently control the opening of the electronic throttle, specifically including:

[0086] Step S21: state variable collection and preprocessing, collecting vehicle speed, engine real-time running speed N, torque T, common rail pressure P rail , fuel temperature T fuel and the mixed fuel density ρ mix , perform sliding average filtering on the collected signal. The number of collected signals is n, and the filtered signal x at time t is:

[0087] Normalize the data after sliding average filtering:

[0088]

[0089] Step S22: The TD3 model is used to improve the problem of overestimation of the value function and frequent policy updates in the DDPG model, further enhancing the stability and robustness of the algorithm. The TD3 model uses two independent Critic networks and takes the minimum value as the final estimate to reduce the overestimation problem and improve the stability of the learning process. The goal of the Actor network is to calculate the value of the current engine operation state s = [N, T, P rail ,T fuel ,ρ mix], the output action a is the electronic throttle opening adjustment amount Δθ th , therefore, the mathematical expression of the Actor network is:

[0090] μ(s|θ μ )=[Δθ th ]=f3(W3·f2(W2·f1(W1s+b1)+b2)+b3),

[0091] Where W1, W2, W3 are the weight matrices of each layer; b1, b2, b3 are the bias terms of each layer; f1, f2, f3 are the activation functions of each layer; θ μ is the parameter set of the Actor network, including all weights and biases. The goal of the Critic network is to evaluate the value function Q(s,a|θ) for a given state-action pair (s,a). Q ), TD3 uses two independent Critic networks to reduce overestimation bias. The first Critic network Q1 is:

[0092]

[0093] The second Critic network Q2 is:

[0094] Where,

[0095] [s,a]=[N,T,P rail ,T fuel ,ρ mix ,Δθ diesel ,Δθ butanol ]; For the parameter set of the two critic networks, the minimum value output by the two critic networks is selected as the final value estimate:

[0096]

[0097] Step S23: Set the reward function and decouple the reward function R of the collaborative control system t for:

[0098]

[0099] F(N actual )=f(N actual -N min )·f(N max -N actual ),

[0100]

[0101] ρ mix_target =g(Nactual ,T actual ,λ,ρ diesel ,ρ mixture ,NOx,m air ),

[0102] In the formula, F(N actual ) is the speed related item; G(T actual ) is the torque-related term; f(x) is the sigmoid activation function; ρ mix_actual is the actual mixed fuel density; ρ mix_target is the target mixed fuel density; η(T fuel ) is the combustion efficiency related to fuel temperature; σ(P rail ) is the maximum burst pressure related to the common rail pressure; N min is the minimum permissible speed; N max is the maximum allowable speed; ρ diesel is the density of biodiesel; ρ mixture is the fuel density in the common rail; w1, w2, w3, and w4 are weight coefficients. The reward function maximizes engine power output and fuel economy by optimizing the speed range, ensuring smooth torque output, and improving the accuracy of the fuel mixture ratio, thereby achieving efficient, stable, and powerful powertrain operation and promoting overall system optimization and performance improvement.

[0103] Step S24: Lower layer control execution, in this example, the output action a of the TD3 algorithm t =[Δθ th ] is the adjustment of the electronic throttle, that is, the change relative to the current valve opening. In order to convert these adjustment values ​​into actual valve opening commands, this example uses a PID controller for further processing, and defines the error e(t) as: e(t) = a t -θ th (t-1), then the output θ of the PID controller is th (t) is:

[0104] Where K p is the proportional gain; K i is the integral gain; K d The PID controller converts the electronic throttle target adjustment generated by the TD3 algorithm into an actual valve opening command to ensure that the electronic throttle can reach the target position quickly, smoothly and accurately.

[0105] Step S3: multivariable decoupling control, including the following steps:

[0106] Step S31: state variable acquisition, acquisition of boost pressure P in the engine air system boost、EGR rateEGR actual , these two variables can effectively reflect the engine operating status and are used to control the opening of the EGR valve and the wastegate valve, achieving efficient coordinated control of the air system;

[0107] Step S32: PSO parameter optimization. In this example, the parameters of the EGR valve and wastegate valve controllers are optimized by particle swarm optimization (PSO) algorithm to achieve effective management of exhaust gas recirculation rate and exhaust gas emissions. First, the position and speed of the particle swarm are initialized, and the number of particle swarms is set to 100. The position of each particle represents a set of PID parameters [K p ,K i ,K d ], and continuously update the particle speed and position during each iteration:

[0108] v i =w·v i-1 +c1·rand1·(pbest i-1 -x i )+c2·rand2·(gbest-x i-1 ), where v i is the velocity of the current particle; v i-1 is the velocity of the particle in the last iteration; w is the inertia weight; c1 and c2 are learning factors;

[0109] rand1 and rand2 are random numbers between [0,1]; pbest i-1 is the optimal position of the particle itself; g best is the optimal position of the entire particle swarm; x i-1 is the position of the particle in the last iteration, and the particle position iteration update formula is:

[0110] x i =x i-1 +v i-1 , where x i is the updated particle position. When the maximum number of iterations is reached or the fitness value converges to a certain threshold, the iteration is stopped and the optimal PID parameter [K p ,K i ,K d ];

[0111] Step S33: Lower-level control is executed. In this example, the multivariable system in the air system is divided into independent single-variable control systems, which simplifies the control logic and achieves precise adjustment of the EGR rate and boost pressure, laying the foundation for the optimal balance of economy, emission and power of the dual-fuel engine. Then, the EGR valve opening u EGR for:

[0112]

[0113] e EGR =EGR target -EGR actual ,

[0114] EGR target =f(N actual ,T actual ,λ,NOx), where e EGR is the EGR rate difference; EGR target is the EGR rate target value; EGR actual is the actual value of EGR rate; N actual is the real-time speed; T actual is the real-time torque; λ is the air-fuel ratio; NOx is the emission sensor data, and the exhaust bypass valve opening u WG for:

[0115]

[0116] e WG =P boost_target -P boost ,

[0117] P boost_target =g(N actual ,T actual ,m air ), where e WG is the opening error of the wastegate valve; P boost_target is the boost pressure target value; P boost is the actual value of boost pressure; m air is the intake air mass flow rate.

[0118] Step S4: fuel injection system control, specifically including the following steps:

[0119] Step S41: state variable collection and preprocessing, collecting engine fuel consumption m f , NOx emissions and in-cylinder gas content Perform sliding average filtering on the collected signals. The number of collected signals is n, and the filtered signal x at time t is: Normalize the data after sliding average filtering:

[0120] Step S42: Establish the DDPG network structure. The Actor network and Critic network under the DDPG framework play different roles and cooperate with each other. The Actor network is responsible for outputting a specific action strategy based on the current state, that is, generating an action, and updating the parameters through the gradient ascent method to maximize the cumulative reward; the Critic network evaluates the value function of a given state-action pair, predicts the cumulative reward that may be obtained after taking a certain action, and updates the parameters by minimizing the mean square error loss function. The goal of the Actor network is to calculate the value function of the given state-action pair based on the current engine operating state. Output action a is the biodiesel injection valve adjustment amount Δθ diesel and the adjustment amount Δθ of the butanol injection valve butanol , therefore, the mathematical expression of the Actor network is:

[0121] μ(s|θ μ )=[Δθ diesel ,Δθ butanol ]=f3(W3·f2(W2·f1(W1s+b1)+b2)+b3), where W1, W2, W3 are the weight matrices of each layer; b1, b2, b3 are the bias terms of each layer; f1, f2, f3 are the activation functions of each layer; θ μ is the parameter set of the Actor network, including all weights and biases. The goal of the Critic network is to evaluate the value function Q(s,a|θ) for a given state-action pair (s,a). Q ), the critic network Q is:

[0122] Q(s,a|θ Q )=f3(W3·f2(W2·[s,a]+b2)+b3), where

[0123] θ Q is the parameter set of the Critic network;

[0124] Step S43: Set the reward function. The reward function defines the behavior goal of the agent and guides the optimization of the strategy through feedback signals. In this example, the reward function design in the fuel injection system comprehensively considers the fuel consumption and NOx emissions of the dual-fuel engine to ensure synchronization with the in-cylinder gas volume and improve the economy and emission performance of the vehicle under transient conditions. The reward function R t for:

[0125] Where w1 and w2 are weight coefficients; m ref is the reference value of fuel consumption; E ref is the reference value of NOx emissions;

[0126] Step S44: Lower layer control execution, in this example, the output action of the DDPG algorithm

[0127] a t =[Δθ diesel ,Δθ butanol ] is the adjustment of the dual fuel injection valve, that is, the change relative to the current valve opening. In order to convert these adjustment values ​​into actual valve opening commands, the present invention further processes them through a PID controller and defines the error e(t) as:

[0128] e diesel (t) = a t -θ diesel (t-1), e butanol (t) = a t -θ butanol (t-1), where e diesel (t) is the current opening error of the diesel flow control valve; θ diesel (t-1) is the opening of the diesel flow control valve at the previous moment; e butanol (t) is the current opening error of the butanol flow control valve; θ butanol (t-1) is the opening of the butanol flow control valve at the previous moment, and the output θ(t) of the PID controller is:

[0129]

[0130] Where θ diesel (t) is the opening of the diesel flow control valve; θ butanol (t) is the opening of the butanol flow control valve; K p is the proportional gain; K i is the integral gain; K d is the differential gain; the PID controller converts the target adjustment generated by the DDPG algorithm into an actual flow control valve opening command, quickly matching the air volume in the cylinder to ensure that the valve can reach the target position quickly, smoothly, and accurately. In this step, the fuel injection control, air system control, and decoupling control are organically combined to construct a hierarchical collaborative control architecture. The upper layer is responsible for global optimization and coordinating the goals of each subsystem; the lower layer performs specific control tasks to ensure real-time and accuracy. Through information interaction and feedback mechanisms, multi-objective dynamic collaborative optimization is achieved, the overall performance and robustness of the system are improved, and efficient control under complex working conditions is guaranteed.

[0131] Step S5: Fuel injection control, air system control, and decoupling control are linked to form a three-layer linkage control architecture for dynamic matching of engine air and fuel. This three-layer linkage control architecture includes an air system control layer based on the TD3-PID algorithm to achieve electronic throttle optimization control that matches power demand; a PSO-PID multivariable decoupling control layer that adjusts PID control parameters to decouple the EGR valve and wastegate valve openings, achieving stable control of the EGR rate and boost pressure; and a fuel injection control layer based on the DDPG-PID algorithm to ensure real-time control and synchronous response under transient conditions. Through information interaction and feedback mechanisms, multi-objective dynamic collaborative optimization is achieved, improving the overall performance and robustness of the system and providing intelligent decision-making support for efficient control under complex operating conditions.

Claims

1. A multivariable coordinated control method for a dual-fuel engine, characterized in that: A multivariable collaborative control model for a dual-fuel engine based on deep reinforcement learning and dynamic decoupling is established. The multivariable collaborative control model for a dual-fuel engine consists of an air system control layer, a multivariable decoupling layer, and a fuel injection control layer. The air system control layer uses the TD3-PID algorithm to calculate the vehicle speed, real-time operating speed N, torque T, and common rail pressure P. rail , fuel temperature T fuel and the mixed fuel density ρ mix As the state variable, data is collected and processed to obtain the dynamic adjustment value of the electronic throttle; The fuel injection control layer realizes intelligent optimization control of dual fuel injection through DDPG-PID algorithm, and takes the engine fuel consumption m f , NOx emissions and in-cylinder gas content As a state variable, data is collected and processed, and a comprehensive reward function is calculated to obtain the optimal flow control valve opening for each fuel. The injection amount and fuel ratio of biodiesel and butanol are precisely controlled through the actuator; The multivariable decoupling control layer adopts the PSO-PID algorithm to collaboratively control the control parameters of the EGR valve and the exhaust bypass valve.

2. The multivariable coordinated control method for a dual-fuel engine according to claim 1, characterized in that: The specific steps include: Step 1: Real-time collection of engine operating status data, including the fuel system's common rail pressure, fuel temperature, mixed fuel density, boost pressure, EGR rate, vehicle speed, rotational speed, and torque; Step 2: The fuel injection control layer uses the DDPG-PID intelligent control algorithm to dynamically optimize the dual-fuel injection ratio, synchronously match the intake volume, generate the dual-fuel valve adjustment amount, and convert it into an actual opening instruction; Step 3: The multivariable decoupling control layer uses the PSO-PID algorithm to collaboratively control the EGR valve and the wastegate valve. Particle swarm optimization technology is used to adjust the PID parameters in real time to optimize the boost pressure and EGR rate. Step 4: The air system control layer adjusts the electronic throttle opening using the TD3-PID algorithm, and achieves dynamic decoupling control between the fuel system and the air system through the self-learning capability of the neural network; Step 5: Integrate the control instructions of each subsystem based on the hierarchical collaborative architecture, and finally output a coordinated and consistent control signal to complete the entire process of engine multivariable collaborative optimization control.

3. The multivariable coordinated control method for a dual-fuel engine according to claim 2, characterized in that: The step 2 specifically includes: Step 2-1: Collect real-time engine operating status data, including fuel consumption, NOx emissions, intake pressure, intake volume, EGR rate, and air-fuel ratio. Perform sliding average filtering and normalization on the collected data to provide accurate data support for subsequent control. Step 2-2: Actor-Critic network learning: The actor network generates adjustments to the dual-fuel injection valves, while the critic network evaluates the value of each state-action pair to ensure synchronous matching of in-cylinder gas under instantaneous operating conditions. Step 2-3: Solve the comprehensive reward function and fully optimize the performance of the system; Step 2-4: Execute command conversion. The PID controller converts the adjustment value output by the Actor network into an actual valve opening command to achieve precise control of the dual-fuel injection process.

4. The multivariable coordinated control method for a dual-fuel engine according to claim 3, characterized in that: For a biodiesel-butanol dual-fuel engine, step 2 specifically performs the following steps: Step 2-1: State variable collection and preprocessing: collect vehicle speed, engine real-time speed N, torque T, common rail pressure P rail , fuel temperature T fuel and the mixed fuel density ρ mix , perform sliding average filtering on the collected signal. The number of collected signals is n, and the filtered signal x at time t is: Normalize the data after sliding average filtering: Step 2-2: Using the Actor network in the TD3 model, according to the current engine operating state s=[N,T,P rail ,T fuel ,ρ mix ], the output action a is the electronic throttle opening adjustment amount Δθ th , the mathematical expression of the Actor network is: μ(s|θ μ )=[Δθ th ]=f3(W3·f2(W2·f1(W1s+b1)+b2)+b3), Where W1, W2, W3 are the weight matrices of each layer; b1, b2, b3 are the bias terms of each layer; f1, f2, f3 are the activation functions of each layer; θ μ A collection of parameters for the Actor network, including all weights and biases; Using the TD3 model Critic network, the value function Q(s,a|θ) of a given state-action pair (s,a) is evaluated. Q ), where the TD3 model uses two independent Critic networks to reduce overestimation bias: The first Critic network Q1 is: The second Critic network Q2 is: Where [s,a]=[N,T,P rail ,T fuel ,ρ mix ,Δθ diesel ,Δθ butanol ]; is the parameter set of the two Critic networks, The minimum value output by the two critic networks is selected as the final value estimate: Step 2-3: Set the reward function and decouple the reward function R of the cooperative control system t for: F(N actual )=f(N actual -N min )·f(N max -N actual ), r mix_target =g(N actual ,T actual ,l,r diesel ,r mixture ,NOx,m air ), In the formula, F(N actual ) is the speed related item; G(T actual ) is the torque-related term; f(x) is the sigmoid activation function; ρ mix_actual is the actual mixed fuel density; ρ mix_target is the target mixed fuel density; η(T fuel ) is the combustion efficiency related to fuel temperature; σ(P rail ) is the maximum burst pressure related to the common rail pressure; N min is the minimum permissible speed; N max is the maximum allowable speed; ρ diesel is the density of biodiesel; ρ mixture is the fuel density in the common rail; w1, w2, w3, and w4 are weight coefficients. The reward function maximizes the engine's power output and fuel economy by optimizing the speed range, ensuring smooth torque output, and improving the accuracy of the fuel mixture ratio. Steps 2-4: Lower-level control execution, using the PID controller to execute the electronic throttle target adjustment generated by the TD3 model: Define the error e(t) as: e(t)=a t -θ th (t-1), then the output θ of the PID controller is th (t) is: Where K p is the proportional gain; K i is the integral gain; K d The PID controller converts the electronic throttle target adjustment amount generated by the TD3 algorithm into an actual valve opening command to ensure that the electronic throttle reaches the target position.

5. The multivariable coordinated control method for a dual-fuel engine according to claim 4, characterized in that: In step 3, the multivariable decoupling control layer uses the PSO-PID algorithm for optimization control. The specific steps are as follows: Step 3-1: Key state variables are collected, including boost pressure and EGR rate, to provide real-time data for control. Step 3-2: Particle swarm optimization is used to initialize the particle swarm. Each particle represents a set of PID controller parameters that will be used to adjust the EGR valve and wastegate valve. The particle speed and position are then updated through an iterative process. During this process, the particle swarm continuously explores to find the optimal PID parameter combination, thereby achieving precise optimization of valve control. Step 3-3: The optimized PID parameters are transmitted to the controllers of the EGR valve and the exhaust bypass valve to achieve multivariable coordinated optimization control of the air system.

6. The multivariable coordinated control method for a dual-fuel engine according to claim 5, characterized in that: Step 3 The multivariable decoupling control layer performs the following steps: Step 3-1: State variable collection: Collect the boost pressure P in the engine air system boost 、EGR rateEGR actual ; Step 3-2: Use the particle swarm optimization algorithm PSO to optimize the parameters of the EGR valve and wastegate valve controllers: First, the position and velocity of the particle swarm are initialized, the number of particles is set to 100, and the position of each particle represents a set of PID parameters [K p ,K i ,K d ], and continuously update the particle speed and position during each iteration: v i =w·v i-1 +c1·rand1·(pbest i-1 -x i )+c2·rand2·(gbest-x i-1 ), Where, v i is the velocity of the current particle; v i-1 is the velocity of the particle in the last iteration; w is the inertia weight; c1 and c2 are learning factors; rand1 and rand2 are random numbers between [0,1]; pbest i-1 is the optimal position of the particle itself; gbest is the optimal position of the entire particle group; x i-1 is the position of the particle in the last iteration, and the particle position iteration update formula is: x i =x i-1 +v i-1 , Where x i is the updated particle position, When the maximum number of iterations is reached or the fitness value converges to a certain threshold, the iteration is stopped and the optimal PID parameter [K p ,K i ,K d ]; Step 3-3: Lower level control execution, EGR valve opening u EGR for: e EGR =EGR target -EGR actual , EGR target =f(N actual ,T actual ,λ,NOx), Where, e EGR is the EGR rate difference; EGR target is the EGR rate target value; EGR actual is the actual value of EGR rate; N actual is the real-time speed; T actual is the real-time torque; λ is the air-fuel ratio; NOx is the emission sensor data. WG for: e WG =P boost_target -P boost , P boost_target =g(N actual ,T actual ,m air ), Where, e WG is the opening error of the wastegate valve; P boost_target is the boost pressure target value; P boost is the actual value of boost pressure; m air is the intake air mass flow rate.

7. The multivariable coordinated control method for a dual-fuel engine according to claim 6, characterized in that: In step 4, the fuel injection control layer performs dynamic optimization calculation of the dual fuel injection ratio through the DDPG-PID intelligent control algorithm, which specifically includes the following steps: Step 4-1: State variable collection and preprocessing, collecting engine fuel consumption m f , NOx emissions and in-cylinder gas content Perform sliding average filtering on the collected signals. The number of collected signals is n, and the filtered signal x at time t is: Normalize the data after sliding average filtering: Step 4-2: Establish the DDPG network structure. The goal of the Actor network under the DDPG framework is to Output action a is the biodiesel injection valve adjustment amount Δθ diesel and the adjustment amount Δθ of the butanol injection valve butanol , the mathematical expression of the Actor network is: μ(s|θ μ )=[Δθ diesel ,Dth butanol ]=f3(W3·f2(W2·f1(W1s+b1)+b2)+b3), Where W1, W2, W3 are the weight matrices of each layer; b1, b2, b3 are the bias terms of each layer; f1, f2, f3 are the activation functions of each layer; θ μ is the parameter set of the Actor network, including all weights and biases, The goal of the critic network is to evaluate the value function Q(s,a|θ) for a given state-action pair (s,a) Q ), Critic network Q is: Q(s,a|θ Q )=f3(W3·f2(W2·[s,a]+b2)+b3) Where, θ Q is the parameter set of the Critic network; Step 4-3: Set the reward function to define the agent's behavior goals and guide the optimization of the strategy through feedback signals. The reward function R t for: Where w1 and w2 are weight coefficients; m ref is the reference value of fuel consumption; E ref is the reference value of NOx emissions; Step 4-4: Lower-level control execution, DDPG algorithm output action a t =[Δθ diesel ,Δθ butanol ] is the adjustment of the dual fuel injection valve, that is, the change relative to the current valve opening. In order to convert the adjustment into the actual valve opening command, it is further processed by the PID controller and the error e(t) is defined as: e diesel (t)=a t -θ diesel (t-1), e butanol (t)=a t -θ butanol (t-1), Where, e diesel (t) is the current opening error of the diesel flow control valve; θ diesel (t-1) is the opening of the diesel flow control valve at the previous moment; e butanol (t) is the current opening error of the butanol flow control valve; θ butanol (t-1) is the opening of the butanol flow control valve at the previous moment; Then the output θ(t) of the PID controller is: Where θ diesel (t) is the opening of the diesel flow control valve; θ butanol (t) is the opening of the butanol flow control valve; K p is the proportional gain; K i is the integral gain; K d For differential gain, the PID controller converts the target adjustment generated by the DDPG algorithm into an actual fuel flow control valve opening instruction, quickly matching the air volume in the cylinder to ensure that the valve can reach the target position quickly, smoothly and accurately.

Images