Full-digital electronic injection dynamic ignition optimization method based on reinforcement learning

Abstract

The invention discloses an all-digital electronic injection dynamic ignition optimization method based on reinforcement learning, which comprises the following steps of: preprocessing acquired operation data of an engine to obtain a preprocessed operation state data set; based on the preprocessed operation state data, establishing an improved near-end strategy optimization algorithm model; based on the improved near-end strategy optimization algorithm model, obtaining a task adaptive strategy parameter set; aggregating the task adaptive strategy parameter set to obtain global meta strategy parameters; deploying the global meta-strategy parameters to an engine electronic control unit, and outputting a dynamic ignition control result; on the basis of the dynamic ignition control result, updated control strategy parameters are obtained; and the updated control strategy parameters are used for continuous iterative optimization, and a cross-working-condition self-adaptive ignition optimization control result is formed. An improved near-end strategy optimization algorithm is adopted, and self-adaptive control over oil injection and ignition of the engine is achieved.

Description

Technical Field

[0001] This invention relates to the field of engine electronic control technology, and in particular to a fully digital electronic fuel injection dynamic ignition optimization method based on reinforcement learning. Background Technology

[0002] Current engine electronic control systems generally employ fuel injection and ignition control strategies based on lookup tables and model predictions. These control methods typically determine the injection pulse width and ignition advance angle using fixed empirical calibration values ​​or static mapping tables to adapt to different load, speed, and temperature conditions. In traditional systems, control parameters rely primarily on offline calibration and expert experience, making it difficult to adaptively adjust to changes in environment and conditions during engine operation. With increasing operating complexity, fixed mapping tables and rule-based control can easily lead to decreased combustion efficiency, increased emissions, and a higher risk of knocking, especially when operating across different conditions or changing fuel types, where system stability and fuel economy are limited.

[0003] In recent years, engine control methods based on machine learning and neural networks have been studied, but most algorithms still rely on static training data and lack online update capabilities. While traditional reinforcement learning methods can acquire control strategies through trial and error, they suffer from unstable training, slow convergence, and performance degradation during multi-condition transitions, making them difficult to apply directly to real-time control scenarios. Furthermore, existing reinforcement learning algorithms in engine control often focus on single-task optimization, failing to effectively achieve coordinated optimization of fuel injection and ignition control, and lack a global strategy aggregation mechanism for cross-condition tasks.

[0004] Therefore, how to provide a reinforcement learning-based all-digital electronic fuel injection dynamic ignition optimization method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a reinforcement learning-based all-digital electronic fuel injection dynamic ignition optimization method. This invention employs an improved proximal strategy optimization algorithm model, introducing a two-layer structure of a meta-parameter layer and a task adaptation layer. It samples, trains, and optimizes engine operating data under different speeds, loads, and temperatures, achieving coordinated control of injection pulse width and ignition advance angle. Through cross-condition task aggregation learning and online parameter adaptive updates, this invention enables the engine electronic control unit to adjust its control strategy in real time under complex operating environments, resulting in advantages such as high combustion efficiency, stable knock control, and excellent emission performance.

[0006] A method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0007] Collect engine operating data, preprocess the operating data, and obtain a preprocessed operating status dataset;

[0008] Based on preprocessed runtime data, an improved near-end policy optimization algorithm model is established, and the parameter mapping relationship between the state input, action output, comprehensive reward function, meta-parameter layer and task adaptive layer is set.

[0009] Based on the improved near-end policy optimization algorithm model, multi-condition task sets are sampled from different working condition data, and inner-layer policy optimization is performed to obtain a set of task adaptive policy parameters.

[0010] Aggregate the set of task adaptive policy parameters, calculate the average loss across tasks, and update the outer meta-parameters to obtain the global meta-policy parameters;

[0011] The global meta-strategy parameters are deployed to the engine electronic control unit, and dynamic ignition control results are output based on preprocessed operating status data.

[0012] Based on the dynamic ignition control results, the instantaneous reward value is calculated and fed back to the improved near-end strategy optimization algorithm model to obtain the updated control strategy parameters.

[0013] By continuously iterating and optimizing the improved proximal strategy optimization algorithm model using the updated control strategy parameters, adaptive ignition optimization control results across operating conditions are generated.

[0014] Optionally, the process of collecting engine operating data and preprocessing the operating data to obtain a preprocessed operating status dataset specifically includes:

[0015] The engine's operating parameters are collected synchronously through the engine data acquisition module. These operating parameters include engine speed, load, intake pressure, throttle opening, fuel injection pulse width, coolant temperature, and knock signal. The collected operating parameters are recorded and organized in a time series to form a raw data set.

[0016] The original dataset is noise-filtered, and the sampled sequence is smoothed using a moving average filter. The average value of each sampled point is calculated within a set time window.

[0017] Outlier removal is performed on the filtered data. The mean and standard deviation of each sampling sequence are calculated. When the value of a sampling point deviates from the mean of the sampling sequence by more than a preset multiple threshold, the sampling point is identified as outlier data and removed.

[0018] The data after outlier removal is normalized by scaling the minimum and maximum values ​​linearly to map the range of each running parameter to between zero and one, thus obtaining a preprocessed running status dataset.

[0019] Optionally, the step of establishing an improved proximal policy optimization algorithm model based on preprocessed runtime state data, and setting the parameter mapping relationship between the state input, action output, comprehensive reward function, meta-parameter layer, and task adaptive layer specifically includes:

[0020] The preprocessed operating status data is organized into a time series form of a state vector. The state vector includes normalized operating parameters such as engine speed, engine load, intake pressure, throttle opening, fuel injection pulse width, coolant temperature, and knock signal. Corresponding action output vectors are set, including fuel injection pulse width control quantity and ignition advance angle control quantity.

[0021] An improved proximal policy optimization algorithm model is established, which includes an outer meta-parameter layer and an inner task adaptation layer.

[0022] The parameters of the outer meta-parameter layer are denoted as meta-parameters, the policy network parameters of the inner task adaptation layer are denoted as policy parameters, and the value network parameters are denoted as value parameters. The initial parameters of the inner task adaptation layer are set as the parameter values ​​of the outer meta-parameter layer to complete the parameter initialization.

[0023] Define the policy function and value function in the improved proximal policy optimization algorithm model, and set the action output space of the policy function and the output estimation range of the value function;

[0024] A comprehensive reward function is set up, with combustion efficiency, fuel consumption index, emission index and knock rate as input parameters for reward calculation. Weight coefficients are set according to the importance of each index. The weight coefficient corresponding to combustion efficiency is recorded as the first weight coefficient, the weight coefficient corresponding to fuel consumption index is recorded as the second weight coefficient, the weight coefficient corresponding to emission index is recorded as the third weight coefficient, and the weight coefficient corresponding to knock rate is recorded as the fourth weight coefficient. The instant reward value is calculated.

[0025] The policy probability ratio at the current moment is calculated based on the policy function. The probability of the current policy outputting a specific action in a given state is compared with the probability of the previous policy outputting the same action in the same state to obtain the policy probability ratio at the current moment.

[0026] The advantage function estimate is calculated by subtracting the value estimate of the current state from the cumulative return at the current moment, in order to obtain the advantage function estimate used to measure the direction and magnitude of policy improvement.

[0027] A pruning objective function for near-end policy optimization is established. The pruning objective function is used to limit the range of policy parameter updates. During the calculation process, the policy probability ratio is multiplied by the estimated value of the advantage function to obtain the first term. At the same time, the policy probability ratio is limited to a preset upper and lower threshold range to obtain the second term. The smaller value between the first term and the second term is taken as the policy optimization objective value in the current state.

[0028] Calculate the value loss function, which measures the deviation between the value function prediction result and the return estimate. The loss value is obtained by calculating the squared difference between the two.

[0029] The strategy entropy term is calculated, which is used to characterize the diversity of the strategy output actions and is obtained by calculating the entropy value of the strategy output probability distribution.

[0030] The pruning objective function, value loss function, and policy entropy term are weighted and summed according to preset weight coefficients to form the total objective function for jointly optimizing policy parameters and value parameters. The total objective function is used as the basis for updating the improved proximal policy optimization algorithm model.

[0031] Based on the policy output of multiple working conditions, the gradient update results of each task are calculated, and the updated meta-parameters are obtained by performing aggregate update through the outer meta-parameter layer.

[0032] The updated meta-parameters are used to reinitialize the policy parameters of the task adaptation layer, mapping the values ​​of the meta-parameters to the parameter space of the task adaptation layer to establish a one-to-one correspondence between the meta-parameters and the task adaptation layer.

[0033] Optionally, the improved near-end policy optimization algorithm model, which samples multi-condition task sets from different working condition data, performs inner-layer policy optimization to obtain a task adaptive policy parameter set, specifically includes:

[0034] Based on the improved near-end strategy optimization algorithm model, several sets of sample working conditions are selected from the working condition data of different speeds, different loads and different coolant temperature combinations. Each set of sample working conditions is defined as an independent working condition task, and all selected working condition tasks are numbered to form a multi-working condition task set.

[0035] For each working condition task, the state vector corresponding to the working condition task is input into the task adaptation layer. The current strategy function is used to output the corresponding action sequence according to the input state. The action sequence includes the fuel injection pulse width control quantity and the ignition advance angle control quantity. The output action sequence is applied to the environment for interaction, and state transition information and instant reward value are obtained from the environment to form a state transition sequence containing the current state, action, reward and next state.

[0036] Calculate the cumulative reward value for each task condition, and then sum the instantaneous rewards obtained by the task condition in multiple time steps in chronological order.

[0037] The policy parameters are updated according to the pruning objective function. First, the product of the policy probability ratio and the estimated value of the advantage function is calculated and compared with the product after pruning by upper and lower thresholds. The smaller value of the two is selected as the current optimization objective to limit the magnitude of policy updates.

[0038] Then, the policy parameters are adjusted along the gradient direction of the optimization objective with a policy learning rate as the step size to obtain the updated policy parameters. The policy learning rate is used to control the speed of parameter updates.

[0039] The strategy parameters obtained after the strategy update for each task are summarized and arranged in order according to the task number to form a set of task adaptive strategy parameters.

[0040] Optionally, the aggregation of the task adaptive policy parameter set, calculation of the average loss across tasks, and updating of the outer meta-parameters to obtain the global meta-policy parameters specifically include:

[0041] Receive a set of task adaptive strategy parameters. Each task under a working condition corresponds to an independent strategy parameter item. The set of task adaptive strategy parameters is used as input data for cross-working condition aggregation calculation and meta-parameter update.

[0042] Based on the overall objective function, the pruning objective function, value loss function, and strategy entropy term are weighted and summed according to preset weight coefficients to calculate the overall objective function value for each task.

[0043] The average loss across all task conditions is obtained by summing the total objective function values ​​of each task in the current training round and dividing by the number of task conditions.

[0044] The gradient update results of the meta-parameters are calculated based on the average loss. The values ​​of the meta-parameters are adjusted in the opposite direction of the gradient of the average loss function according to the step size determined by the meta-learning rate to obtain the updated meta-parameters, which are then output as global meta-policy parameters.

[0045] Optionally, the step of deploying global meta-strategy parameters to the engine electronic control unit and outputting dynamic ignition control results based on preprocessed operating status data specifically includes:

[0046] The global meta-strategy parameters are imported into the engine electronic control unit to initialize the strategy network built by the improved proximal strategy optimization algorithm model inside the electronic control unit. This enables the strategy network to have the global optimal control capability under cross-operating conditions and to maintain parameter consistency with the improved proximal strategy optimization algorithm model.

[0047] The pre-processed operating status data is input into the strategy network in the electronic control unit in real time, and the strategy function is used to calculate the output action based on the input status. The output action includes the fuel injection pulse width control amount and the ignition advance angle control amount.

[0048] The output action is converted into an execution signal by the instruction parsing module and transmitted to the fuel injection control module and the ignition control module respectively, driving the fuel injector and the ignition coil to perform the corresponding fuel injection and ignition operations.

[0049] During engine operation, the output sequences of the fuel injection pulse width control quantity and the ignition advance angle control quantity are recorded step by step to form a dynamic control command sequence, which is used to describe the output results of control execution at each time step.

[0050] Engine feedback data after executing a sequence of dynamic control commands is collected. The feedback data includes combustion efficiency, torque output, knock rate, and emission concentration. The feedback data is matched with the dynamic control commands to generate dynamic ignition control results, which are then sent back to the electronic control unit for subsequent performance evaluation and parameter updates.

[0051] Optionally, the step of calculating the instantaneous reward value based on the dynamic ignition control results and feeding it back to the improved near-end strategy optimization algorithm model to obtain the updated control strategy parameters specifically includes:

[0052] Four types of feedback indicators—combustion efficiency, fuel consumption, emission concentration, and knock rate—are extracted from the dynamic ignition control results. The real-time data of each feedback indicator are normalized to form standardized feedback signals.

[0053] The instant reward value at each time step is calculated based on the comprehensive reward function. The instant reward value is combined with the corresponding state vector, action vector and next state vector to form a state transition sample and stored in the experience buffer.

[0054] The state transition samples are input into the improved proximal policy optimization algorithm model to calculate the advantage function estimate and optimize the policy based on the pruning objective function, thereby updating the policy network parameters.

[0055] The updated strategy parameters are written into the electronic control unit to replace the original parameters, thereby enabling adaptive adjustment of the fuel injection pulse width and ignition advance angle control strategies, and generating updated control strategy parameters for use in subsequent control cycles.

[0056] Optionally, the step of continuously iteratively optimizing the improved near-end strategy optimization algorithm model using the updated control strategy parameters to form cross-condition adaptive ignition optimization control results specifically includes:

[0057] During long-term engine operation, the improved proximal strategy optimization algorithm model is continuously trained iteratively by calling the updated control strategy parameters, and the engine's operating data under different time periods and operating conditions are continuously input into the model for incremental learning.

[0058] The collected operational data is normalized and averaged to form a continuous time series input, and the corresponding action sequence is output based on the current strategy function. The action sequence includes the fuel injection pulse width control amount and the ignition advance angle control amount.

[0059] The instant reward value for each time step is calculated based on the actual operation feedback information of the engine. The rewards obtained in consecutive time steps are weighted and accumulated in chronological order to obtain the cumulative reward value corresponding to each time step. The reward discount factor is used to determine the proportion of the impact of future rewards on the current cumulative reward, and the maximum time step length is used to limit the time range of the cumulative calculation.

[0060] The cumulative return value is subtracted from the value function output to obtain the advantage function value at each time step. Combined with the pruning objective function, the strategy parameters are iteratively optimized to obtain the updated strategy parameters.

[0061] The updated strategy parameters are written into the electronic control unit, replacing the original strategy parameters. The output of the injection pulse width and ignition advance angle are adjusted in real time according to the new strategy parameters, so that the engine can maintain stable operation under changes in fuel type, ambient temperature and load, forming a cross-condition adaptive ignition optimization control result.

[0062] The beneficial effects of this invention are:

[0063] This invention achieves dynamic coordinated control of fuel injection and ignition strategies by introducing an improved proximal strategy optimization algorithm into fully digital electronic fuel injection control. The proposed method establishes a two-layer structure within the engine electronic control unit, comprising a meta-parameter layer and a task adaptation layer. Through adaptive training under multiple operating conditions and cross-condition meta-parameter aggregation updates, the control system can automatically correct the control strategy under different speeds, loads, and temperatures. This method solves the problem that traditional lookup-table or fixed-model-based control struggles to adapt to complex operating environments, ensuring coordinated matching of injection pulse width and ignition advance angle under varying operating conditions, thereby maintaining combustion stability and energy efficiency balance.

[0064] By introducing a mechanism for estimating the objective function and the advantage function during the strategy update process, this invention achieves constrained optimization of strategy parameters, ensuring the convergence stability of the control process. Combined with the dynamic calculation of the immediate reward function, the system can evaluate the comprehensive performance indicators among combustion efficiency, fuel consumption, knock rate, and emission levels in real time, achieving self-learning and self-adjustment of engine control. Through continuous training and online iteration, the control model of this invention maintains high response speed and control accuracy under different operating conditions, exhibiting a comprehensive effect of high control accuracy, strong dynamic adaptability, improved combustion efficiency, and reduced emission levels. Attached Figure Description

[0065] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0066] Figure 1 This is a flowchart of a fully digital electronic fuel injection dynamic ignition optimization method based on reinforcement learning proposed in this invention;

[0067] Figure 2 This is a schematic diagram of the structure of the improved proximal strategy optimization algorithm model in the reinforcement learning-based all-digital electronic fuel injection dynamic ignition optimization method proposed in this invention;

[0068] Figure 3 This is a schematic diagram of the parameter update process of the task adaptive layer and the meta-parameter layer in the all-digital electronic fuel injection dynamic ignition optimization method based on reinforcement learning proposed in this invention. Detailed Implementation

[0069] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0070] refer to Figure 1-3 A method for optimizing dynamic ignition in fully digital electronic fuel injection based on reinforcement learning includes the following steps:

[0071] Collect engine operating data, preprocess the operating data, and obtain a preprocessed operating status dataset;

[0072] Based on preprocessed runtime data, an improved near-end policy optimization algorithm model is established, and the parameter mapping relationship between the state input, action output, comprehensive reward function, meta-parameter layer and task adaptive layer is set.

[0073] Based on the improved near-end policy optimization algorithm model, multi-condition task sets are sampled from different working condition data, and inner-layer policy optimization is performed to obtain a set of task adaptive policy parameters.

[0074] Aggregate the set of task adaptive policy parameters, calculate the average loss across tasks, and update the outer meta-parameters to obtain the global meta-policy parameters;

[0075] The global meta-strategy parameters are deployed to the engine electronic control unit, and dynamic ignition control results are output based on preprocessed operating status data.

[0076] Based on the dynamic ignition control results, the instantaneous reward value is calculated and fed back to the improved near-end strategy optimization algorithm model to obtain the updated control strategy parameters.

[0077] By continuously iterating and optimizing the improved proximal strategy optimization algorithm model using the updated control strategy parameters, adaptive ignition optimization control results across operating conditions are generated.

[0078] In this embodiment, the process of collecting engine operating data and preprocessing the operating data to obtain a preprocessed operating status dataset specifically includes:

[0079] The engine's operating parameters are collected synchronously through the engine data acquisition module. These operating parameters include engine speed, load, intake pressure, throttle opening, fuel injection pulse width, coolant temperature, and knock signal. The collected operating parameters are recorded and organized in a time series to form a raw data set containing all real-time operating characteristics.

[0080] Noise is filtered out from the original dataset, and the sampling sequence is smoothed by using a moving average filter. The average value of each sampling point is calculated within a set time window to replace the original data value at the center of the set time window, in order to reduce the impact of instantaneous fluctuations on subsequent feature extraction.

[0081] Outlier removal is performed on the filtered data. The mean and standard deviation of each sampling sequence are calculated. When the value of a sampling point deviates from the mean of the sampling sequence by more than a preset multiple threshold, the sampling point is identified as outlier data and removed to ensure that the data used in subsequent processing is stable and conforms to the normal operating range of the engine.

[0082] The data after outlier removal is normalized by scaling the minimum and maximum values ​​linearly to map the range of each operating parameter to between zero and one, thereby eliminating numerical differences between variables of different dimensions and obtaining a preprocessed operating status dataset.

[0083] In this embodiment, the step of establishing an improved near-end policy optimization algorithm model based on preprocessed runtime state data, and setting the parameter mapping relationship between the state input, action output, comprehensive reward function, meta-parameter layer, and task adaptive layer specifically includes:

[0084] The preprocessed operating status data is organized into a time series form of a state vector. The state vector includes normalized operating parameters such as engine speed, engine load, intake pressure, throttle opening, fuel injection pulse width, coolant temperature, and knock signal. Corresponding action output vectors are set, including fuel injection pulse width control quantity and ignition advance angle control quantity.

[0085] An improved near-end strategy optimization algorithm model is established, which includes an outer meta-parameter layer and an inner task adaptation layer. The outer meta-parameter layer is used to aggregate and update parameters between tasks under different working conditions, and the inner task adaptation layer is used to adjust the strategy for the current working condition.

[0086] The parameters of the outer meta-parameter layer are denoted as meta-parameters, the policy network parameters of the inner task adaptation layer are denoted as policy parameters, and the value network parameters are denoted as value parameters. The initial parameters of the inner task adaptation layer are set as the parameter values ​​of the outer meta-parameter layer to complete the parameter initialization.

[0087] Define the policy function and value function in the improved proximal policy optimization algorithm model. The policy function is used to describe the probability distribution of the output action in a given state, and the value function is used to estimate the expected reward in the current state. Set the action output space of the policy function and the output estimation range of the value function to constrain the policy output and value evaluation of the model during training and execution.

[0088] A comprehensive reward function is defined, with combustion efficiency, fuel consumption, emission, and knock rate as input parameters for reward calculation. Weight coefficients are set according to the importance of each indicator. The weight coefficient corresponding to combustion efficiency is recorded as the first weight coefficient, the weight coefficient corresponding to fuel consumption is recorded as the second weight coefficient, the weight coefficient corresponding to emission is recorded as the third weight coefficient, and the weight coefficient corresponding to knock rate is recorded as the fourth weight coefficient. The calculated immediate reward value is used to guide the policy update of the improved near-end policy optimization algorithm model.

[0089] The policy probability ratio at the current moment is calculated based on the policy function. The probability of the current policy outputting a specific action in a given state is compared with the probability of the previous policy outputting the same action in the same state to obtain the policy probability ratio at the current moment.

[0090] Calculate the advantage function estimate by subtracting the value estimate of the current state from the cumulative return at the current moment to obtain the advantage function estimate used to measure the direction and magnitude of policy improvement, which is used to guide subsequent policy parameter updates;

[0091] A pruning objective function for near-end policy optimization is established. The pruning objective function is used to limit the range of policy parameter updates. During the calculation process, the policy probability ratio is multiplied by the estimated value of the advantage function to obtain the first term. At the same time, the policy probability ratio is limited to a preset upper and lower threshold range to obtain the second term. The smaller value between the first term and the second term is taken as the policy optimization objective value in the current state to prevent the policy update range from being too large and causing performance degradation.

[0092] Calculate the value loss function, which measures the deviation between the value function prediction result and the return estimate. The loss value is obtained by calculating the squared difference between the two and is used to guide the parameter update of the value function.

[0093] The policy entropy term is calculated, which is used to characterize the diversity of policy output actions. It is obtained by calculating the entropy value of the policy output probability distribution, and is used to encourage the policy to maintain sufficient exploration ability during training.

[0094] The pruning objective function, value loss function, and policy entropy term are weighted and summed according to preset weight coefficients to form the total objective function for jointly optimizing policy parameters and value parameters. The total objective function is used as the basis for updating the improved proximal policy optimization algorithm model.

[0095] Based on the policy outputs of multiple task conditions, the gradient update results for each task are calculated, and then aggregated updates are performed through the outer meta-parameter layer to obtain the updated meta-parameters:

[0096] ;

[0097] in, For the updated meta-parameters, For meta-parameters, The meta-learning rate controls the step size for updating meta-parameters. The number of work condition tasks participating in the update. For the first Task adaptive layer strategy parameters corresponding to each working condition task ) is the first The overall objective function value of each task during the training process. For the first The gradient of the overall objective function of each task condition with respect to the policy parameters;

[0098] The updated meta-parameters are used to reinitialize the policy parameters of the task adaptation layer, and the values ​​of the meta-parameters are mapped to the parameter space of the task adaptation layer to establish a one-to-one correspondence between the meta-parameters and the task adaptation layer. This allows the task adaptation layer to inherit the global policy information contained in the meta-parameters under new working conditions, thereby achieving rapid adaptation and stable control in subsequent multi-working-condition self-learning and cross-working-condition migration optimization processes.

[0099] In this embodiment, the improved near-end policy optimization algorithm model, which samples multi-condition task sets from different working condition data, performs inner-layer policy optimization, and obtains a task adaptive policy parameter set, specifically includes:

[0100] Based on the improved near-end strategy optimization algorithm model, several sets of sample working conditions are selected from the working condition data of different speed, different load and different coolant temperature combinations. Each set of sample working conditions is defined as an independent working condition task, and all selected working condition tasks are numbered to form a multi-working condition task set to support the subsequent task adaptive training process.

[0101] For each working condition task, the state vector corresponding to the working condition task is input into the task adaptation layer. The current strategy function is used to output the corresponding action sequence based on the input state. The action sequence includes the fuel injection pulse width control quantity and the ignition advance angle control quantity. The output action sequence is applied to the environment to interact and obtain state transition information and real-time reward value from the environment to form a state transition sequence containing the current state, action, reward and the next state, which is used for subsequent strategy optimization and parameter update.

[0102] Calculate the cumulative reward value for each task condition, and then sum the instantaneous rewards obtained by the task condition in multiple time steps in chronological order.

[0103] The policy parameters are updated according to the pruning objective function. First, the product of the policy probability ratio and the estimated value of the advantage function is calculated and compared with the product after pruning by upper and lower thresholds. The smaller value of the two is selected as the current optimization objective to limit the magnitude of policy updates.

[0104] Then, the policy parameters are adjusted along the gradient direction of the optimization objective with the policy learning rate as the step size to obtain the updated policy parameters. The policy learning rate is used to control the speed of parameter updates, and the updated policy parameters are used to characterize the state of each task after policy optimization.

[0105] The strategy parameters obtained after the strategy update for each task are summarized and arranged in order according to the task number to form a set of task adaptive strategy parameters.

[0106] In this embodiment, the aggregation of the task adaptive policy parameter set, the calculation of the average loss across tasks, and the updating of the outer meta-parameters to obtain the global meta-policy parameters specifically include:

[0107] Receive a set of task adaptive strategy parameters. Each task under a working condition corresponds to an independent strategy parameter item. The set of task adaptive strategy parameters is used as input data for cross-working condition aggregation calculation and meta-parameter update.

[0108] Based on the overall objective function, the pruning objective function, value loss function, and strategy entropy term are weighted and summed according to preset weight coefficients to calculate the overall objective function value for each task.

[0109] The average loss across all task conditions is obtained by summing the total objective function values ​​of each task in the current training round and dividing by the number of task conditions.

[0110] The gradient update results of the meta-parameters are calculated based on the average loss. The values ​​of the meta-parameters are adjusted in the opposite direction of the gradient of the average loss function according to the step size determined by the meta-learning rate to obtain the updated meta-parameters, which are then output as global meta-policy parameters.

[0111] In this embodiment, the step of deploying global meta-strategy parameters to the engine electronic control unit and outputting dynamic ignition control results based on preprocessed operating status data specifically includes:

[0112] The global meta-strategy parameters are imported into the engine electronic control unit to initialize the strategy network built by the improved proximal strategy optimization algorithm model inside the electronic control unit. This enables the strategy network to have the global optimal control capability under cross-operating conditions and to maintain parameter consistency with the improved proximal strategy optimization algorithm model.

[0113] The pre-processed operating status data is input into the strategy network in the electronic control unit in real time, and the strategy function is used to calculate the output action based on the input status. The output action includes the fuel injection pulse width control amount and the ignition advance angle control amount.

[0114] The output action is converted into an execution signal by the instruction parsing module and transmitted to the fuel injection control module and the ignition control module respectively, driving the fuel injector and the ignition coil to perform the corresponding fuel injection and ignition operations.

[0115] During engine operation, the output sequences of the fuel injection pulse width control quantity and the ignition advance angle control quantity are recorded step by step to form a dynamic control command sequence, which is used to describe the output results of control execution at each time step.

[0116] Engine feedback data after executing a sequence of dynamic control commands is collected. The feedback data includes combustion efficiency, torque output, knock rate, and emission concentration. The feedback data is matched with the dynamic control commands to generate dynamic ignition control results, which are then sent back to the electronic control unit for subsequent performance evaluation and parameter updates.

[0117] In this embodiment, the step of calculating the instantaneous reward value based on the dynamic ignition control result and feeding it back to the improved near-end strategy optimization algorithm model to obtain the updated control strategy parameters specifically includes:

[0118] Four types of feedback indicators—combustion efficiency, fuel consumption, emission concentration, and knock rate—are extracted from the dynamic ignition control results. The real-time data of each feedback indicator are normalized to form standardized feedback signals.

[0119] The instant reward value at each time step is calculated based on the comprehensive reward function. The instant reward value is combined with the corresponding state vector, action vector and next state vector to form a state transition sample and stored in the experience buffer.

[0120] The state transition samples are input into the improved proximal policy optimization algorithm model to calculate the advantage function estimate and optimize the policy based on the pruning objective function, thereby updating the policy network parameters.

[0121] The updated strategy parameters are written into the electronic control unit to replace the original parameters, thereby enabling adaptive adjustment of the fuel injection pulse width and ignition advance angle control strategies, and generating updated control strategy parameters for use in subsequent control cycles.

[0122] In this embodiment, the step of using the updated control strategy parameters to continuously iteratively optimize and improve the near-end strategy optimization algorithm model to form cross-condition adaptive ignition optimization control results specifically includes:

[0123] During long-term engine operation, the improved proximal strategy optimization algorithm model is continuously trained iteratively by calling the updated control strategy parameters, and the engine's operating data under different time periods and operating conditions are continuously input into the model for incremental learning.

[0124] The collected operational data is normalized and averaged to form a continuous time series input, and the corresponding action sequence is output based on the current strategy function. The action sequence includes the fuel injection pulse width control amount and the ignition advance angle control amount.

[0125] The instant reward value for each time step is calculated based on the actual operation feedback information of the engine. The rewards obtained in consecutive time steps are weighted and accumulated in chronological order to obtain the cumulative reward value corresponding to each time step. The reward discount factor is used to determine the proportion of the impact of future rewards on the current cumulative reward, and the maximum time step length is used to limit the time range of the cumulative calculation.

[0126] The cumulative return value is subtracted from the value function output to obtain the advantage function value at each time step. Combined with the pruning objective function, the strategy parameters are iteratively optimized to obtain the updated strategy parameters.

[0127] The updated strategy parameters are written into the electronic control unit, replacing the original strategy parameters. The output of the injection pulse width and ignition advance angle are adjusted in real time according to the new strategy parameters, so that the engine can maintain stable operation under changes in fuel type, ambient temperature and load, forming a cross-condition adaptive ignition optimization control result.

[0128] Example 1:

[0129] To verify the feasibility of this invention in practice, it was applied to a fully digital electronic fuel injection system control experiment on an engine test bench. The test system consists of an engine electronic control unit, a fuel injection module, an ignition control module, a sensor acquisition module, and a data monitoring platform. The engine displacement is 2.0L, rated speed is 6000 rpm, and maximum torque is 230 Nm. The test conditions cover four operating states: idle, partial load, high speed, and high temperature, with an ambient temperature range of −10℃ to 45℃. Traditional systems use a lookup-based fuel injection and ignition control method, while this invention employs an improved near-end strategy optimization algorithm model to achieve adaptive joint optimization of fuel injection pulse width and ignition advance angle.

[0130] During the experiment, the engine control unit collected real-time data on engine speed, load, intake pressure, throttle opening, coolant temperature, and knock signals, and input these data into the improved algorithm model of this invention for dynamic strategy generation. The system automatically adjusts the fuel injection quantity and ignition timing under different operating conditions to ensure a balance between combustion stability and energy output. Through online training and multiple rounds of iterative optimization, the model converged after 2000 decision loops, with a strategy update cycle of 0.2 seconds, enabling real-time self-learning and rapid response. Compared with traditional fixed-mapping control methods, this invention demonstrates improvements in combustion efficiency, fuel economy, and emission performance.

[0131] Experimental results show that, under the same engine speed and load conditions, the method of this invention can increase the average combustion efficiency from 37.5% to 41.8% under traditional control, reduce fuel consumption from 7.6 liters / 100 km to 6.8 liters / 100 km, decrease the knock rate from 5.3% to 1.2%, and reduce carbon monoxide and hydrocarbon emissions by 23.4% and 19.7%, respectively. In cross-condition testing, the model can maintain stable output within a temperature fluctuation range of ±15℃ and a load change of 30%, with an ignition advance angle error not exceeding 0.8° crankshaft angle and a response delay of less than 50 milliseconds. These results demonstrate that this invention can effectively solve the problem of traditional electronic fuel injection control strategies relying on static calibration and being difficult to adapt to changing operating conditions, achieving intelligent, dynamic, and adaptive optimization of engine fuel injection and ignition strategies.

[0132] Table 1. Comparison of experimental performance of the reinforcement learning-based all-digital EFI dynamic ignition optimization method.

[0133] Operating conditions Rotational speed (r / min) load(%) Coolant temperature (°C) Average combustion efficiency (%) Fuel consumption (L / 100km) Knock rate (%) CO emissions (g / km) HC emissions (g / km) Response latency (ms) Ignition advance angle error (°CA) Idle condition 850 15 85 35.8→39.6 8.2→7.5 3.8→1.5 0.91→0.69 0.18→0.14 72→48 1.3→0.7 Partial load 2000 45 88 37.5→41.8 7.6→6.8 5.3→1.2 0.84→0.64 0.21→0.17 68→46 1.1→0.8 High-speed operating conditions 4500 80 92 39.2→43.9 8.4→7.2 4.7→1.0 0.97→0.74 0.25→0.19 61→45 1.2→0.6 High temperature working conditions 2500 60 105 36.9→41.1 8.0→6.9 6.2→1.3 0.89→0.68 0.23→0.18 66→47 1.5→0.8 Low temperature conditions 2000 50 −5 34.7→38.5 8.8→7.8 7.1→1.8 1.02→0.79 0.27→0.21 74→52 1.6→0.9

[0134] As can be seen from Table 1 above, the present invention improves upon traditional lookup table control methods in several key performance indicators of combined engine injection and ignition control. Firstly, regarding combustion efficiency, the method of the present invention improves the average combustion efficiency by approximately 3.8 to 4.7 percentage points under five operating conditions: idling, partial load, high speed, high temperature, and low temperature. Specifically, under partial load, the efficiency increases from 37.5% to 41.8%, and under high temperature, it increases from 36.9% to 41.1%, effectively improving fuel utilization and enhancing engine thermal efficiency. Secondly, regarding fuel consumption control, the present invention introduces an improved near-end strategy optimization algorithm, enabling real-time adaptive adjustment of the injection pulse width. Overall fuel consumption decreases from 7.6 liters / 100 km to 6.8 liters / 100 km, with an average fuel saving rate of approximately 10.5%, demonstrating a significant fuel economy advantage.

[0135] Regarding knock suppression performance, the reinforcement learning model of this invention achieves a stable combustion process by dynamically adjusting the ignition advance angle and injection timing, reducing the knock rate from 5.3% under traditional control to 1.2%, with a maximum reduction of 78%. Furthermore, this invention also demonstrates outstanding performance in emission control, reducing carbon monoxide and hydrocarbon emissions by 23.4% and 19.7% respectively, improving engine emission characteristics and indicating that this method can effectively reduce pollutant formation while improving combustion efficiency.

[0136] In terms of dynamic response, this invention reduces the strategy output delay from an average of 68 milliseconds in traditional control to 47 milliseconds through parallel optimization of multiple operating conditions and real-time parameter updates. The ignition advance angle control error decreases from 1.3° crankshaft angle to 0.8° crankshaft angle, demonstrating high response sensitivity and control stability. Comprehensive analysis shows that the all-digital electronic fuel injection dynamic ignition optimization method based on the improved near-end strategy optimization algorithm can achieve coordinated adaptive optimization of fuel injection and ignition control under complex operating conditions, improving engine energy efficiency, control accuracy, and emission performance. This verifies the practicality and technological advancement of this invention in the field of intelligent engine control.

[0137] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fully digital electronic fuel injection dynamic ignition optimization method based on reinforcement learning, characterized in that, Includes the following steps: Collect engine operating data, preprocess the operating data, and obtain a preprocessed operating status dataset; Based on preprocessed runtime data, an improved near-end policy optimization algorithm model is established, and the parameter mapping relationship between the state input, action output, comprehensive reward function, meta-parameter layer and task adaptive layer is set. Based on the improved near-end policy optimization algorithm model, multi-condition task sets are sampled from different working condition data, and inner-layer policy optimization is performed to obtain a set of task adaptive policy parameters. Aggregate the set of task adaptive policy parameters, calculate the average loss across tasks, and update the outer meta-parameters to obtain the global meta-policy parameters; The global meta-strategy parameters are deployed to the engine electronic control unit, and dynamic ignition control results are output based on preprocessed operating status data. Based on the dynamic ignition control results, the instantaneous reward value is calculated and fed back to the improved near-end strategy optimization algorithm model to obtain the updated control strategy parameters. By continuously iterating and optimizing the improved proximal strategy optimization algorithm model using the updated control strategy parameters, adaptive ignition optimization control results across operating conditions are generated.

2. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The preprocessing includes noise reduction, outlier removal, normalization, and moving average filtering.

3. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The improvement of the near-end policy optimization algorithm model based on preprocessed runtime data, and the setting of the parameter mapping relationship between the state input, action output, comprehensive reward function, meta-parameter layer, and task adaptive layer, specifically include: The preprocessed running status data is organized into a time series form of state vector, and a corresponding action output vector is set to establish an improved near-end policy optimization algorithm model. The improved near-end policy optimization algorithm model includes an outer meta-parameter layer and an inner task adaptation layer. The parameters of the outer meta-parameter layer are denoted as meta-parameters, the policy network parameters of the inner task adaptation layer are denoted as policy parameters, the value network parameters are denoted as value parameters, and the initial parameters of the inner task adaptation layer are set as the parameter values ​​of the outer meta-parameter layer. Define the policy function and value function in the improved proximal policy optimization algorithm model, and set the action output space of the policy function and the output estimation range of the value function; A comprehensive reward function is set up, with combustion efficiency, fuel consumption index, emission index and knock rate as input parameters for reward calculation. Weight coefficients are set according to the importance of each index to calculate the instant reward value. Based on the policy function, calculate the policy probability ratio and the estimated value of the advantage function at the current time, and establish the pruning objective function for near-end policy optimization; The value loss function is obtained by calculating the squared difference between the value function prediction result and the return estimate. The strategy entropy term is obtained by calculating the entropy value of the strategy output probability distribution. The pruning objective function, value loss function, and strategy entropy term are weighted and summed according to preset weight coefficients to form the overall objective function; Based on the policy output of multiple working conditions, the gradient update results of each task are calculated, and the updated meta-parameters are obtained by performing aggregate update through the outer meta-parameter layer. The updated meta-parameters are used to reinitialize the policy parameters of the task adaptation layer, mapping the values ​​of the meta-parameters to the parameter space of the task adaptation layer, thus establishing a correspondence between the meta-parameters and the task adaptation layer.

4. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The improved near-end policy optimization algorithm model samples multi-condition task sets from different working condition data, performs inner-layer policy optimization, and obtains a set of task adaptive policy parameters, specifically including: Based on the improved near-end strategy optimization algorithm model, several sets of sample working conditions are selected from the working condition data of different speeds, different loads and different coolant temperature combinations to form a multi-working-condition task set. For each task, the corresponding state vector is input into the task adaptation layer, and the corresponding action sequence is output using the current policy function. This sequence interacts with the environment to form a state transition sequence. The cumulative reward value is obtained by weighting and summing the instantaneous rewards obtained in multiple time steps of the task according to the time sequence. The policy parameters are updated based on the pruning objective function. The policy parameters are adjusted along the gradient direction of the optimization objective with the policy learning rate as the step size to obtain the updated policy parameters. The strategy parameters obtained after the strategy update for each task are summarized and arranged in order according to the task number to form a set of task adaptive strategy parameters.

5. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The aggregation of the task adaptive policy parameter set, calculation of the average loss across tasks, and updating of the outer meta-parameters to obtain the global meta-policy parameters specifically include: Receive a set of adaptive strategy parameters for the task, with each task under a working condition corresponding to an independent strategy parameter item; Based on the overall objective function, the pruning objective function, value loss function, and strategy entropy term are weighted and summed according to preset weight coefficients to calculate the overall objective function value for each task. The average loss across all task conditions is obtained by averaging and aggregating the total objective function values ​​for all task conditions. The gradient update results of the meta-parameters are calculated based on the average loss. The values ​​of the meta-parameters are then adjusted in the opposite direction of the gradient of the average loss function according to the step size determined by the meta-learning rate, so as to obtain the updated meta-parameters.

6. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The step of deploying global meta-strategy parameters to the engine electronic control unit and outputting dynamic ignition control results based on preprocessed operating status data specifically includes: The global meta-policy parameters are imported into the engine electronic control unit to initialize the policy network built by the improved proximal policy optimization algorithm model inside the electronic control unit; The pre-processed operating status data is input into the strategy network in the electronic control unit in real time, and the strategy function is used to calculate the output action based on the input status. The output action is converted into an execution signal by the instruction parsing module and transmitted to the fuel injection control module and the ignition control module respectively, driving the fuel injector and the ignition coil to perform the corresponding fuel injection and ignition operations. During engine operation, the output sequences of fuel injection pulse width control and ignition advance angle control are recorded step by step to form a dynamic control command sequence. The engine feedback data after executing the dynamic control command sequence is collected, and the feedback data is matched with the dynamic control command to generate dynamic ignition control results.

7. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The process of calculating the instantaneous reward value based on the dynamic ignition control results and feeding it back to the improved near-end strategy optimization algorithm model to obtain the updated control strategy parameters specifically includes: Four types of feedback indicators—combustion efficiency, fuel consumption, emission concentration, and knock rate—are extracted from the dynamic ignition control results. The real-time data of each feedback indicator are normalized to form standardized feedback signals. The instant reward value at each time step is calculated based on the comprehensive reward function. The instant reward value is combined with the corresponding state vector, action vector and next state vector to form a state transition sample and stored in the experience buffer. The state transition samples are input into the improved proximal policy optimization algorithm model to calculate the advantage function estimate and optimize the policy based on the pruning objective function, thereby updating the policy network parameters. The updated strategy parameters are written into the electronic control unit to replace the original parameters, thus forming the updated control strategy parameters.

8. The method for optimizing dynamic ignition of fully digital electronic fuel injection based on reinforcement learning according to claim 1, characterized in that, The improved near-end strategy optimization algorithm model, which utilizes updated control strategy parameters for continuous iterative optimization, to form cross-condition adaptive ignition optimization control results, specifically includes: During long-term engine operation, the improved proximal strategy optimization algorithm model is continuously trained iteratively by calling the updated control strategy parameters. The collected operational data is normalized and averaged to form a continuous time series input, and the corresponding action sequence is output based on the current policy function. The instant reward value for each time step is calculated based on the actual operation feedback information of the engine. The rewards obtained in consecutive time steps are weighted and accumulated in chronological order to obtain the cumulative reward value corresponding to each time step. The advantage function value is calculated based on the cumulative return value, and the updated strategy parameters are obtained by iteratively optimizing the strategy parameters in conjunction with the pruning objective function. The updated strategy parameters are written into the electronic control unit, replacing the original strategy parameters. The output of the injection pulse width and ignition advance angle are adjusted in real time according to the new strategy parameters to form an adaptive ignition optimization control result across operating conditions.

Images