Deep Reinforcement Learning for Engine Air Handling and Fuel System Referencing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing engine tuning techniques, such as model predictive control and engine mapping, are computationally expensive and struggle to accurately predict engine behavior in dynamic conditions, leading to suboptimal performance and increased dependency on initial calibration models, especially during transient events.
Innovation Solution
The implementation of deep reinforcement learning for air handling and fuel system referencing, which uses a reference managing unit with sensors and a processing unit that includes an agent trained to determine policy functions for air handling and fuel system references, allowing for real-time optimization and reduced dependency on initial model accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If model predictive control (MPC) is used for engine tuning, then prediction accuracy can be improved, but computational cost becomes prohibitively expensive and cannot be implemented in engine control units
Solution Approach 1:
The patent creates a simplified copy of the complex MPC system by training a neural network to replicate MPC's prediction capabilities. The neural network is trained offline using MPC-generated data, then deployed as a lightweight model that mimics MPC behavior without requiring its computational resources, enabling implementation in embedded control units.
Solution Approach 2:
The patent performs preliminary computation by training the neural network offline before deployment. All heavy computational work is done in advance during the training phase, allowing the trained model to make rapid predictions during actual engine operation without real-time computational burden.
2Device complexity
If linear functions are used to approximate dynamic models in MPC, then computational complexity is reduced, but the system fails to capture actual dynamic behavior leading to poor performance
Solution Approach 1:
The patent transforms the system from using fixed linear parameters to using adaptive neural network parameters. The neural network learns optimal parameters during training that capture non-linear dynamic behavior, allowing the system to maintain low computational complexity during operation while achieving high fidelity in modeling dynamic engine behavior.
3Ease of manufacture
If offline calibration techniques are used to establish steady-state operating maps, then initial engine setup can be completed, but the system cannot adapt to real operating conditions that differ from calibration scenarios
Solution Approach 1:
The patent implements feedback by continuously comparing neural network predictions with actual engine sensor measurements during operation. The system uses this feedback to correct discrepancies between predicted and actual behavior, enabling adaptive adjustment to real operating conditions while maintaining the benefits of offline calibration.
Solution Approach 2:
The patent transitions from static offline calibration maps to a dynamic adaptive system. The neural network model is designed to adapt its predictions based on current operating conditions, allowing the system to evolve from fixed calibration data to real-time dynamic adjustment that responds to changing engine states and environmental conditions.
4Manufacturing precision
If transient state calibration is performed via trial-and-error duty cycles, then some transient performance can be optimized, but not all duty cycles can be tested leading to suboptimal performance
Solution Approach 1:
The patent uses the neural network as a virtual copy of the engine system that can be tested extensively during training without physical constraints. This digital twin approach allows evaluation of all possible duty cycles and transient conditions during the offline training phase, identifying optimal control strategies that would be impossible to discover through limited physical trial-and-error testing.
Data Source
AI summary
An engine system includes an air handling and fuel system whose states are managed by a reference managing unit. The engine system has a plurality of sensors whose sensor signals at least partially define a current state of the engine system. The reference managing unit includes a controller which controls the air handling and fuel system of the engine system as well as a processing unit coupled to the sensors and the controller. The processing unit includes an agent which learns a policy function that is trained to process the current state, determines air handling references and fuel system references by using the policy function after receiving the current state as an input, and outputs the air handling references and fuel system references to the controller. Then, the agent receives a next state and a reward value from the processing unit and updates the policy function using a policy evaluation algorithm and a policy improvement algorithm based on the received reward value. Subsequently, the controller controls the air handling and fuel system of the engine in response to receiving the air handling references and the fuel system references.