Motor Control Unit Reinforcement Learning for Engine Cranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motor control systems struggle to quickly start an engine while minimizing vibration and noise during the cranking process, as they rely on traditional learning methods that do not effectively optimize torque command values in real-time.
Innovation Solution
A reinforcement learning method that optimizes a Q table by repeatedly simulating engine cranking processes, using sound pressure and acceleration data to calculate rewards and update the Q table, thereby selecting actions that maximize the Q value and reduce noise and vibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional motor control methods are used, then the control system is simple, but the engine cannot be started quickly and vibration and noise are not suppressed
Solution Approach 1:
The patent replaces traditional mechanical control systems with a reinforcement learning-based intelligent control system. The Q table stores optimized torque command values learned through reinforcement learning, allowing the system to quickly start engines while suppressing vibration and noise without complex real-time calculations during actual operation.
Solution Approach 2:
The patent performs preliminary learning and optimization offline to create the Q table before actual engine starting operations. The reinforcement learning agent learns optimal torque commands by simulating numerous cranking scenarios in advance, storing the results in the Q table. During actual operation, the system simply queries the pre-computed Q table for the appropriate torque command based on current engine state, eliminating the need for complex real-time optimization.
2Object-affected harmful factors
If reinforcement learning with multiple sensors is used, then vibration and noise are suppressed, but the device complexity increases
Solution Approach 1:
The patent implements feedback mechanisms where the reinforcement learning agent uses sensor data (engine rotation speed, torque command values) to evaluate the effectiveness of its actions. The agent receives reward signals based on cranking performance and vibration/noise levels, allowing it to learn optimal control strategies that suppress harmful factors while maintaining system manageability through structured feedback loops.
Solution Approach 2:
The patent performs all complex learning and optimization calculations offline during the training phase, creating the Q table in advance. During actual operation, the system simply queries the pre-computed Q table based on current sensor readings, eliminating the need for complex real-time reinforcement learning calculations and reducing operational system complexity.
3Productivity
If real-time torque optimization is performed, then cranking efficiency is improved, but calculation time increases
Solution Approach 1:
The patent performs all complex torque optimization calculations offline during the reinforcement learning training phase. The agent explores numerous cranking scenarios and learns optimal torque commands in advance, storing the results in the Q table. During actual engine starting operations, the system simply queries the pre-computed Q table for the appropriate torque command based on current engine state, eliminating the need for time-consuming real-time optimization calculations.
Solution Approach 2:
The patent creates a lookup table (Q table) that copies and stores pre-computed optimal torque command values for various engine states. During actual operation, the system retrieves the appropriate torque command by copying the relevant entry from the Q table based on current sensor readings, replacing complex real-time calculations with simple table lookup operations that are computationally inexpensive and time-efficient.
Data Source
AI summary
A reinforcement learning method optimizes a Q table through reinforcement learning. The reinforcement learning method includes causing a computer to repeatedly calculate a first reward based on sound pressure or acceleration in each trial, calculate a second reward such that a return decreases as a time until starting of an engine is completed increases, and update the Q table such that an action is selected with which a return that is the sum of the rewards in the trials becomes larger.