Virtual RL Calibration for Vehicle State Zone Transitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current reinforcement learning systems face challenges in calibrating vehicle parameters effectively within virtual environments, particularly in managing transitions between adjacent operation state space zones and ensuring convergence within predetermined computation or time budgets.
Innovation Solution
The system employs a reinforcement learning agent to generate calibration parameters within a simulated environment, dividing the operation state space into zones when convergence is not achieved, and utilizes a supervisor reinforcement learning agent to manage transitions between these zones by calculating transition calibration parameters using coefficients generated by individual agents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the operation state space is divided into multiple zones with separate reinforcement learning agents, then the calibration precision for different driving situations is improved, but the device complexity increases due to multiple agents and transition management
Solution Approach 1:
The operation state space is divided into multiple adjacent zones, each corresponding to a specific driving situation. A separate reinforcement learning agent is trained for each zone to optimize calibration parameters for that specific situation, thereby improving calibration precision while managing complexity through structured division
Solution Approach 2:
A supervisor reinforcement learning agent is introduced as an intermediary to manage transitions between adjacent zones. The supervisor agent receives outputs from multiple zone-specific agents and generates transition calibration parameters using a weighted combination formula, smoothing transitions and reducing parameter jumps while maintaining overall system coordination
2Productivity
If reinforcement learning agents are trained for predetermined computation or time budgets, then the productivity of parameter calibration is improved, but the manufacturing precision of calibration parameters may be reduced due to limited training time
Solution Approach 1:
Multiple reinforcement learning agents are trained in advance during an offline phase with predetermined computation or time budgets. This preliminary training prepares the agents for rapid deployment in production environments, balancing training efficiency with acceptable calibration accuracy by completing most learning before actual use
Solution Approach 2:
The outputs of multiple pre-trained reinforcement learning agents are merged through a weighted combination formula in the supervisor agent. This merging approach allows the system to leverage knowledge from multiple agents trained under budget constraints, achieving better overall calibration accuracy than any single agent could provide alone
Data Source
AI summary
A system is disclosed that includes a computer including a processor and a memory. The memory including instructions such that the processor is programmed to: generate a simulated environment, the simulated environment representing a plurality of driving situations, and generate, via a reinforcement learning agent, at least one calibration parameter based on simulated vehicle operations within a simulated environment.