Battery Charging Control Using Reinforcement Learning and Aging Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing battery charging technologies rely heavily on electro-chemical models that struggle to accurately predict battery aging, leading to discrepancies between simulated and real-world performance, resulting in faster-than-predicted degradation.
Innovation Solution
A hybrid model combining electro-chemical models with reinforcement learning, utilizing neural networks to adapt charging strategies based on real-world data, optimizing battery health through state-of-health variables and charge time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If electro-chemical models are used to optimize battery charging, then charging strategy can be determined, but prediction accuracy of battery aging deteriorates due to substantial differences between model and real batteries
Solution Approach 1:
The patent introduces reinforcement learning algorithms as an intermediary between the electro-chemical model and real battery behavior. The RL agent learns from simulation data generated by the electro-chemical model and field data from real batteries, acting as a mediator that translates model predictions into accurate real-world charging strategies without requiring direct complex modeling of all real battery variations
Solution Approach 2:
The patent creates a virtual copy of the battery system through reinforcement learning training in simulated environments. The RL agent learns optimal charging policies by interacting with a virtual battery model, then transfers this learned knowledge to control real batteries, avoiding the need to directly model all complex real-world variations
2Adaptability or versatility
If reinforcement learning is used to optimize battery charging, then adaptability to real-world conditions improves, but training consistency deteriorates due to inconsistent or bizarre results
Solution Approach 1:
The patent performs preliminary training of the reinforcement learning agent in a controlled simulation environment before deploying to real batteries. The agent learns basic charging policies and explores the state space in advance, which stabilizes subsequent real-world training and prevents inconsistent results by establishing a consistent baseline behavior beforehand
Solution Approach 2:
The patent implements a feedback mechanism where the reinforcement learning agent receives continuous rewards based on battery health outcomes and charging efficiency. This feedback loop allows the agent to learn from actual results and adjust its policy, improving training consistency by providing clear directional guidance rather than relying on inconsistent exploration alone
3Ease of operation
If electro-chemical models are used for battery charging optimization, then charging control can be achieved, but measurement precision of battery state deteriorates due to difficulty in directly measuring factors influencing battery life
Solution Approach 1:
The patent replaces direct physical measurement of difficult-to-measure battery states (such as internal degradation processes) with reinforcement learning-based estimation. The RL agent infers battery state from observable charging responses and performance data, substituting direct measurement with intelligent inference based on learned patterns
Data Source
AI summary
Methods and systems of optimizing battery charging are disclosed. Battery state sensors are used to determine anode overpotential of a battery multiple times during multiple charge cycles. In a first phase, a reinforcement learning model (e.g., actor-critic model) is trained with rewards given throughout each charge cycle of the battery to optimize training. The reinforcement learning model can determine state-of-health characteristics of the battery over the charge cycles, and in a second phase, the reinforcement learning model is augmented accordingly. During this augmentation, the reinforcement learning model is trained with rewards given on a charge cycle-by-cycle basis, wherein rewards are given after looking at the charging optimization after the conclusion of each charge cycle. Commands are given to charge an on-field battery based on the augmented reinforcement learning model, associated state-of-health characteristics of the on-field battery are determined, and the reinforcement learning model is further augmented accordingly.