ESS Reward Control Using Reinforcement Learning for Peak Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional energy storage system (ESS) scheduling control methods fail to optimally manage peak load of power consumption due to reliance on predetermined seasonal load times, neglecting actual building-specific fluctuations and seasonal changes in power usage patterns.
Innovation Solution
A reinforcement training model is employed to generate optimal control information for an ESS using power consumption data, setting reward functions to incentivize charging as a loss and discharging as a gain, or both actions as gains, to maximize rewards and minimize peak load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional scheduling control method is used with predetermined seasonal load time, then the control system is simple to implement, but it cannot adapt to individual building variations and seasonal changes in power usage patterns
Solution Approach 1:
The system uses reinforcement learning to enable the ESS control system to automatically learn and adapt to individual building power consumption patterns without requiring expert intervention. The training model autonomously processes monitored power consumption data and generates optimal control strategies, making the system self-adjusting to variations in building usage patterns and seasonal changes.
Solution Approach 2:
The system changes the control parameters dynamically by using a trained reinforcement learning model that adjusts ESS operation based on learned patterns from historical power consumption data. Instead of fixed seasonal schedules, the system adapts parameters like charging/discharging timing and magnitude based on building-specific patterns, achieving adaptability without proportionally increasing system complexity.
2Manufacturing precision
If expert analysis of monitored power consumption data is performed, then optimal control of ESS can be achieved for specific buildings, but it requires significant time and resources for data analysis and control strategy development
Solution Approach 1:
The system performs preliminary action by pre-training a reinforcement learning model using historical power consumption data before actual ESS operation begins. This offline training phase captures building-specific power usage patterns, enabling the system to automatically generate optimal control strategies during operation without requiring real-time expert analysis. The preprocessing of data into training sets and the subsequent automated model application eliminate the need for time-consuming expert intervention during actual peak load management.
3Adaptability or versatility
If reinforcement training model is used to generate optimal control information, then adaptability to varying power usage patterns is improved, but computing power consumption and processing time increase
Solution Approach 1:
The system performs the computationally intensive reinforcement learning training in advance during an offline phase using historical power consumption data. Once trained, the model generates control information efficiently during actual operation by simply processing current power consumption readings through the already-trained algorithm. This separates the heavy computational workload from real-time operation, reducing the computing power consumption during peak load management while maintaining high adaptability to seasonal changes and building-specific patterns.
Data Source
AI summary
A reward generating method for reducing peak load of power consumption and a computing device for performing the same are provided. The reward generating method for reducing the peak load of the power consumption includes calculating an energy index according to a predetermined time interval using power consumption data of a specific building, to which an energy storage system (ESS) is applied, during a predetermined period, determining a reward index according to an action of the ESS using a reward function generated based on the energy index, and training a training model to which the reward function is applied through a reward generated based on the reward index.