MADDPG Grid Frequency Control with Large-Scale Energy Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing scale of renewable energy generation, particularly from wind and photovoltaic sources, poses challenges to frequency stability in power grids. Conventional frequency regulation units, mainly thermal power units, are less effective in new-type power systems, limiting the development of new energy grid connections. Additionally, the complexity of random load fluctuations and real-time frequency deviations in power grids requires more advanced control methods to ensure accurate frequency regulation.
Innovation Solution
A control platform and method for power grid frequency regulation using large-scale energy storage based on the Multiple Agents Deep Deterministic Policy Gradient (MADDPG) algorithm. This three-layer architecture includes a state space layer for data monitoring, an interaction transfer layer for data processing and control instruction execution, and a control command layer for optimal control policy execution. The MADDPG algorithm enables synergistic control between different control objects, reducing computation time and achieving quick responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional frequency regulation units (thermal power units) are used, then the system structure is simple and easy to operate, but the frequency regulation effectiveness gradually decreases and cannot meet the control accuracy requirements in complex frequency regulation scenes
Solution Approach 1:
The control system is segmented into multiple independent agents, each responsible for specific control objects (thermal power units, energy storage devices, etc.). Each agent independently learns and executes control policies, allowing the system to handle complex frequency regulation tasks through distributed intelligence rather than a monolithic controller.
Solution Approach 2:
The patent replaces conventional mechanical control systems with an intelligent software-based MADDPG algorithm. The control decisions are made through neural network policies that process system states and generate control actions, substituting traditional mechanical governors and control mechanisms with data-driven intelligent agents.
2Reliability
If conventional control methods are used in large-scale multiple-control-object scenes, then the device complexity is low, but the coordination of control strategies between control objects is ineffective
Solution Approach 1:
The patent merges multiple individual control agents into a unified MADDPG framework where agents share a common learning environment and policy structure. This allows coordinated control strategies to emerge through the interaction of multiple agents that all optimize for the same frequency regulation objective, achieving effective coordination without requiring complex centralized control.
Solution Approach 2:
The MADDPG algorithm provides a universal control framework that can handle multiple types of control objects (thermal power units, energy storage devices, renewable energy sources) through a single integrated system. The same algorithmic approach works across different device types, providing multi-functional coordination capability.
3Measurement precision
If conventional frequency regulation policies are used, then the computation time is acceptable, but the control accuracy requirement cannot be satisfied in complex frequency regulation scenes with large-scale energy storage
Solution Approach 1:
The control agents undergo offline training using the MADDPG algorithm to learn optimal control policies before actual frequency regulation events occur. This preliminary learning phase allows the agents to internalize complex control strategies, so that during real-time operation, they can execute pre-learned policies with minimal computation, achieving both high accuracy and fast response.
Solution Approach 2:
The patent uses simulation environments to create virtual copies of the power system for training purposes. Agents learn control policies in the simulated environment before deploying to the real system, allowing extensive computation and learning to occur in the virtual copy without impacting real-time operational performance.
Data Source
AI summary
The present disclosure relates to the field of frequency control technologies of power grid systems and in particular to a platform and method for power grid frequency regulation with participation of large-scale energy storage based on MADDPG. Firstly, based on a scene of power grid frequency regulation with participation of large-scale energy storage, a control platform architecture for power grid frequency regulation with participation of large-scale energy storage based on MADDPG is designed. Then, based on the multiple agents deep deterministic policy gradient algorithm, under the drive of environmental interaction data, training and learning are performed to obtain optimal control multiple agents for power grid frequency regulation with participation of energy storage. Finally, by using the optimal control multiple agents, control on the output power of the thermal power unit and the charge and discharge of the large-scale energy storage is performed for participation of power grid frequency regulation.