Method for automatic adjustment of power grid operation mode base on reinforcement learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional manual adjustment methods for power grid operation modes are time-consuming and labor-intensive, and traditional reinforcement learning models struggle with high exploration costs and ineffective operation modes due to uncertainties in renewable energy systems, failing to meet convergence requirements and leading to inefficient power grid adjustments.
Innovation Solution
A method combining an expert system with reinforcement learning to optimize power grid operation modes by determining active power adjustments for thermal units, redistributing power flow, and adjusting unit terminal voltage, using an active power-line load rate sensitivity matrix for rapid overload identification, and employing a comprehensive evaluation index to ensure safe and stable operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If traditional reinforcement learning models are used for automatic adjustment of power grid operation modes, then automation is improved, but exploration costs increase dramatically due to exponential growth of state and action spaces
Solution Approach 1:
The patent segments the large state and action spaces into smaller, manageable subspaces by categorizing system nodes into different types (e.g., generator nodes, load nodes, transmission nodes) and applying targeted exploration strategies to each segment. This reduces the overall exploration burden while maintaining comprehensive coverage of the power grid operation space.
Solution Approach 2:
The patent changes the parameter representation of the power grid state by using aggregated metrics (e.g., total active power, total reactive power, voltage levels) instead of individual node parameters. This parameter transformation reduces the dimensionality of the state space and enables more efficient reinforcement learning exploration.
2Extent of automation
If traditional reinforcement learning models are used for automatic adjustment of power grid operation modes, then automation is improved, but exploration efficiency decreases due to randomly generated modes failing to meet convergence requirements
Solution Approach 1:
The patent applies preliminary actions by pre-processing the power grid state to ensure feasibility before reinforcement learning exploration. This includes checking convergence requirements, validating operation mode feasibility, and adjusting initial states to meet power flow constraints. Such preliminary actions prevent wasted exploration on invalid modes and improve overall exploration efficiency.
Solution Approach 2:
The patent introduces an intermediary layer between the reinforcement learning agent and the power grid system. This intermediary validates and filters generated operation modes, ensuring they meet convergence requirements before being applied to the actual system. It acts as a mediator that translates raw RL outputs into feasible grid operations.
3Stability of the object's composition
If manual adjustment methods are used for power grid operation modes, then operation stability is maintained, but time consumption and labor intensity increase significantly
Solution Approach 1:
The patent implements self-service by enabling the power grid system to automatically adjust its own operation modes through reinforcement learning. The system learns from historical data and real-time measurements to autonomously optimize generation dispatch, load management, and transmission operations, eliminating the need for manual intervention while maintaining operational stability through learned policies.
Solution Approach 2:
The patent incorporates feedback mechanisms where the reinforcement learning model continuously receives information about system performance, constraint violations, and operational outcomes. This feedback loop enables the system to learn from past decisions, adjust future actions, and maintain stability while adapting to changing grid conditions, thereby replacing manual adjustment with intelligent automation.
Data Source
AI summary
A method for automatic adjustment of a power grid operation mode based on reinforcement learning is provided. An expert system for automatic adjustment is designed, which relies on the control sequence of thermal power units, enabling automatic decision-making for power grid operation mode adjustment. A sensitivity matrix is extracted from the historical operating data of the power grid, from which a foundational thermal power unit control sequence is derived. An overload control strategy for lines within the expert system is devised. A reinforcement learning model optimizes the thermal power unit control sequence, which refines the foundational thermal power unit control sequence and provides the expert system with the optimized control sequence for automatic decision-making in power grid operation mode adjustment. This method offers a solution to balancing and absorption challenges brought about by fluctuations on both the supply and demand sides in high-proportion renewable energy power systems.