Power grid dispatching decision-making method based on interpretable deep reinforcement learning

Abstract

The invention discloses a power grid dispatching decision-making method based on interpretable deep reinforcement learning, and belongs to the technical field of combination of a power system and artificial intelligence, and the method comprises the steps: obtaining new energy output data, load data and flexible resource state data, and constructing a power grid multi-source data set; aiming at safe operation requirements of a power grid, constructing operation constraints of a power system and flexible resources; establishing a power grid dispatching model based on a Markov decision process for a power grid dispatching decision demand; training a power grid dispatching decision-making agent by adopting an interpretable deep reinforcement learning theory; and generating a power grid real-time scheduling strategy in a rolling manner through the explainable deep reinforcement learning scheduling decision-making agent obtained through training. The method enhances the interpretability of the decision-making process while realizing efficient generation of the scheduling strategy, and can better support the safe and reliable operation of the novel power system.

Description

Technical Field

[0001] The present invention belongs to the technical field of the combination of power systems and artificial intelligence, and specifically relates to a power grid dispatching decision method based on explainable deep reinforcement learning. Background Art

[0002] As the global energy industry gradually transforms from fossil energy to sustainable solutions, new energy is increasingly becoming the mainstream choice for energy production due to its abundance and environmentally friendly characteristics. In this context, the dispatching and management of the power grid, as a key link between energy production and consumption, faces unprecedented challenges and opportunities. Especially in the context of large-scale grid connection of renewable energy, the optimization of grid dispatching is not only the key to improving energy efficiency, but also an important means to promote energy transformation. Therefore, how to improve the acceptance capacity of renewable energy and optimize energy configuration while ensuring the safe and stable operation of the power grid has become a core task facing the power system.

[0003] In recent years, dispatch optimization methods based on deep reinforcement learning have demonstrated outstanding capabilities in power grids, especially in scenarios dealing with system uncertainty and complexity. However, deep reinforcement learning faces the problem of black box models, so that it is difficult to extend it to actual power grid dispatch applications. Therefore, how to improve the decision transparency of deep reinforcement learning is one of the main issues in the field of combining power systems with artificial intelligence. Summary of the invention

[0004] In order to solve the above-mentioned technical problems existing in the prior art, the purpose of the present invention is to provide a power grid dispatching decision-making method based on interpretable deep reinforcement learning, which can enhance the interpretability of the decision-making process while realizing efficient generation of dispatching strategies.

[0005] To achieve the above-mentioned purpose of the invention, the present invention provides a power grid dispatch decision method based on interpretable deep reinforcement learning, comprising the following steps:

[0006] S110, obtaining new energy output data, load data and flexibility resource status data, and constructing a multi-source data set for the power grid;

[0007] S120. Construct the operation constraints of the power system and flexibility resources according to the requirements of safe operation of the power grid;

[0008] S130. Aiming at the needs of power grid dispatching decision-making, a power grid dispatching model based on Markov decision process is established;

[0009] S140, using interpretable deep reinforcement learning to train power grid dispatch decision-making agents;

[0010] S150. The interpretable deep reinforcement learning dispatch decision-making agent obtained through training is used to generate real-time dispatch strategies for the power grid in a rolling manner.

[0011] Furthermore, the method also includes: collecting and preprocessing status data from new energy output, load and flexibility resources to obtain a multi-source data set. Among them, the new energy output data usually comes from the power generation of new energy sources such as photovoltaics and wind power, which are volatile and uncertain. The load data includes the electricity demand of residential, industrial and commercial users. The flexibility resource status data includes real-time status information of dispatchable thermal power units, battery energy storage and other resources. After data collection, the preprocessing process includes standardization, missing value filling, time series data alignment, filtering and cleaning of outliers and noise data, etc., to ensure that the multi-source data set has high quality and consistency.

[0012] Furthermore, the method includes: when constructing a power grid dispatching model, the safe operation requirements of the power grid must be considered. This includes not only basic constraints such as power balance and voltage safety of the system, but also constraints on flexible resources to ensure that they can safely participate in the operation of the system during the operation of the power grid. Among them, constraints such as the output limit of the generator set and the capacity limit of the energy storage device need to be clearly constructed in the dispatching model to support the safe and stable operation of the power grid.

[0013] Furthermore, the method includes: defining the state space, action space and reward function in view of the needs of power grid dispatching decision-making, and establishing a power grid dispatching model based on the Markov decision process. Among them, the state space is defined as a multi-source data set obtained by the state data of the new energy output, load and flexibility resources; the action space is defined as the action of each controllable flexibility resource, such as the output size of the generator set and the charging and discharging power of the energy storage device; the reward function is defined as the economic cost of the dispatching decision and the penalty function for violating the operation constraints of the power system and flexibility resources.

[0014] Further, the method includes: using interpretable deep reinforcement learning theory to train a power grid dispatch decision-making agent. Among them, using an interpretable prototype mapping network, multiple interpretable prototypes are connected to corresponding dispatch strategies. The power grid state will calculate the similarity with the interpretable prototype and generate the final dispatch strategy by weighting the corresponding dispatch strategy. Based on the constructed Markov process, the agent is trained so that the agent can maximize the economic benefits of the power grid under the premise of satisfying safety constraints and ensure the interpretability of deep reinforcement learning decisions.

[0015] Furthermore, the method includes: generating a grid dispatching strategy in a rolling manner by combining a trained grid dispatching decision-making intelligent agent with a grid status.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] Based on the multi-source data sets of renewable energy output, load and flexibility resource status, a power grid dispatching decision model based on Markov process is established, and the interpretable deep reinforcement learning theory is used to train the power grid dispatching decision agent. The training obtains an interpretable deep reinforcement learning dispatching decision agent, which realizes the efficient generation of dispatching strategies and enhances the interpretability of the decision-making process, which can better support the safe and reliable operation of the new power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the power grid dispatching decision-making method based on interpretable deep reinforcement learning of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0020] In response to the black box problem of traditional deep reinforcement learning methods, this embodiment provides a power grid dispatching decision-making method based on interpretable deep reinforcement learning, so as to enhance the interpretability of the decision-making process while achieving efficient generation of dispatching strategies.

[0021] Figure 1 is a flow chart of a power grid dispatch decision method based on interpretable deep reinforcement learning provided by an embodiment. Figure 1 As shown, the power grid dispatch decision method provided in the embodiment includes the following steps:

[0022] S110: Acquire new energy output data, load data and flexibility resource status data to build a multi-source data set for the power grid.

[0023] In order to construct a high-quality multi-source data set for the power grid, the status data from renewable energy output, load and flexibility resources are first collected. Among them, renewable energy output data usually comes from the power generation of renewable energy sources such as photovoltaics and wind power, which are volatile and uncertain. Load data includes the power demand of residential, industrial and commercial users. Flexible resource status data includes real-time status information of dispatchable thermal power units, battery energy storage and other resources. After data collection, the preprocessing process includes standardization, missing value filling, time series data alignment, filtering and cleaning of outliers and noise data, etc., to ensure the high quality and consistency of the multi-source data set.

[0024] S120: Construct operation constraints of the power system and flexibility resources based on the requirements for safe operation of the power grid.

[0025] In one embodiment, power system power balance constraints and flexibility resource operation constraints are constructed.

[0026] The power balance constraint of the power grid can be expressed as follows:

[0027]

[0028] in, , and Respectively represent the number of photovoltaic systems, energy storage systems and thermal power units. Indicates time The load demand, Indicates time No. The output power of each photovoltaic unit, Indicates time No. The power of a battery, Indicates time No. The power of a thermal generator, Indicates time The transmission power of the adjacent power grid.

[0029] The operating constraints of thermal power units can be expressed in the following forms:

[0030]

[0031] in, is the operating cost of thermal power units, , and is the cost coefficient of thermal power units; Indicates time Time Active power output of each generator; and Respectively represent the upper and lower limits of the thermal generator output power, and Indicates the climbing limit of the unit; Indicates the time difference of the output change of thermal power units; Indicates The output power of the generator in the previous time step.

[0032] The operating constraints of battery energy storage can be expressed as follows:

[0033]

[0034] in, It's time Time The state of charge of the battery energy storage system is subject to the maximum value of the state of charge. and the minimum state of charge Constraints; Indicates time Time The energy of a battery energy storage system, whose maximum capacity is ; Charging power and discharge power The maximum charging rate and maximum discharge rate restrictions; and are the efficiency factors for charging and discharging, respectively.

[0035] The grid connects adjacent grids to handle energy imbalances caused by the intermittent nature of renewable energy. When there is a power shortage, the grid provides power; when there is excess power generation, the grid absorbs the excess power. Power trading is subject to real-time market prices. control. Indicates time The transmission power of the neighboring grid, where a positive value indicates that power is purchased from the neighboring grid and a negative value indicates that power is sold to the neighboring grid.

[0036] S130: Aiming at the needs of power grid dispatching decision-making, a power grid dispatching model based on Markov decision process is established.

[0037] In one implementation, in view of the power grid dispatch decision-making requirements, the state space, action space and reward function are defined, and a power grid dispatch model based on the Markov decision process is established.

[0038] The state space is defined as the state data of the obtained renewable energy output, load and flexibility resources to obtain a multi-source data set, which can be expressed as follows:

[0039]

[0040] The action space is defined as the actions of each controllable flexibility resource, such as the output of thermal power units and the charging and discharging power of battery energy storage. It can be expressed in the following form:

[0041]

[0042] The reward function is defined as the economic cost of the dispatch decision and the penalty function for violating the operating constraints of the power system and flexibility resources.

[0043]

[0044] in, Indicates time The total operating cost of the grid, Indicates time Unbalanced power in the grid, represents the penalty coefficient for power grid violation constraints, represents the battery energy storage charging cost coefficient, represents the battery energy storage discharge cost coefficient, represents the cost of thermal power generation, Represents the market electricity price.

[0045] S140: Using explainable deep reinforcement learning to train intelligent agents for power grid dispatching decisions.

[0046] In one embodiment, interpretable deep reinforcement learning is used to train a power grid dispatch decision-making agent. In this case, an interpretable prototype mapping network is used to establish a connection between multiple interpretable prototypes and corresponding dispatch strategies. The interpretable prototype mapping network architecture includes four main parts: encoding the state, mapping the encoded state, calculating the similarity, and generating the final action strategy. First, the current state is input into a pre-trained deep reinforcement learning neural network, which encodes the state into a representation ;Then, is passed to multiple prototype branches, each branch corresponds to a specific interpretable prototype, and in each branch, the calculation The similarity with the prototype of the branch is scored; finally, the manually predefined weight matrix These similarity scores are combined and the strategies corresponding to the weighted prototypes are finally generated to generate the control action value. By mapping the current state to the prototype that is most similar to it, the interpretable prototype mapping network architecture is able to generate interpretable grid dispatch strategies.

[0047] Based on the constructed Markov decision process, the intelligent agent is trained so that it can maximize the economic benefits of the power grid while satisfying safety constraints and ensure the interpretability of deep reinforcement learning decisions.

[0048] S150: Through the trained interpretable deep reinforcement learning power grid dispatch decision-making intelligent agent, the real-time power grid dispatch strategy is generated in a rolling manner.

[0049] The power grid dispatching decision-making intelligent agent with interpretable deep reinforcement learning obtained through training can generate real-time power grid dispatching strategies in a rolling manner, which not only realizes the efficient generation of dispatching strategies but also enhances the interpretability of the decision-making process, and can better support the safe and reliable operation of the new power system.

Claims

1. A power grid dispatch decision-making method based on interpretable deep reinforcement learning, characterized in that: The following steps are involved: S110, obtaining new energy output data, load data and flexibility resource status data, and constructing a multi-source data set for the power grid; S120. Construct the operation constraints of the power system and flexibility resources according to the requirements of safe operation of the power grid; S130. Aiming at the needs of power grid dispatching decision-making, a power grid dispatching model based on Markov decision process is established; S140, using interpretable deep reinforcement learning to train power grid dispatch decision-making agents; S150. The interpretable deep reinforcement learning dispatch decision-making agent obtained through training is used to generate real-time dispatch strategies for the power grid in a rolling manner.

2. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 1, characterized in that: In step S110, the new energy output data comes from the new energy power generation situation with volatility and uncertainty; the load data includes the electricity demand of residential, industrial and commercial users; the flexibility resource status data includes the real-time status information of dispatchable thermal power units and battery energy storage resources.

3. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 1, characterized in that: In step S120, the operation constraints of the power system include the power balance constraints of the power grid, and the operation constraints of the flexible resources include the operation constraints of the thermal power units and the operation constraints of the battery energy storage.

4. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 1, characterized in that: In step S130, the state space, action space and reward function are defined in view of the power grid dispatch decision-making requirements.

5. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 4 is characterized in that: The state space is defined as a multi-source data set obtained by obtaining state data of new energy output, load and flexibility resources.

6. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 4 is characterized in that: The action space is defined as the actions of each controllable flexibility resource, including the output of thermal power units and the charging and discharging power of battery energy storage.

7. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 4 is characterized in that: The reward function is defined as a function of the economic cost of the dispatch decision and the penalty for violating the operating constraints of the power system and flexibility resources.

8. The power grid dispatch decision-making method based on interpretable deep reinforcement learning according to claim 1, characterized in that: In step S140, the interpretable prototype mapping network is used to establish a connection between multiple interpretable prototypes and corresponding scheduling strategies; The interpretable prototype mapping network architecture includes: encoding the state, mapping the encoded state, calculating the similarity, and generating the final action strategy; By mapping the current state to its most similar prototype, the interpretable prototype mapping network architecture generates interpretable grid dispatch strategies.

Images