A multi-objective optimization scheduling system and method for smart grid

Abstract

This invention provides a multi-objective optimization scheduling system and method for smart grids, belonging to the field of power grid scheduling optimization technology. The multi-objective optimization scheduling system for smart grids includes: a scheduling objective combination module, a state space construction module, an action space construction module, an agent training module, and a scheduling scheme generation module. In this invention, multiple scheduling objective combinations are set according to the scheduling objective combination module, and multiple agents are trained based on these combinations using the agent training module. Each agent corresponds to one scheduling objective combination. During the training process of multiple agents, the shared state space and action space reduce the waste of human and computational resources during training. By configuring the working time interval of each agent, a power output allocation scheme for each generator unit is generated, which enhances the flexibility and controllability of power grid scheduling while ensuring the safe and stable operation of the power grid.

Description

Technical Field

[0001] This invention relates to the field of power grid dispatch optimization technology, and in particular to a multi-objective optimization dispatch system and method for smart grids. Background Technology

[0002] Grid dispatching is a crucial means of ensuring the safe, stable, and economical operation of the power grid. With global warming, new energy power generation such as wind and solar power is receiving increasing attention. Accelerating the development of new energy technologies and improving the grid's capacity to absorb green electricity are essential for balancing economic development and the green transformation of the grid's energy structure. However, the intermittency and instability of new energy sources increase the complexity of grid dispatching. Traditional single-objective dispatching methods are insufficient to meet the demands, and grid dispatching schemes need to balance multiple aspects, including safety, stability, economy, and environmental protection.

[0003] Reinforcement learning is an interactive trial-and-error learning method that enables agents to make decisions and self-optimize in complex and dynamic power grid environments through interactive learning with the environment, improving the efficiency and flexibility of power grid dispatch and showing promising application prospects in power grid dispatch. However, in the process of multi-objective reinforcement training, due to the mutual constraints between objectives, the agent needs to try more actions, which prolongs the training time and reduces the learning efficiency. Furthermore, if the dispatch objective is changed, the environment needs to be reconstructed and the agent needs to be trained again, which is a cumbersome process, and the repeated training process wastes human and computing resources. Summary of the Invention

[0004] To address at least some of the aforementioned problems, this invention provides a multi-objective optimization scheduling system and method for smart grids. This system is significant for rapidly and conveniently training and flexibly utilizing intelligent agents, thereby lowering the barrier to entry for intelligent agents in the field of grid scheduling. To achieve the above objectives, the following technical solutions are provided:

[0005] A first aspect of this invention provides a multi-objective optimization scheduling system for smart grids, characterized by comprising the following functional modules:

[0006] The scheduling target combination module selects a combination of scheduling targets S from the set of scheduling targets S. i Scheduling target combination S i Used to guide the agent i Assess changes in environmental conditions and optimize actions based on the assessment results;

[0007] The state space construction module is used to describe the agent. iThe set of states observed during the decision-making process; the state space includes grid topology information, wind and solar power generation, energy storage device state of charge, load forecast information, electricity market prices, weather conditions, and generator unit operating status;

[0008] The action space construction module is used to guide the agent to output the optimal action based on the current state information. The action space includes the unit output adjustment value, the switching status of the transmission line, the charging or discharging power of the energy storage device, the power trading decision, and the equipment maintenance decision.

[0009] The agent training module, based on different scheduling target combinations S i Construct reward function R i The agent is trained based on a shared state space and action space. i ;

[0010] The scheduling scheme generation module configures each agent. i Based on the working time interval, generate the output allocation scheme for each generator set;

[0011] Where i>1, for all objective combinations S i The result of taking the union is the target set S.

[0012] Preferably, the set of scheduling objectives S includes cost minimization, carbon emission minimization, line loss minimization, revenue maximization, renewable energy utilization maximization, supply and demand balance, reliability maximization, and minimization of the peak-to-average load ratio; preferably, each set of scheduling objectives includes 3 scheduling objectives.

[0013] Preferably, according to different scheduling target combinations S i Construct reward function R i Specifically, the dimensions of the three scheduling objectives included in the scheduling objective combination Si are normalized to obtain...

[0014] Z t Let be the scheduling scheme at time t. Let Z be the scheduling target value at time t; calculate Z respectively. t The Euclidean distances D1 and D2 between the optimal and worst solutions are used to construct the reward function R. i ;

[0015]

[0016] Preferably, a policy-based learning approach is used to train the agent. i Specifically:

[0017] S1: Initialize training iterations j=1, set update policy time t and maximum training iterations; S2: Agent i S3: The action values ​​for each generator set are random samples of the policy function; S4: Obtain the action variable values, state variable values, and reward values ​​based on the output of each generator set; S5: Repeat steps S2-S3. If the training time is greater than or equal to the policy update time t, execute step S5; S6: Update the policy, j = j + 1; S7: Repeat steps S2-S5 until j reaches the maximum number of training iterations; S8: Save the Agent. i Strategy complete, training over.

[0018] Preferably, the agent is trained using a policy-based learning approach. i Among them, policy-based learning methods include any of the following:

[0019] Adaptive Policy Gradient Methods, Deterministic Policy Gradient Methods, and Deep Policy Gradient Methods.

[0020] Preferably, each agent is automatically generated based on meteorological conditions. i Working time interval Q i Configuration information for each agent, which can be manually edited; i During the working time interval Q i Generate real-time power grid dispatch plans.

[0021] A second aspect of the present invention provides a multi-objective optimization scheduling method for smart grids, characterized by comprising the following steps;

[0022] The state space construction step involves building an agent based on grid topology information, wind and solar power generation, energy storage device state of charge, load forecasting information, electricity market prices, weather conditions, and generator unit operating status information. i The set of states observed during the decision-making process;

[0023] Constructing action space steps, and building an agent based on the output adjustment values ​​of each unit, the switching status of transmission lines, the charging or discharging power of energy storage devices, power trading decisions, and equipment maintenance decisions. i A set of actions;

[0024] The scheduling target combination step involves selecting a combination of scheduling targets S from the set of scheduling targets S. iScheduling target combination S i Used to guide the agent i Assess changes in environmental conditions and optimize actions based on the assessment results;

[0025] The agent training steps are based on different scheduling target combinations S. i Construct reward function R i The agent is trained based on a shared state space and action space. i ;

[0026] The scheduling scheme generation steps include configuring each agent. i The power output allocation scheme for each generator set is generated within the working time interval.

[0027] The beneficial effects of this invention are as follows:

[0028] 1. This invention sets up multiple scheduling target combinations according to the scheduling target combination module, and uses the agent training module to train multiple agents based on the multiple scheduling target combinations. Each agent corresponds to a scheduling target combination. During the training process of multiple agents, the shared state space and action space are used to reduce the waste of human and computing resources during the training process.

[0029] 2. This invention generates a power distribution scheme for each generator set by configuring the working time interval of each intelligent agent. Especially when major events or equipment maintenance arrangements are known in advance, it supports manual editing of the working time interval of the automatically generated intelligent agents, thereby enhancing the flexibility and controllability of power grid dispatching. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0031] Figure 1 This is a functional structure diagram of the multi-objective optimization scheduling system for smart grids according to the present invention;

[0032] Figure 2 This is a flowchart of the training of an intelligent agent using a policy-based learning method according to the present invention;

[0033] Figure 3 This is a flowchart of the multi-objective optimization scheduling method for smart grids according to the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0035] Example 1

[0036] like Figure 1 As shown, the present invention provides a multi-objective optimization scheduling system for smart grids, characterized by comprising the following functional modules:

[0037] The scheduling target combination module selects a combination of scheduling targets S from the set of scheduling targets S. i Scheduling target combination S i Used to guide the agent i The changes in environmental conditions are evaluated, and the actions are optimized based on the evaluation values. Where i>1, for all objective combinations S... i The result of taking the union is the target set S.

[0038] With the increasing complexity of power grids and the need to actively improve the utilization of green energy sources such as wind and solar power, traditional single-objective scheduling methods are struggling to meet the demands. This invention first constructs a set of scheduling objectives S, which includes multiple scheduling objectives. Preferably, the set of scheduling objectives S includes cost minimization, carbon emission minimization, line loss minimization, revenue maximization, renewable energy utilization maximization, supply-demand balance, reliability maximization, and minimizing the peak-to-average load ratio.

[0039] Based on the set of scheduling targets S, the scheduling target combination module of this invention allows the scheduling targets to be combined according to actual needs, obtaining multiple scheduling target combinations S. i Preferably, the scheduling target combination module provides a visual interface for users to select targets. For example, a power dispatcher can select three scheduling targets on the visual interface and click the OK button to generate a scheduling target combination.

[0040] For the combination of dispatching objectives S determined by the power dispatcher i It automatically generates scheduling objective functions. For example, for a scheduling objective combination S that includes minimizing cost, minimizing carbon emissions, and minimizing line loss...i The comprehensive scheduling objective function is generated as min(F) = F1 + F2 + F3, where F1, F2, and F3 represent unit operating cost, carbon emissions, and line power loss, respectively, and min() represents minimization. Preferably, function F1 includes unit output parameters and operation and maintenance cost parameters, and function F2 includes parameters such as fossil fuel consumption, carbon oxidation rate of fossil fuels, and unit calorific value of fossil fuels. Each scheduling objective function can be pre-designed by professionals according to industry standards, and the comprehensive scheduling objective function is automatically generated by the scheduling objective combination module based on the designed scheduling objective functions. Constructing the scheduling objective function is not the focus of this invention and will not be described in detail here.

[0041] Reinforcement learning is a machine learning method that leverages environmental perception and decision-making abilities, and is currently widely researched and applied in fields such as power grid dispatching and reactive power optimization. The agent and the environment are two crucial components of reinforcement learning. At each time step t, the agent generates an action based on its observed environmental state and its policy. The environment calculates a reward value and the state for the next time step based on the action. The agent updates its policy accordingly, gradually learning the optimal policy through continuous interaction with the environment. Solving reinforcement learning problems essentially involves maximizing the cumulative reward gained by the agent during its interaction with the environment. When the environmental state is fully observable, the agent can describe the entire reinforcement learning problem by constructing a Markov Decision Process (MDP). Constructing a reasonable state space, action space, and reward function is the most critical and costly part of training the agent, and is a fundamental basis for training an effective and usable agent.

[0042] The state space construction module is used to describe the agent. i The state space is the set of states observed during the decision-making process; it includes grid topology information, wind and solar power generation, energy storage device state of charge, load forecast information, electricity market prices, weather conditions, and generator unit operating status. Generator unit operating status includes the output of thermal power units.

[0043] The action space construction module is used to guide the agent to output the optimal action based on the current state information. The action space includes the unit output adjustment value, the switching status of the transmission line, the charging or discharging power of the energy storage device, the power trading decision, and the equipment maintenance decision. The power trading decision includes purchasing or selling electricity.

[0044] The agent training module, based on different scheduling target combinations S i Construct reward function R i The agent is trained based on a shared state space and action space. i .

[0045] Preferably, according to different scheduling target combinations S i Construct reward function R i Specifically, the dimensions of the three scheduling objectives included in the scheduling objective combination Si are normalized to obtain...

[0046] Z t Let be the scheduling scheme at time t. Let Z be the scheduling target value at time t; calculate Z respectively. t The Euclidean distances D1 and D2 between the optimal and worst solutions. For example, if the optimal solution D1 is (1, 0, 1) and the worst solution D2 is (0, 1, 0), then the calculation process for D1 and D2 is as follows:

[0047]

[0048] The closer the target is to the optimal solution and the farther it is from the worst solution, the better the scheduling scheme corresponding to the scheduling objective. The reward function R is constructed based on the Euclidean distances D1 and D2. i :

[0049] 0<=R i <=1,R i The closer the value is to 1, the better the scheduling scheme.

[0050] Preferably, a policy-based learning approach is used to train the agent. i , specifically Figure 2 As shown:

[0051] S1: Initialize the training count j=1, set the update policy time t and the maximum training count; S2: Calculate the Agent i S3: The action values ​​for each generator set are random samples of the policy function; S4: Obtain the action variable values, state variable values, and reward values ​​based on the output of each generator set; S5: Repeat steps S2-S3. If the training time is greater than or equal to the policy update time t, execute step S5; S6: Update the policy, j = j + 1; S7: Repeat steps S2-S5 until j reaches the maximum number of training iterations; S8: Save the Agent. i Strategy complete, training over.

[0052] Preferably, the agent is trained using a policy-based learning approach. i Among them, policy-based learning methods include any of the following:

[0053] Adaptive Policy Gradient Methods, Deterministic Policy Gradient Methods, and Deep Policy Gradient Methods.

[0054] The scheduling scheme generation module configures each agent. i Within the working time interval, a power output allocation scheme for each generator unit is generated. Grid dispatching includes three optimization types based on time scales: day-ahead dispatching, intraday rolling optimization, and real-time adjustment. Dispatching typically uses a 15-minute time window. In actual grid dispatching, factors such as day and night, different seasons, and different temperatures all affect dispatching objectives. For example, during the summer daytime, while ensuring grid stability, appropriate intelligent agents should be dispatched in real-time during daytime hours to maximize the utilization rate of photovoltaic power generation and maximize the revenue of photovoltaic units, avoiding "curtailment." Within different time windows, reasonable intelligent agents are assigned to execute dispatching based on factors such as weather, load forecasting, maintenance schedules, and social events, improving dispatching flexibility.

[0055] Preferably, each agent is automatically generated based on meteorological conditions. i Working time interval Q i The configuration information, which can be manually edited, is for each agent. i During the working time interval Q i Generate real-time power grid dispatch plans. For example, based on weather conditions, the automatically generated configuration information specifies that from 12:00 to 12:15, agents will be assigned to maximize the utilization rate of photovoltaic power generation and the revenue of photovoltaic units. However, considering that the day is a holiday and the load forecast is high, agents that pursue supply and demand balance and maximize reliability can be manually assigned to perform dispatch tasks for the corresponding time period.

[0056] Example 2

[0057] A second aspect of this invention provides a multi-objective optimization scheduling method for smart grids, characterized by comprising the following steps, such as... Figure 3 As shown:

[0058] The state space construction step involves building an agent based on grid topology information, wind and solar power generation, energy storage device state of charge, load forecasting information, electricity market prices, weather conditions, and generator unit operating status information. i The set of states observed during the decision-making process.

[0059] Constructing action space steps, and building an agent based on the output adjustment values ​​of each unit, the switching status of transmission lines, the charging or discharging power of energy storage devices, power trading decisions, and equipment maintenance decisions. i A set of actions.

[0060] The scheduling target combination step involves selecting a combination of scheduling targets S from the set of scheduling targets S. i Scheduling target combination S i Used to guide the agent i Evaluate changes in environmental conditions and optimize actions based on the evaluation results.

[0061] The agent training steps are based on different scheduling target combinations S. i Construct reward function R i The agent is trained based on a shared state space and action space. i .

[0062] The scheduling scheme generation steps include configuring each agent. i The power output allocation scheme for each generator set is generated within the working time interval.

[0063] Preferably, the step of constructing the action space is based on the output adjustment value of each unit, including the output adjustment value of thermal power units, the output adjustment value of wind power units, and the output adjustment value of photovoltaic units.

[0064] Preferably, according to different scheduling target combinations S i Construct reward function R i Specifically, the dimensions of the three scheduling objectives included in the scheduling objective combination Si are normalized to obtain...

[0065] Z t Let be the scheduling scheme at time t. Let Z be the scheduling target value at time t; calculate Z respectively. t The Euclidean distances D1 and D2 between the optimal and worst solutions are used to construct the reward function R. i ;

[0066]

[0067] This invention configures each agent. i During the working time interval, the output allocation scheme of each generator unit is generated. On the basis of ensuring the safe and stable operation of the power grid, the utilization rate of green and renewable energy can be improved, the cost of repeatedly constructing action space and state space can be saved, the emission of greenhouse gases can be effectively reduced, and the flexibility and controllability of power grid dispatch can be enhanced.

[0068] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of the present invention can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0069] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded over a network. Thus, the apparatus described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA) for such software processing. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the apparatus described herein. Furthermore, when a general-purpose computer accesses code used to implement the apparatus shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the apparatus shown herein.

[0070] Those skilled in the art will recognize that the unit and device steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different devices to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present invention.

[0071] The above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present invention, and the patent protection scope of the embodiments of the present invention should be defined by the claims.

Claims

1. A multi-objective optimization scheduling system for smart grid, characterized in that, Comprise the following functional modules: The scheduling target combination module selects a scheduling target combination S from the scheduling target set S i For all target combinations S i The result of the union set is the target set S, each scheduling target combination includes 3 scheduling targets, and the scheduling target combination S i For guiding the intelligent agent Agent i The change of the environment state is evaluated, and the action is optimized according to the evaluation value; the scheduling target combination module provides a visual interface for the user to check, checks 3 scheduling targets on the visual interface, and clicks a confirmation button to generate a scheduling target combination; the scheduling target set S includes cost minimization, carbon emission minimization, line loss minimization, benefit maximization, renewable energy utilization rate maximization, supply and demand balance, reliability maximization, and minimum load curve peak-to-average ratio a state space construction module for describing the agent i a set of states observed in the decision process; the state space includes grid topology information, wind and photovoltaic power generation, energy storage device state of charge, load prediction information, power market price, weather conditions, generator set operating status; An action space construction module, configured to guide the agent to output an optimal action according to current state information, the action space comprising unit output adjustment values, switch states of power transmission lines, charging or discharging powers of energy storage devices, power transaction decisions and device maintenance decisions; An agent training module trains the agent according to different scheduling target combinations S i A reward function R is constructed i An agent Agent is trained based on the shared state space and action space i ; The scheduling scheme generation module configures each agent. i Based on the working time interval, generate the output allocation scheme for each generator set; The configuration information of the working time interval Q of each intelligent agent Agent is automatically generated according to the meteorological condition information, the configuration information supports manual editing, each intelligent agent Agent generates the real-time power grid scheduling plan in the working time interval Q. i i i i ​​​​ 2.The smart grid oriented multi-objective optimal dispatching system according to claim 1, wherein, According to different scheduling target combinations S i Constructing reward function R i Specifically, the dimensions of the three scheduling targets included in the scheduling target combination Si are normalized to obtain Wherein Z t is the scheduling scheme at time t, is the scheduling target value at time t; the Euclidean distances D1 and D2 between Z t and the optimal solution and the worst solution are calculated respectively; the reward function R i is constructed according to the Euclidean distances D1 and D2. 3.The smart grid oriented multi-objective optimal dispatching system according to claim 2, characterized in that, The agent is trained using a policy-based learning approach. i Specifically: S1: Initialize the number of training iterations j=1, set the update policy time t and the maximum number of training iterations; S2: Calculate the Agent i S3: Obtain action variable values, state variable values, and reward values ​​based on the output of each generator set; S4: Repeat steps S2-S3. If the training time is greater than or equal to the policy update time t, execute step S5. S5: update the policy, j = j + 1; S6: loop steps S2-S5 until j reaches the maximum number of training; S7: save the agent Agent i Strategy, training is complete.

4. The multi-objective optimization scheduling system for smart grid of claim 3, wherein, The agent is trained by using a policy-based learning method i The policy-based learning method is any one of the following: adaptive policy gradient methods, deterministic policy gradient methods, and deep policy gradient methods.

5. A multi-objective optimal dispatch method for smart grid, characterized in that, Comprise the following steps: The state space construction step is constructed according to the power grid topology information, wind power and photovoltaic power generation, energy storage device state of charge, load prediction information, power market price, weather conditions, and generator set operation state information i a set of states observed during the decision process; The action space constructing step constructs an action set of the Agent according to the output adjustment value of each unit, the switch state of the transmission line, the charging or discharging power of the energy storage device, the power transaction decision, and the device maintenance decision i The output adjustment value of each unit includes the output adjustment value of a thermal power unit, the output adjustment value of a wind power unit, and the output adjustment value of a photovoltaic unit. The scheduling target combination step selects a scheduling target combination S from the scheduling target set S i Each scheduling target combination includes 3 scheduling targets, and the scheduling target combination S i The scheduling target combination module is used for guiding the agent i The change of the environment state is evaluated, and the action is optimized according to the evaluation value; the scheduling target combination module provides a visual interface for the user to check, checks 3 scheduling targets on the visual interface, and clicks a confirmation button to generate a scheduling target combination; the scheduling target set S includes cost minimization, carbon emission minimization, line loss minimization, benefit maximization, renewable energy utilization rate maximization, supply and demand balance, reliability maximization, and minimum load curve peak-to-average ratio Agent training step, combining S according to different scheduling targets i Constructing a reward function R i Training an agent Agent based on shared state space and action space i ; The scheduling scheme generating step configures the working time interval of each intelligent agent Agent i , and generates the output distribution scheme of each generator set; the working time interval Q i of each intelligent agent Agent i is automatically configured according to the meteorological condition information, the configuration information supports manual editing, and each intelligent agent Agent i generates the real-time power grid scheduling plan in the working time interval Q i .

6. The method of claim 5, wherein, According to different scheduling target combinations S i Constructing the reward function R i Specifically, the dimensions of the three scheduling targets included in the scheduling target combination Si are normalized to obtain Wherein Z t is the scheduling scheme at time t, is the scheduling target value at time t; the Euclidean distances D1 and D2 between Z t and the optimal solution and the worst solution are calculated respectively; the reward function R i is constructed according to the Euclidean distances D1 and D2.

Images