Collaborative optimization scheduling method for power distribution and micro-grid and related device

Abstract

The invention belongs to the field of power distribution networks, and discloses a power distribution network and micro-grid collaborative optimization scheduling method and a related device, and the method comprises the steps: obtaining the state observation data of a power distribution network intelligent body and each micro-grid intelligent body; adopting a mixed multi-agent soft actor commentator algorithm, based on a pre-trained strategy network, according to the state observation data of the power distribution network agent and each micro-grid agent, solving a preset power distribution and micro-grid collaborative optimization model, and obtaining a control strategy of the power distribution network and each micro-grid; wherein the pre-trained strategy network is obtained through the following steps: designing the distribution microgrid collaborative optimization model into a reinforcement learning model in a multi-agent environment, and pre-training a strategy network of the reinforcement learning model based on a generative adversarial imitation learning mode to obtain the pre-trained strategy network. The method solves the problem that an existing method for solving the problem of cooperative optimization scheduling of the power distribution and micro-grids oriented to high-proportion distributed resource access is not economical, efficient and reliable, and improves the efficiency and effect of cooperative optimization scheduling of the power distribution and micro-grids.

Description

Technical Field

[0001] The present invention belongs to the field of power distribution networks, and relates to a method for collaborative optimization and dispatching of power distribution microgrids and related devices. Background Art

[0002] As an important public infrastructure, the distribution network plays an important role in ensuring power supply, supporting economic and social development, and serving to improve people's livelihood. With the widespread access to distributed resources and energy storage equipment, the coupling between distribution and microgrids has become increasingly close. However, due to the inherent strong uncertainty of spatiotemporal coupling, the diversity of controllable resources, the heavy communication burden, and concurrent competitive local behaviors of the distribution microgrid system, supervision and coordination face serious challenges. In order to ensure the safe and stable operation of the distribution microgrid system, improve the convergence efficiency of large-scale power grid model training, and reduce the system operation cost, it is of great significance to make full use of the various flexibility resources in the distribution microgrid and the information interaction and collaborative optimization capabilities between regions under the condition of protecting privacy.

[0003] At present, the model-driven centralized optimization method is mainly used for the collaborative optimization problem of distribution microgrids. Traditional centralized optimization requires intensive communication resources, large data collection volume, and cannot guarantee the privacy of multiple subjects. Therefore, distributed or decentralized optimization has been proposed one after another. Under the coordination of weak central entities, limited information exchange is used to reach consensus and global optimality, which has better flexibility and privacy protection. However, the distribution microgrid side usually contains discrete equipment, such as on-load tap-changing transformers and switching capacitor banks, to reduce voltage over-limit and reduce economic losses. However, the model-driven method has problems such as convergence difficulties and low computational efficiency for non-convex optimization problems with discrete variables. In addition, the model-driven method usually uses methods such as stochastic optimization or robust optimization to deal with uncertain factors, which further increases the complexity of building and solving the distribution micro-cooperative optimization model. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a distribution microgrid coordinated optimization scheduling method and related devices.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a distribution-microgrid collaborative optimization scheduling method, comprising: obtaining state observation data of a distribution network agent and each microgrid agent; using a hybrid multi-agent soft actor critic algorithm, based on a pre-trained strategy network, according to the state observation data of the distribution network agent and each microgrid agent, solving a preset distribution-microgrid collaborative optimization model, and obtaining control strategies for the distribution network and each microgrid; wherein the pre-trained strategy network is obtained by the following method: designing the distribution-microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment, and pre-training the strategy network of the reinforcement learning model based on a generative adversarial imitation learning method to obtain a pre-trained strategy network; the distribution-microgrid collaborative optimization model comprises: an optimization scheduling model for each microgrid constructed with the minimum total production cost as the optimization target, and a distribution network optimization scheduling model constructed with the minimum total operating cost as the optimization target.

[0007] Optionally, the state observation data of the distribution network intelligent body include: observation data of static VAR compensators in the distribution network, observation data of renewable energy units, observation data of micro gas turbines, observation data of energy storage systems, observation data of on-load tap-changing transformers, observation data of switching capacitor groups, the number of times the on-load tap-changing transformers have been operated and the number of times the switching capacitor groups have been operated, as well as the active load of the distribution network with a normal distribution error superimposed on it, the set of electricity purchase and sales prices of the distribution network, and boundary information from each microgrid intelligent body; the state observation data of the microgrid intelligent body include: observation data of renewable energy units in the microgrid, observation data of micro gas turbines and observation data of the energy storage system, as well as the active load of the microgrid with a normal distribution error superimposed on it and boundary information from the distribution network intelligent body.

[0008] Optionally, the distribution network intelligent entity is set up in the following manner: the distribution network is virtually divided into a number of autonomous areas, a continuous intelligent entity is set up for each autonomous area, and a separate intelligent entity is set up for independent and multi-level discrete devices in the distribution network, and they are combined with each other to obtain a distribution network intelligent entity; each microgrid intelligent entity is set up in the following manner: an autonomous continuous intelligent entity is set up for each microgrid to obtain each microgrid intelligent entity.

[0009] Optionally, the objective function of the microgrid optimization scheduling model is:

[0010]

[0011] Among them, F MG,i is the total production cost of microgrid i; T is the scheduling duration; is the number of micro gas turbines in microgrid i; is the power generation cost of the jth micro gas turbine in period t; is the number of energy storage units in microgrid i; is the power generation cost of the j-th energy storage unit in period t; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t.

[0012] The constraints of the microgrid optimization scheduling model include: renewable energy unit output constraints, micro gas turbine output constraints, energy storage system constraints, microgrid internal power flow balance constraints and distribution microgrid transmission power constraints.

[0013] The objective function of the distribution network optimization dispatching model is:

[0014]

[0015] Among them, F ADN is the total operating cost of the distribution network; and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and are the power generation costs of the jth micro gas turbine and energy storage unit in the distribution network during period t; F t OLTC and are the operating costs of the on-load tap-changing transformer and the j-th switching capacitor bank in the distribution network during period t respectively; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, F t B′-S′ is the cost of electricity purchased and sold from the distribution network to the main grid during period t.

[0016] The constraints of the distribution network optimization dispatching model include: on-load tap-changing transformer operation times constraint, capacitor bank operation times constraint, static VAR compensator output constraint, distribution network flow balance constraint, node voltage constraint and current carrying capacity constraint.

[0017] Optionally, designing the distribution and microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment includes: designing both the microgrid optimization scheduling model and the distribution network optimization scheduling model as reinforcement learning models according to the observation space, action space and reward function.

[0018] Observation Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for:

[0019]

[0020] in, and are the observation data of the energy storage system and the micro gas turbine of microgrid i in period t, and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; It is the boundary information from the distribution network intelligent agent in period t-1.

[0021] Action Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for:

[0022]

[0023] in, is the reactive power prediction value of the renewable energy unit in microgrid i during period t, and are the active power and reactive power of the micro gas turbine in microgrid i during period t, is the charging and discharging active power of the energy storage system in microgrid i during period t.

[0024] Reward function of the reinforcement learning model for microgrid optimal dispatch model for:

[0025]

[0026] in, and are the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t respectively; F is the cost of microgrid i purchasing and selling electricity from the distribution network during period t; t TL is the penalty function for exceeding the transmission power constraint between distribution microgrids; 1,2 is the penalty weight for exceeding the limit; δ i F is the willingness coefficient of microgrid i to support the safe operation of distribution network; t penalty It is a penalty function for constraining the safe operation of distribution network.

[0027] Observation Space of Reinforcement Learning Model for Distribution Network Optimal Dispatch Model for:

[0028]

[0029] in, and They are the observation data of the on-load tap-changing transformer and the observation data of the switched capacitor bank, and are the observation data of the static VAR compensator, the observation data of the energy storage system and the observation data of the micro gas turbine of microgrid i, respectively. and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the electricity purchase and sales price set of the distribution network in period t; is the boundary information from the microgrid agent i.

[0030] The action space of the reinforcement learning model of the distribution network optimization dispatching model is:

[0031]

[0032] in, and is the discrete action space and continuous action space in region i of the distribution network, and They are the gear positions of the on-load tap-changing transformer and the gear positions of the switching capacitor bank in area i of the distribution network during period t; is the reactive power prediction value of the renewable energy unit i in microgrid during period t, and are the active power and reactive power of the micro gas turbine in microgrid i at time period t, is the charging and discharging active power of the energy storage system of microgrid i during period t, is the reactive power output by the static VAR compensator of microgrid i during period t.

[0033] Reward function of reinforcement learning model for distribution network optimal dispatch model for:

[0034]

[0035] Among them, F t OLTC and are the operating cost of the on-load tap-changing transformer in period t and the operating cost of the j-th switching capacitor bank, respectively. and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and is the power generation cost of the j-th micro gas turbine and the j-th energy storage unit in period t, F t B′-S′ is the cost of electricity purchase and sale from the distribution network to the main grid, is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, It is the penalty function for exceeding the limit of distribution network voltage and branch power flow constraints.

[0036] Optionally, the hybrid multi-agent soft actor critic algorithm is based on a pre-trained strategy network, and when solving a preset distribution-microgrid collaborative optimization model according to the state observation data of the distribution network agent and each microgrid agent, the distribution-microgrid collaborative optimization model adopts a two-layer architecture of the distribution network layer and the microgrid layer; wherein the distribution network optimization scheduling model of the distribution network layer is set as a discrete-continuous agent including a Critic network, a discrete Actor network and a continuous Actor network, and the Critic network is used to guide the update of the discrete Actor network and the continuous Actor network; each microgrid optimization scheduling model of the microgrid layer is set to include a dual Critic network and a continuous intelligent agent of a continuous Actor network, and a dual Critic network is used to guide the update of the continuous Actor network; the Critic network of the microgrid optimization scheduling model updates parameters by minimizing the Bellman residual, and both the Actor network and temperature parameters use the policy gradient with maximum entropy to update parameters; the Critic network of the distribution network optimization scheduling model updates parameters through a centralized joint regression loss function, and introduces OU noise into the continuous Actor network of the distribution network optimization scheduling model, as well as a multi-head attention mechanism for selectively focusing on information from each microgrid optimization scheduling model in the Critic network of the distribution network optimization scheduling model.

[0037] According to a second aspect of the present invention, a distribution-microgrid collaborative optimization dispatching system is provided, comprising: a data acquisition module for acquiring state observation data of a distribution network agent and each microgrid agent; a strategy generation module for using a hybrid multi-agent soft actor critic algorithm, based on a pre-trained strategy network, according to the state observation data of the distribution network agent and each microgrid agent, to solve a preset distribution-microgrid collaborative optimization model, and obtain control strategies for the distribution network and each microgrid; wherein the pre-trained strategy network is obtained by the following method: the distribution-microgrid collaborative optimization model is designed as a reinforcement learning model in a multi-agent environment, and the strategy network of the reinforcement learning model is pre-trained based on a generative adversarial imitation learning method to obtain a pre-trained strategy network; the distribution-microgrid collaborative optimization model comprises: an optimization dispatching model for each microgrid constructed with the minimum total production cost as the optimization target, and a distribution network optimization dispatching model constructed with the minimum total operating cost as the optimization target.

[0038] Optionally, the state observation data of the distribution network intelligent body include: observation data of static VAR compensators in the distribution network, observation data of renewable energy units, observation data of micro gas turbines, observation data of energy storage systems, observation data of on-load tap-changing transformers, observation data of switching capacitor groups, the number of times the on-load tap-changing transformers have been operated and the number of times the switching capacitor groups have been operated, as well as the active load of the distribution network with a normal distribution error superimposed on it, the set of electricity purchase and sales prices of the distribution network, and boundary information from each microgrid intelligent body; the state observation data of the microgrid intelligent body include: observation data of renewable energy units in the microgrid, observation data of micro gas turbines and observation data of the energy storage system, as well as the active load of the microgrid with a normal distribution error superimposed on it and boundary information from the distribution network intelligent body.

[0039] Optionally, the distribution network intelligent entity is set up in the following manner: the distribution network is virtually divided into a number of autonomous areas, a continuous intelligent entity is set up for each autonomous area, and a separate intelligent entity is set up for independent and multi-level discrete devices in the distribution network, and they are combined with each other to obtain a distribution network intelligent entity; each microgrid intelligent entity is set up in the following manner: an autonomous continuous intelligent entity is set up for each microgrid to obtain each microgrid intelligent entity.

[0040] Optionally, the objective function of the microgrid optimization scheduling model is:

[0041]

[0042] Among them, F MG,i is the total production cost of microgrid i; T is the scheduling duration; is the number of micro gas turbines in microgrid i; is the power generation cost of the jth micro gas turbine in period t; is the number of energy storage units in microgrid i; is the power generation cost of the j-th energy storage unit in period t; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t.

[0043] The constraints of the microgrid optimization scheduling model include: renewable energy unit output constraints, micro gas turbine output constraints, energy storage system constraints, microgrid internal power flow balance constraints and distribution microgrid transmission power constraints.

[0044] The objective function of the distribution network optimization dispatching model is:

[0045]

[0046] Among them, F ADN is the total operating cost of the distribution network; and N MGare the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and are the power generation costs of the jth micro gas turbine and energy storage unit in the distribution network during period t; F t OLTC and are the operating costs of the on-load tap-changing transformer and the j-th switching capacitor bank in the distribution network during period t respectively; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, F t B′-S′ is the cost of electricity purchased and sold from the distribution network to the main grid during period t.

[0047] The constraints of the distribution network optimization dispatching model include: on-load tap-changing transformer operation times constraint, capacitor bank operation times constraint, static VAR compensator output constraint, distribution network flow balance constraint, node voltage constraint and current carrying capacity constraint.

[0048] Optionally, designing the distribution and microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment includes: designing both the microgrid optimization scheduling model and the distribution network optimization scheduling model as reinforcement learning models according to the observation space, action space and reward function.

[0049] Observation Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for:

[0050]

[0051] in, and are the observation data of the energy storage system and the micro gas turbine of microgrid i in period t, and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; It is the boundary information from the distribution network intelligent agent in period t-1.

[0052] Action Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for:

[0053]

[0054] in, is the reactive power prediction value of the renewable energy unit in microgrid i during period t, and are the active power and reactive power of the micro gas turbine in microgrid i during period t, is the charging and discharging active power of the energy storage system in microgrid i during period t.

[0055] Reward function of the reinforcement learning model for microgrid optimal dispatch model for:

[0056]

[0057] in, and are the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t respectively; F is the cost of microgrid i purchasing and selling electricity from the distribution network during period t; t TL is the penalty function for exceeding the transmission power constraint between distribution microgrids; 1,2 is the penalty weight for exceeding the limit; δ i Ft is the willingness coefficient of microgrid i to support the safe operation of distribution network; penalty It is a penalty function for constraining the safe operation of distribution network.

[0058] Observation Space of Reinforcement Learning Model for Distribution Network Optimal Dispatch Model for:

[0059]

[0060] in, and They are the observation data of the on-load tap-changing transformer and the observation data of the switched capacitor bank, and are the observation data of the static VAR compensator, the observation data of the energy storage system and the observation data of the micro gas turbine of microgrid i, respectively. and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the electricity purchase and sales price set of the distribution network in period t; is the boundary information from the microgrid agent i.

[0061] The action space of the reinforcement learning model of the distribution network optimization dispatching model is:

[0062]

[0063] in, and is the discrete action space and continuous action space in region i of the distribution network, and They are the gear positions of the on-load tap-changing transformer and the gear positions of the switching capacitor bank in area i of the distribution network during period t; is the reactive power prediction value of the renewable energy unit i in microgrid during period t, and are the active power and reactive power of the micro gas turbine in microgrid i at time period t, is the charging and discharging active power of the energy storage system of microgrid i during period t, is the reactive power output by the static VAR compensator of microgrid i during period t.

[0064] Reward function of reinforcement learning model for distribution network optimal dispatch model for:

[0065]

[0066] Among them, F t OLTC and are the operating cost of the on-load tap-changing transformer in period t and the operating cost of the j-th switching capacitor bank, respectively. and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and is the power generation cost of the j-th micro gas turbine and the j-th energy storage unit in period t, F t B′-S′ is the cost of electricity purchase and sale from the distribution network to the main grid, is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, It is the penalty function for exceeding the limit of distribution network voltage and branch power flow constraints.

[0067] Optionally, the hybrid multi-agent soft actor critic algorithm is based on a pre-trained strategy network, and when solving a preset distribution-microgrid collaborative optimization model according to the state observation data of the distribution network agent and each microgrid agent, the distribution-microgrid collaborative optimization model adopts a two-layer architecture of the distribution network layer and the microgrid layer; wherein the distribution network optimization scheduling model of the distribution network layer is set as a discrete-continuous agent including a Critic network, a discrete Actor network and a continuous Actor network, and the Critic network is used to guide the update of the discrete Actor network and the continuous Actor network; each microgrid optimization scheduling model of the microgrid layer is set to include a dual Critic network and a continuous intelligent agent of a continuous Actor network, and a dual Critic network is used to guide the update of the continuous Actor network; the Critic network of the microgrid optimization scheduling model updates parameters by minimizing the Bellman residual, and both the Actor network and temperature parameters use the policy gradient with maximum entropy to update parameters; the Critic network of the distribution network optimization scheduling model updates parameters through a centralized joint regression loss function, and introduces OU noise into the continuous Actor network of the distribution network optimization scheduling model, as well as a multi-head attention mechanism for selectively focusing on information from each microgrid optimization scheduling model in the Critic network of the distribution network optimization scheduling model.

[0068] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned distribution and microgrid collaborative optimization scheduling method when executing the computer program.

[0069] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned distribution microgrid collaborative optimization scheduling method are implemented.

[0070] Compared with the prior art, the present invention has the following beneficial effects:

[0071] The distribution microgrid collaborative optimization scheduling method of the present invention designs the distribution microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment, and pre-trains the policy network of the reinforcement learning model based on the generative adversarial imitation learning method to obtain a pre-trained policy network. It can fully consider the complex state space and action space of the multi-microgrid and distribution network collaborative model, thereby alleviating the problem of low training efficiency in the initial stage of reinforcement learning. Next, a hybrid multi-agent soft actor-critic algorithm is adopted. Based on the pre-trained strategy network, according to the state observation data of the distribution network agent and each microgrid agent, the preset distribution-microgrid collaborative optimization model is solved to obtain the control strategies of the distribution network and each microgrid. The optimization problem is designed as a distributed optimization problem in a multi-agent reinforcement learning environment through the hybrid multi-agent soft actor-critic algorithm. This method can effectively solve high-dimensional, non-convex and nonlinear distribution-microgrid collaborative optimization scheduling problems. It is suitable for complex non-convex models and the collaborative optimization of distribution microgrids with discrete devices under multiple uncertainties. It has a faster convergence speed and better training results, has a better ability to cope with environmental non-stationarity, and can meet the needs of real-time decision-making of the dispatching system. It has obvious advantages in dealing with complex environmental problems and effectively maintains the safety and stability of microgrids and distribution networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flow chart of a method for collaborative optimization and dispatching of distribution and microgrids according to an embodiment of the present invention.

[0073] Figure 2 This is a schematic diagram of the area division of the IEEE33-node standard system agent according to an embodiment of the present invention.

[0074] Figure 3 This is a structural block diagram of a distribution microgrid collaborative optimization dispatching system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0076] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0077] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0078] See also Figure 1 In one embodiment of the present invention, a distribution microgrid collaborative optimization scheduling method is provided, specifically a distribution microgrid collaborative optimization scheduling method based on reinforcement learning efficient training technology, which can effectively solve the problem that the existing solution method for the distribution microgrid collaborative optimization scheduling problem facing a high proportion of distributed resource access is not economical, efficient and reliable enough, and improve the efficiency and effect of the distribution microgrid collaborative optimization.

[0079] Specifically, the coordinated optimization dispatching method of the distribution microgrid of the present invention comprises the following steps:

[0080] S1: Obtain the state observation data of the distribution network intelligent agent and each microgrid intelligent agent.

[0081] S2: A hybrid multi-agent soft actor-critic algorithm is used. Based on the pre-trained strategy network, the preset distribution-microgrid collaborative optimization model is solved according to the state observation data of the distribution network agent and each microgrid agent, and the control strategies of the distribution network and each microgrid are obtained.

[0082] Among them, the pre-trained policy network is obtained by the following method: the distribution microgrid collaborative optimization model is designed as a reinforcement learning model in a multi-agent environment, and the policy network of the reinforcement learning model is pre-trained based on the generative adversarial imitation learning method to obtain the pre-trained policy network; the distribution microgrid collaborative optimization model includes: each microgrid optimization scheduling model constructed with the minimum total production cost as the optimization goal and the distribution network optimization scheduling model constructed with the minimum total operating cost as the optimization goal.

[0083] The distribution microgrid collaborative optimization scheduling method of the present invention designs the distribution microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment, and pre-trains the policy network of the reinforcement learning model based on the generative adversarial imitation learning method to obtain a pre-trained policy network. It can fully consider the complex state space and action space of the multi-microgrid and distribution network collaborative model, thereby alleviating the problem of low training efficiency in the initial stage of reinforcement learning. Next, a hybrid multi-agent soft actor-critic algorithm is adopted. Based on the pre-trained strategy network, according to the state observation data of the distribution network agent and each microgrid agent, the preset distribution-microgrid collaborative optimization model is solved to obtain the control strategies of the distribution network and each microgrid. The optimization problem is designed as a distributed optimization problem in a multi-agent reinforcement learning environment through the hybrid multi-agent soft actor-critic algorithm. This method can effectively solve high-dimensional, non-convex and nonlinear distribution-microgrid collaborative optimization scheduling problems. It is suitable for complex non-convex models and the collaborative optimization of distribution microgrids with discrete devices under multiple uncertainties. It has a faster convergence speed and better training results, has a better ability to cope with environmental non-stationarity, and can meet the needs of real-time decision-making of the dispatching system. It has obvious advantages in dealing with complex environmental problems and effectively maintains the safety and stability of microgrids and distribution networks.

[0084] In a possible implementation, the state observation data of the distribution network intelligent body includes: observation data of static VAR compensators in the distribution network, observation data of renewable energy units, observation data of micro gas turbines, observation data of energy storage systems, observation data of on-load tap-changing transformers, observation data of switching capacitor banks, the number of times the on-load tap-changing transformers have been actuated and the number of times the switching capacitor banks have been actuated, as well as the active load of the distribution network superimposed with normal distribution errors, the set of power purchase and sale prices of the distribution network, and boundary information from each microgrid intelligent body. The state observation data of the microgrid intelligent body includes: observation data of renewable energy units in the microgrid, observation data of micro gas turbines, observation data of energy storage systems, as well as the active load of the microgrid superimposed with normal distribution errors and boundary information from the distribution network intelligent body.

[0085] Explanatory data, the observation data of renewable energy units and micro gas turbines are generally output data, the observation data of static VAR compensators are reactive power, the observation data of energy storage systems are charging and discharging power, the observation data of on-load tap-changing transformers and the observation data of switched capacitor banks are generally the current gear. In addition, the normal distribution error is superimposed on the active load to fit the uncertainty and ensure the adaptability of the final result.

[0086] In a possible implementation manner, the distribution network intelligent agent is configured in the following manner: the distribution network is virtually divided into a number of autonomous areas, a continuous intelligent agent is configured for each autonomous area, and separate intelligent agents are configured for independent and multi-level discrete devices in the distribution network, and the devices are combined with each other to obtain a distribution network intelligent agent; and each microgrid intelligent agent is configured in the following manner: an autonomous continuous intelligent agent is configured for each microgrid to obtain each microgrid intelligent agent.

[0087] Explanatory, the power grid is divided into agents: The setting of the multi-agent model in the distribution micro-system mainly depends on two points: 1. The type of adjustable resources in the system (discrete / continuous adjustment equipment); 2. The decision-making dimensions of each agent should not be too many. Based on this, an autonomous continuous agent is set on the microgrid side. Considering that there are many scattered controllable resources on the distribution network side, in order to ensure the appropriate decision-making dimensions, the distribution network can be virtually divided into several autonomous areas, and continuous agents are set for each area, and separate agents are set for independent and multi-gear discrete devices.

[0088] For example, see Figure 2 ,Taking the IEEE33-node standard system as an example, the IEEE33-node standard system is divided into three autonomous areas as shown in the figure. Three microgrids are connected to nodes 17, 21 and 32. A distributed training-distributed execution framework is adopted to realize the coordinated optimization operation of the multi-microgrid-distribution network system.

[0089] In a possible implementation, the objective function of the microgrid optimization scheduling model is:

[0090]

[0091] Among them, F MG,i is the total production cost of microgrid i; T is the scheduling duration; is the number of micro gas turbines in microgrid i; is the power generation cost of the jth micro gas turbine in period t; is the number of energy storage units in microgrid i; is the power generation cost of the j-th energy storage unit in period t; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t.

[0092] The objective function of the distribution network optimization dispatching model is:

[0093]

[0094] Among them, F ADN is the total operating cost of the distribution network; and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and are the power generation costs of the jth micro gas turbine and energy storage unit in the distribution network during period t; F t OLTC and are the operating costs of the on-load tap-changing transformer and the j-th switching capacitor bank in the distribution network during period t respectively; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, F t B′-S′ is the cost of electricity purchased and sold from the distribution network to the main grid during period t.

[0095] Explanatory, since the distribution network and each microgrid operate in a decentralized manner as a sequential decision-making process, only partial state information of the environment can be observed, and the hybrid multi-agent on the distribution network side adopts partitioned control of discrete-continuous devices, and each agent only observes local information of its area. Therefore, the distribution-microgrid collaborative optimization problem under study is modeled as a partially observable Markov decision process.

[0096] When constructing a coordinated optimization model for distribution microgrids with a high proportion of distributed resources, the goal of optimization for microgrid i is to minimize the total production cost. The objective function of the microgrid optimization scheduling model is as follows:

[0097]

[0098] The optimization goal of the distribution network is to minimize the total operating cost, that is, to minimize the sum of the unit power generation cost and the interaction cost with the main grid and multiple microgrids. The objective function of the distribution network optimization scheduling model is as follows:

[0099]

[0100] According to the constructed distribution microgrid collaborative optimization model containing a high proportion of distributed resources, the constraints of the distribution microgrid collaborative optimization model are determined, including the constraints of the microgrid optimal scheduling model and the distribution network optimal scheduling model.

[0101] Optionally, the constraints of the microgrid optimization scheduling model include: renewable energy unit output constraints, micro gas turbine output constraints, energy storage system constraints, microgrid internal power flow balance constraints and distribution microgrid transmission power constraints.

[0102] For example, for renewable energy generation unit (RES) constraints, in order to improve economic efficiency, it is stipulated that RES generates electricity in maximum power point tracking mode, and only reactive capacity is used to participate in regulation, specifically:

[0103]

[0104] in, are the predicted active and reactive values ​​of the jth RES in microgrid i during period t, respectively; is the active power of the jth RES maximum tracking point operation mode in microgrid i during period t, is the number of RES units in microgrid i; is the maximum apparent power of the jth RES.

[0105] At the same time, in order to simulate the randomness of the prediction process, the prediction data of RES and load are superimposed with the prediction error that obeys the normal distribution:

[0106]

[0107] in, is the predicted active power of the jth RES in microgrid i during period t; is the RES prediction error during period t; μ RES , σ RES is the expected value and standard deviation of RES prediction error.

[0108] The output constraints of the micro gas turbine (MT) are as follows:

[0109]

[0110] in, is the active power of MT, They are the minimum and maximum values ​​of MT active power respectively; They are the maximum downward and upward climbing power of MT respectively; They are the reactive power and maximum apparent power of MT respectively.

[0111] The corresponding energy storage system (ESS) constraints are:

[0112]

[0113] in, are the minimum and maximum values ​​of ESS charging and discharging active power respectively; soc j,t is the charge during period t; soc j,min 、soc j,max are the minimum and maximum values ​​of charge respectively; are charging power and discharging power respectively; is the maximum storage capacity; It is the charge and discharge efficiency.

[0114] For the internal power flow balance constraints of the microgrid, specifically:

[0115]

[0116] in, are respectively the active and reactive loads inside microgrid i during period t; are the predicted active and reactive values ​​of the jth RES in microgrid i during period t, respectively; are the active and reactive power of MT respectively; To distribute active and reactive power between micro-stations,

[0117] For the transmission power constraints of the distribution microgrid, they are as follows:

[0118]

[0119] in, To distribute active and reactive power between micro-stations; are the minimum and maximum values ​​of active power transmitted between distribution microgrids, respectively; is the maximum apparent power.

[0120] Optionally, the constraints of the distribution network optimization dispatching model include: on-load tap-changing transformer action times constraint, switching capacitor bank action times constraint, static VAR compensator output constraint, distribution network flow balance constraint, node voltage constraint and current carrying capacity constraint. Among them, the on-load tap-changing transformer (OLTC) action times constraint and the switching capacitor bank (CB) action times constraint are specifically:

[0121]

[0122]

[0123] in, Record respectively whether the OLTC and the jth CB are in action, the number of actions, and the upper limit of the number of actions in the t period; V0, V 1,t are the upper and lower terminal voltages of the OLTC feeder, is the voltage regulation step size per unit gear of OLTC; It is the middle position of the tap of OLTC; is the adjustable reactive power of the jth CB unit; is the reactive power output by the jth CB in period t.

[0124] The output constraints of the static VAR compensator (SVC) are as follows:

[0125]

[0126] in, is the reactive power output by the jth SVC in period t; are the minimum and maximum reactive power output by the jth SVC respectively; is the number of SVCs in the distribution network.

[0127] The distribution network power flow balance constraints are as follows:

[0128]

[0129] Among them, P k,t , Q k,t are respectively the net active and reactive power injected at node k in the distribution network during period t; V k,t 、V l,t are the voltage amplitudes at nodes k and l during period t respectively; G kl , B kl ,θ kl,t are the conductance, susceptance and phase difference of the branch between nodes k and l respectively.

[0130] The node voltage constraints are as follows:

[0131] V min ≤V k,t ≤V max

[0132] Among them, V k,t is the voltage amplitude at node k during period t, V min 、V max are the upper and lower limits of the node voltage constraints respectively.

[0133] The current carrying capacity constraints are as follows:

[0134] |I kl,t |≤I kl,max

[0135] Among them, I kl,t is the current flowing through the branch between nodes k and l at time t; I kl,max is the upper limit of branch current.

[0136] In a possible implementation, the design of the distribution-microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment includes: designing both the microgrid optimization scheduling model and the distribution network optimization scheduling model as reinforcement learning models according to the observation space, action space and reward function.

[0137] Specifically, the observation space of the reinforcement learning model of the microgrid optimization scheduling model is for:

[0138]

[0139] in, and are the observation data of the energy storage system and the micro gas turbine of microgrid i in period t, and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; It is the boundary information from the distribution network intelligent agent in period t-1.

[0140] Action Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model It consists of continuous equipment actions, specifically:

[0141]

[0142] in, is the reactive power prediction value of the renewable energy unit in microgrid i during period t, and are the active power and reactive power of the micro gas turbine in microgrid i during period t, is the charging and discharging active power of the energy storage system in microgrid i during period t.

[0143] Reward function of the reinforcement learning model for microgrid optimal dispatch model for:

[0144]

[0145] in, and are the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t respectively; F is the cost of microgrid i purchasing and selling electricity from the distribution network during period t; t TL is the penalty function for exceeding the transmission power constraint between distribution microgrids; 1,2 is the penalty weight for exceeding the limit; δ i F is the willingness coefficient of microgrid i to support the safe operation of distribution network; t penalty It is a penalty function for constraining the safe operation of distribution network.

[0146] Explanatory,transactions with microgrids may affect the safe operation of the distribution network, and,when the above mentioned reward function is applied, the microgrid-based,agent can support the upper distribution network.

[0147] Observation Space of Reinforcement Learning Model for Distribution Network Optimal Dispatch Model for:

[0148]

[0149] in, and They are the observation data of the on-load tap-changing transformer and the observation data of the switched capacitor bank, and are the observation data of the static VAR compensator, the observation data of the energy storage system and the observation data of the micro gas turbine of microgrid i, respectively. and The observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed on them in period t are used to fit the uncertainty state and enhance the robustness of the algorithm. is the electricity purchase and sales price set of the distribution network in period t; is the boundary information from the microgrid agent i.

[0150] The action space of the reinforcement learning model of the distribution network optimization dispatching model consists of discrete-continuous device groups, specifically:

[0151]

[0152] in, and is the discrete action space and continuous action space in region i of the distribution network, and They are the gear positions of the on-load tap-changing transformer and the gear positions of the switching capacitor bank in area i of the distribution network during period t; is the reactive power prediction value of the renewable energy unit i in microgrid during period t, and are the active power and reactive power of the micro gas turbine in microgrid i at time period t, is the charging and discharging active power of the energy storage system of microgrid i during period t, is the reactive power output by the static VAR compensator of microgrid i during period t.

[0153] The reward function of the reinforcement learning model of the distribution network optimization scheduling model is as follows:

[0154]

[0155] Among them, F t OLTC and are the operating cost of the on-load tap-changing transformer in period t and the operating cost of the j-th switching capacitor bank, respectively. and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and is the power generation cost of the j-th micro gas turbine and the j-th energy storage unit in period t, F t B′-S′ is the cost of electricity purchase and sale from the distribution network to the main grid, is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, It is the penalty function for exceeding the limit of distribution network voltage and branch power flow constraints.

[0156] Among them, the reward value of the discrete agent in region i is Negative global discrete device action costs and global continuous device action cost composition.

[0157] Explanation: Based on the reinforcement learning model in the multi-agent environment constructed above, generative adversarial imitation learning is used to pre-train the agent's policy network. Generative Adversarial Imitation Learning (GAIL) uses the generative adversarial method in Generative Adversarial Networks (GANs). The algorithm introduces a measure of the state-action occupancy of the imitator to make it similar to the relevant characteristics of the demonstrator. It uses a discriminator in a GAN to give an estimate of the action-value function based on the demonstration data. For the reinforcement learning process, the action-value can be obtained from the demonstration through a generative method:

[0158]

[0159] in, are the state spaces of the microgrid and distribution network, respectively. is the action space of microgrid i, is the discrete-continuous action space in the distribution network area i; Γ i represents the sample set explored when the iteration number is i, and D is the output value from the discriminator. The parameter of the discriminator is ω, and ω indicates that the Q value is estimated after updating the parameters of the discriminator by one step, so the iteration number is i+1.

[0160] The loss function of the discriminator is defined in the general form:

[0161]

[0162] Among them, Γ i , Γ E are the sample sets from exploration and expert demonstration, respectively.

[0163] In the evaluator network, the following objective function is used to enable the evaluator to accurately judge the state value function of the current generator strategy:

[0164] minE τ [(r t (s t )+V φ (s t+1 )-Vφ (s t )) 2 ]

[0165] Among them, τ represents the strategy trajectory sampled from the expert library, r represents the reward function corresponding to the current state, and V φ represents the evaluator model output, and φ represents the evaluator parameter.

[0166] In a possible implementation, the hybrid multi-agent soft actor-critic algorithm is used, based on a pre-trained strategy network, according to the state observation data of the distribution network agent and each microgrid agent, to solve the preset distribution-microgrid collaborative optimization model. The distribution-microgrid collaborative optimization model adopts a two-layer architecture of the distribution network layer and the microgrid layer; wherein the distribution network optimization scheduling model of the distribution network layer is set as a discrete-continuous agent including a Critic network, a discrete Actor network and a continuous Actor network, and the Critic network is used to guide the update of the discrete Actor network and the continuous Actor network; each microgrid optimization scheduling model of the microgrid layer is set as a double-layer architecture including a dual Crit The continuous intelligent agent of the ic network and the continuous Actor network is proposed, and a dual Critic network is used to guide the update of the continuous Actor network; the Critic network of the microgrid optimization scheduling model updates parameters by minimizing the Bellman residual, and the Actor network and temperature parameters are updated using the policy gradient with maximum entropy; the Critic network of the distribution network optimization scheduling model updates parameters through a centralized joint regression loss function, and introduces OU noise into the continuous Actor network of the distribution network optimization scheduling model, as well as a multi-head attention mechanism for selectively focusing on information from each microgrid optimization scheduling model in the Critic network of the distribution network optimization scheduling model.

[0167] Explanation: The hybrid multi-agent softactor critic (HMASAC) algorithm is used to solve the above distribution-microgrid collaborative optimization model based on the above pre-trained policy network. HMASAC follows the AC architecture and consists of a distribution-micro layer. There is a single MHAM-Critic network in the distribution network layer to guide the update of the discrete-continuous actor network, and there are dual critic networks in the microgrid layer to guide the update of the continuous actor network. The MHAM-Critic network is a critic network that introduces a multi-head attention mechanism.

[0168] The critic network of each microgrid agent updates its parameters by minimizing the Bellman residual, and the loss value is shown as follows:

[0169]

[0170] in, is a double Q network established to avoid overestimation of Q value; τ i is the sampling segment; D is the experience pool; φ 1,2 , They are the estimated and target Critic network parameters respectively; is the local observation space of the microgrid, Represents the boundary information from the distribution network agent; y i,t is the cumulative reward value, γ is the discount factor; a i,t is the action of the microgrid agent, α i is the temperature parameter; is the entropy value.

[0171] The Actor network and temperature parameters of each microgrid agent are updated using the policy gradient with maximum entropy, as shown in the following formula:

[0172]

[0173] Among them, τ is the sampling segment; D is the experience pool; is the local observation space of the microgrid, Represents the boundary information from the distribution network agent; α i is the temperature parameter; is the entropy value; θ i are the policy network parameters; is the target entropy value.

[0174] The critic network of the ADN discrete-continuous agent is updated through a centralized joint regression loss function as shown in the following formula:

[0175]

[0176] Among them, N ADN is the total number of discrete-continuous agents in ADN; τ is the sampling segment; D is the experience pool; is the local observation space of the distribution network, Represents the boundary information from the microgrid agent; a t is the action of the agent, y i,t is the cumulative reward value, γ is the discount factor, α is the temperature parameter; entropy function are the target strategy network parameters; H is the number of multi-head attention.

[0177] The loss function for updating the Actor network of discrete-continuous agents in the distribution network is shown in the following formula:

[0178]

[0179] Among them, A i (·) is the advantage function, which is used to judge the quality of the current action; α is the temperature parameter, τ is the sampling segment; D is the experience pool; is the local observation space of the microgrid, represents the boundary information from the distribution network agent, a t is the action of the agent, is the local observation space of the distribution network, represents the boundary information from the microgrid agent; \i is all agents except agent i; b(o t ,a \i,t ) is a state baseline that aims to highlight agents that are more helpful for the target task, thereby obtaining better learning strategies.

[0180] Introducing OU noise into the actor network of continuous agents to enhance exploration ability, strategy The decision can be rewritten as:

[0181]

[0182] in, is the observation space of the microgrid, Represents the boundary information from the distribution network agent; β is the sheared Gaussian noise added to the Actor network. OU noise, or Ornstein-Uhlenbeck noise, is a random process with regression characteristics. In reinforcement learning, the introduction of time-related OU noise can help the agent better explore the environment and find a better strategy.

[0183] According to the solution principle of the HMASAC algorithm, the overall algorithm flow of the distribution microgrid collaborative optimization model based on the HMASAC algorithm is designed as follows: first, the parameters and experience pool are initialized. In a parallel environment, each intelligent agent makes a decision based on observations and then acts on the power grid environment. After the power flow is calculated, each intelligent agent will obtain a reward value. It will be stored at time t so that it can be connected to the new observation at the next time t+1. All agents sample small batches from the experience pool D to update the neural network parameters, and the microgrid agent soft updates the dual critic network parameters.

[0184]

[0185] Soft update of parameters of critic network with multi-head attention mechanism for discrete-continuous agent in distribution network and the parameters of the Actor network As shown below:

[0186]

[0187] Among them, υ is the soft update factor, υ∈(0,1).

[0188] Exemplarily, when solving the preset distribution microgrid collaborative optimization model, the above model can be programmed and solved based on the PyTorch framework. In addition, by comparing with the non-pre-trained reinforcement learning optimization solution, it is verified that the pre-trained model has a faster training convergence speed and better training results; by comparing the multi-agent deep reinforcement learning method with and without the multi-head attention mechanism, it not only has a faster convergence speed and better training results during the training process, but also has a better ability to cope with environmental non-stationarity, and can also meet the needs of real-time decision-making of the dispatching system, and has obvious advantages in dealing with complex environmental problems. The HMASAC algorithm reduces the dimensionality explosion problem by selectively focusing on information from other agents, and has obvious advantages in flexibly accessing distributed resources on the distribution network side to achieve scalable learning.

[0189] The distribution network collaborative optimization scheduling method based on reinforcement learning efficient training technology proposed in the present invention is mainly aimed at the current coordinated operation of distribution microgrids, which needs to consider multi-agent privacy protection and non-convex model calculation, resulting in great challenges for model-based optimization operation methods. At the same time, uncertain factors such as renewable energy are also receiving more and more attention. Therefore, the present invention constructs a distribution microgrid collaborative optimization model with a high proportion of distributed resources, and adopts a hybrid multi-agent soft actor-critic algorithm to design the optimization problem as a distributed optimization problem in a multi-agent reinforcement learning environment, which effectively solves high-dimensional, non-convex and nonlinear optimization problems.

[0190] The distribution microgrid collaborative optimization scheduling method of the present invention has the following advantages: (1) The pre-training technology based on the generative adversarial imitation learning method can fully consider the complex state space and action space of the distribution microgrid collaborative optimization model, and alleviate the problem of low initial training efficiency of reinforcement learning; (2) The hybrid multi-agent soft actor critic algorithm is adopted to set the distribution network discrete-continuous agent and the microgrid continuous agent for the distribution microgrid collaborative optimization model, and construct a hybrid multi-agent model for high-proportion distributed resources; (3) The multi-head attention mechanism is adopted to achieve coordination between the distribution network discrete-continuous agents, and by selectively paying attention to the information from other agents, the problem of slow convergence speed of the model caused by dimensionality explosion is effectively alleviated. In addition, the method also supports the flexible access of distributed resources on the distribution network side to achieve scalable learning.

[0191] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0192] See also Figure 3 In another embodiment of the present invention, a distribution microgrid collaborative optimization dispatching system is provided, which can be used to implement the above-mentioned distribution microgrid collaborative optimization dispatching method. Specifically, the distribution microgrid collaborative optimization dispatching system includes a data acquisition module and a strategy generation module.

[0193] Among them, the data acquisition module is used to obtain the state observation data of the distribution network intelligent agent and each microgrid intelligent agent; the strategy generation module is used to adopt a hybrid multi-agent soft actor critic algorithm, based on a pre-trained strategy network, according to the state observation data of the distribution network intelligent agent and each microgrid intelligent agent, to solve the preset distribution-microgrid collaborative optimization model and obtain the control strategy of the distribution network and each microgrid.

[0194] Among them, the pre-trained policy network is obtained by the following method: the distribution microgrid collaborative optimization model is designed as a reinforcement learning model in a multi-agent environment, and the policy network of the reinforcement learning model is pre-trained based on the generative adversarial imitation learning method to obtain the pre-trained policy network; the distribution microgrid collaborative optimization model includes: each microgrid optimization scheduling model constructed with the minimum total production cost as the optimization goal and the distribution network optimization scheduling model constructed with the minimum total operating cost as the optimization goal.

[0195] In a possible implementation manner, the state observation data of the distribution network intelligent body includes: observation data of static VAR compensators in the distribution network, observation data of renewable energy units, observation data of micro gas turbines, observation data of energy storage systems, observation data of on-load tap-changing transformers, observation data of switching capacitor groups, the number of times the on-load tap-changing transformers have been actuated and the number of times the switching capacitor groups have been actuated, as well as the active load of the distribution network superimposed with a normal distribution error, a set of distribution network purchase and sale electricity prices, and boundary information from each microgrid intelligent body; the state observation data of the microgrid intelligent body includes: observation data of renewable energy units in the microgrid, observation data of micro gas turbines, and observation data of energy storage systems, as well as the active load of the microgrid superimposed with a normal distribution error and boundary information from the distribution network intelligent body.

[0196] In a possible implementation manner, the distribution network intelligent agent is configured in the following manner: the distribution network is virtually divided into a number of autonomous areas, a continuous intelligent agent is configured for each autonomous area, and separate intelligent agents are configured for independent and multi-level discrete devices in the distribution network, and the devices are combined with each other to obtain a distribution network intelligent agent; and each microgrid intelligent agent is configured in the following manner: an autonomous continuous intelligent agent is configured for each microgrid to obtain each microgrid intelligent agent.

[0197] In a possible implementation, the objective function of the microgrid optimization scheduling model is:

[0198]

[0199] Among them, F MG,i is the total production cost of microgrid i; T is the scheduling duration; is the number of micro gas turbines in microgrid i; is the power generation cost of the jth micro gas turbine in period t; is the number of energy storage units in microgrid i; is the power generation cost of the j-th energy storage unit in period t; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t.

[0200] The constraints of the microgrid optimization scheduling model include: renewable energy unit output constraints, micro gas turbine output constraints, energy storage system constraints, microgrid internal power flow balance constraints and distribution microgrid transmission power constraints.

[0201] The objective function of the distribution network optimization dispatching model is:

[0202]

[0203] Among them, F ADN is the total operating cost of the distribution network; and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and are the power generation costs of the jth micro gas turbine and energy storage unit in the distribution network during period t; F t OLTC and are the operating costs of the on-load tap-changing transformer and the j-th switching capacitor bank in the distribution network during period t respectively; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, F t B′-S′ is the cost of electricity purchased and sold from the distribution network to the main grid during period t.

[0204] The constraints of the distribution network optimization dispatching model include: on-load tap-changing transformer operation times constraint, capacitor bank operation times constraint, static VAR compensator output constraint, distribution network flow balance constraint, node voltage constraint and current carrying capacity constraint.

[0205] In a possible implementation, the design of the distribution-microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment includes: designing both the microgrid optimization scheduling model and the distribution network optimization scheduling model as reinforcement learning models according to the observation space, action space and reward function.

[0206] Observation Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for:

[0207]

[0208] in, and are the observation data of the energy storage system and the micro gas turbine of microgrid i in period t, and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; It is the boundary information from the distribution network intelligent agent in period t-1.

[0209] Action Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for:

[0210]

[0211] in, is the reactive power prediction value of the renewable energy unit in microgrid i during period t, and They are t The active power and reactive power of the micro gas turbine in the microgrid i in time period, is the charging and discharging active power of the energy storage system in microgrid i during period t.

[0212] Reward function of the reinforcement learning model for microgrid optimal dispatch model for:

[0213]

[0214] in, and are the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t respectively; F is the cost of microgrid i purchasing and selling electricity from the distribution network during period t; t TL is the penalty function for exceeding the transmission power constraint between distribution microgrids; 1,2 is the penalty weight for exceeding the limit; δ i F is the willingness coefficient of microgrid i to support the safe operation of distribution network; t penalty It is a penalty function for constraining the safe operation of distribution network.

[0215] Observation Space of Reinforcement Learning Model for Distribution Network Optimal Dispatch Model for:

[0216]

[0217] in, and They are the observation data of the on-load tap-changing transformer and the observation data of the switched capacitor bank, and are the observation data of the static VAR compensator, the observation data of the energy storage system and the observation data of the micro gas turbine of microgrid i, respectively. and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the electricity purchase and sales price set of the distribution network in period t; is the boundary information from the microgrid agent i.

[0218] The action space of the reinforcement learning model of the distribution network optimization dispatching model is:

[0219]

[0220] in, and is the discrete action space and continuous action space in region i of the distribution network, and They are the gear positions of the on-load tap-changing transformer and the gear positions of the switching capacitor bank in area i of the distribution network during period t; is the reactive power prediction value of the renewable energy unit i in microgrid during period t, and are the active power and reactive power of the micro gas turbine in microgrid i at time period t, is the charging and discharging active power of the energy storage system of microgrid i during period t, is the reactive power output by the static VAR compensator of microgrid i during period t.

[0221] Reward function of reinforcement learning model for distribution network optimal dispatch model for:

[0222]

[0223] Among them, F t OLTC and are the operating cost of the on-load tap-changing transformer in period t and the operating cost of the j-th switching capacitor bank, respectively. and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and is the power generation cost of the j-th micro gas turbine and the j-th energy storage unit in period t, F t B′-S′is the cost of electricity purchase and sale from the distribution network to the main grid, is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, It is the penalty function for exceeding the limit of distribution network voltage and branch power flow constraints.

[0224] In a possible implementation, the hybrid multi-agent soft actor-critic algorithm is used, based on a pre-trained strategy network, according to the state observation data of the distribution network agent and each microgrid agent, to solve the preset distribution-microgrid collaborative optimization model. The distribution-microgrid collaborative optimization model adopts a two-layer architecture of the distribution network layer and the microgrid layer; wherein the distribution network optimization scheduling model of the distribution network layer is set as a discrete-continuous agent including a Critic network, a discrete Actor network and a continuous Actor network, and the Critic network is used to guide the update of the discrete Actor network and the continuous Actor network; each microgrid optimization scheduling model of the microgrid layer is set as a double-layer architecture including a dual Crit The continuous intelligent agent of the ic network and the continuous Actor network is proposed, and a dual Critic network is used to guide the update of the continuous Actor network; the Critic network of the microgrid optimization scheduling model updates parameters by minimizing the Bellman residual, and the Actor network and temperature parameters are updated using the policy gradient with maximum entropy; the Critic network of the distribution network optimization scheduling model updates parameters through a centralized joint regression loss function, and introduces OU noise into the continuous Actor network of the distribution network optimization scheduling model, as well as a multi-head attention mechanism for selectively focusing on information from each microgrid optimization scheduling model in the Critic network of the distribution network optimization scheduling model.

[0225] All relevant contents of each step involved in the aforementioned embodiment of the distribution microgrid collaborative optimization scheduling method can be referred to the functional description of the functional modules corresponding to the distribution microgrid collaborative optimization scheduling system in the embodiment of the present invention, and will not be repeated here.

[0226] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0227] In another embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the coordinated optimization scheduling method of the distribution microgrid.

[0228] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in a computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the method for coordinated optimization and scheduling of distribution microgrids in the above embodiment.

[0229] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0230] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0231] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0232] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0233] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for coordinated optimization and dispatching of distribution and microgrids, characterized in that: include: Obtain state observation data of distribution network intelligent agents and each microgrid intelligent agent; A hybrid multi-agent soft actor-critic algorithm is used. Based on the pre-trained strategy network, the preset distribution-microgrid collaborative optimization model is solved according to the state observation data of the distribution network agent and each microgrid agent, and the control strategy of the distribution network and each microgrid is obtained. Among them, the pre-trained policy network is obtained by the following method: The distribution microgrid collaborative optimization model is designed as a reinforcement learning model in a multi-agent environment, and the policy network of the reinforcement learning model is pre-trained based on the generative adversarial imitation learning method to obtain a pre-trained policy network. The distribution microgrid collaborative optimization model includes: The optimal dispatching models of each microgrid are constructed with the minimum total production cost as the optimization objective, and the optimal dispatching model of the distribution network is constructed with the minimum total operating cost as the optimization objective.

2. The coordinated optimization dispatching method for distribution and microgrids according to claim 1 is characterized in that: The state observation data of the distribution network intelligent body includes: Observation data of static VAR compensators in the distribution network, observation data of renewable energy units, observation data of micro gas turbines, observation data of energy storage systems, observation data of on-load tap-changing transformers, observation data of switching capacitor banks, the number of times the on-load tap-changing transformers have been operated and the number of times the switching capacitor banks have been operated, as well as the active load of the distribution network with normal distribution errors superimposed, the distribution network purchase and sale electricity price set, and the boundary information from each microgrid intelligent body; The state observation data of the microgrid intelligent body includes: observation data of renewable energy units in the microgrid, observation data of micro gas turbines and observation data of energy storage systems, as well as active load of the microgrid superimposed with normal distribution errors and boundary information from the distribution network intelligent body.

3. The coordinated optimization dispatching method for distribution and microgrids according to claim 1 is characterized in that: The distribution network intelligent body is set up in the following manner: the distribution network is virtually divided into a number of autonomous areas, a continuous intelligent body is set up for each autonomous area, and a separate intelligent body is set up for independent and multi-level discrete devices in the distribution network, and the distribution network intelligent body is obtained by combining them with each other; each microgrid intelligent body is set up in the following manner: an autonomous continuous intelligent body is set up for each microgrid to obtain each microgrid intelligent body.

4. The coordinated optimization dispatching method for distribution and microgrids according to claim 1 is characterized in that: The objective function of the microgrid optimization scheduling model is: Among them, F MG,i is the total production cost of microgrid i; T is the scheduling duration; is the number of micro gas turbines in microgrid i; is the power generation cost of the jth micro gas turbine in period t; is the number of energy storage units in microgrid i; is the power generation cost of the j-th energy storage unit in period t; The cost of microgrid i purchasing and selling electricity from the distribution network during period t; The constraints of the microgrid optimization scheduling model include: Renewable energy unit output constraints, micro gas turbine output constraints, energy storage system constraints, microgrid internal power flow balance constraints and distribution microgrid transmission power constraints; The objective function of the distribution network optimization dispatching model is: Among them, F ADN is the total operating cost of the distribution network; and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and are the power generation costs of the jth micro gas turbine and energy storage unit in the distribution network during period t respectively; and are the operating costs of the on-load tap-changing transformer and the j-th switching capacitor bank in the distribution network during period t respectively; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, is the cost of electricity purchased and sold from the distribution network to the main grid during period t; The constraints of the distribution network optimization dispatching model include: Constraints on the number of operations of on-load tap-changing transformers, the number of operations of switching capacitor banks, the output constraints of static VAR compensators, the power flow balance constraints of distribution networks, the node voltage constraints and the current carrying capacity constraints.

5. The coordinated optimization dispatching method for distribution and microgrids according to claim 1 is characterized in that: The design of the distribution microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment includes: The microgrid optimization dispatch model and the distribution network optimization dispatch model are designed as reinforcement learning models according to the observation space, action space and reward function; Observation Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for: in, and are the observation data of the energy storage system and the micro gas turbine of microgrid i in period t, and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the boundary information from the distribution network intelligent agent in period t-1; Action Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for: in, is the reactive power prediction value of the renewable energy unit in microgrid i during period t, and are the active power and reactive power of the micro gas turbine in microgrid i during period t, is the charging and discharging active power of the energy storage system in microgrid i during period t; Reward function of the reinforcement learning model for microgrid optimal dispatch model for: in, and are the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t respectively; The cost of microgrid i purchasing and selling electricity from the distribution network during period t; is the penalty function for exceeding the transmission power constraint between distribution microgrids; 1,2 is the penalty weight for exceeding the limit; δ i is the willingness coefficient of microgrid i to support the safe operation of distribution network; Constraints for over-limit penalty functions for safe operation of distribution networks; Observation Space of Reinforcement Learning Model for Distribution Network Optimal Dispatch Model for: in, and They are the observation data of the on-load tap-changing transformer and the observation data of the switched capacitor bank, and are the observation data of the static VAR compensator, the observation data of the energy storage system and the observation data of the micro gas turbine of microgrid i, respectively. and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the electricity purchase and sales price set of the distribution network in period t; is the boundary information from the microgrid agent i; The action space of the reinforcement learning model of the distribution network optimization dispatching model is: in, and is the discrete action space and continuous action space in region i of the distribution network, and They are the gear positions of the on-load tap-changing transformer and the gear positions of the switching capacitor bank in area i of the distribution network during period t; is the reactive power prediction value of the renewable energy unit i in microgrid during period t, and are the active power and reactive power of the micro gas turbine in microgrid i at time period t, is the charging and discharging active power of the energy storage system of microgrid i during period t, is the reactive power output by the static VAR compensator of microgrid i during period t; Reward function of reinforcement learning model for distribution network optimal dispatch model for: in, and are the operating cost of the on-load tap-changing transformer in period t and the operating cost of the j-th switching capacitor bank, respectively. and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and is the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t, is the cost of electricity purchase and sale from the distribution network to the main grid, is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, It is the penalty function for exceeding the limit of distribution network voltage and branch power flow constraints.

6. The coordinated optimization dispatching method for distribution and microgrids according to claim 1 is characterized in that: The hybrid multi-agent soft actor critic algorithm is based on a pre-trained strategy network and solves the preset distribution-microgrid collaborative optimization model according to the state observation data of the distribution network agent and each microgrid agent. The distribution-microgrid collaborative optimization model adopts a double-layer architecture of the distribution network layer and the microgrid layer; Among them, the distribution network optimization dispatching model at the distribution network layer is set as a discrete-continuous intelligent agent including a critic network, a discrete actor network and a continuous actor network, and the critic network is used to guide the update of the discrete actor network and the continuous actor network; the optimization dispatching model of each microgrid at the microgrid layer is set as a continuous intelligent agent including a dual critic network and a continuous actor network, and the dual critic network is used to guide the update of the continuous actor network; The Critic network of the microgrid optimal scheduling model updates parameters by minimizing the Bellman residual, and the Actor network and temperature parameters are updated using the policy gradient with maximum entropy. The Critic network of the distribution network optimal scheduling model updates parameters through a centralized joint regression loss function, and introduces OU noise into the continuous Actor network of the distribution network optimal scheduling model. In addition, a multi-head attention mechanism is introduced into the Critic network of the distribution network optimal scheduling model to selectively focus on the information from each microgrid optimal scheduling model.

7. A distribution microgrid collaborative optimization dispatching system, characterized in that: include: A data acquisition module is used to obtain state observation data of the distribution network intelligent agent and each microgrid intelligent agent; The strategy generation module is used to adopt the hybrid multi-agent soft actor-critic algorithm, based on the pre-trained strategy network, and solve the preset distribution-microgrid collaborative optimization model according to the state observation data of the distribution network agent and each microgrid agent, and obtain the control strategy of the distribution network and each microgrid; Among them, the pre-trained policy network is obtained by the following method: The distribution microgrid collaborative optimization model is designed as a reinforcement learning model in a multi-agent environment, and the policy network of the reinforcement learning model is pre-trained based on the generative adversarial imitation learning method to obtain a pre-trained policy network. The distribution microgrid collaborative optimization model includes: The optimal dispatching models of each microgrid are constructed with the minimum total production cost as the optimization objective, and the optimal dispatching model of the distribution network is constructed with the minimum total operating cost as the optimization objective.

8. The coordinated optimization dispatching system for distribution and microgrids according to claim 7 is characterized in that: The state observation data of the distribution network intelligent body includes: Observation data of static VAR compensators in the distribution network, observation data of renewable energy units, observation data of micro gas turbines, observation data of energy storage systems, observation data of on-load tap-changing transformers, observation data of switching capacitor banks, the number of times the on-load tap-changing transformers have been operated and the number of times the switching capacitor banks have been operated, as well as the active load of the distribution network with normal distribution errors superimposed, the distribution network purchase and sale electricity price set, and the boundary information from each microgrid intelligent body; The state observation data of the microgrid intelligent body includes: observation data of renewable energy units in the microgrid, observation data of micro gas turbines and observation data of energy storage systems, as well as active load of the microgrid superimposed with normal distribution errors and boundary information from the distribution network intelligent body.

9. According to claim 7, the distribution and microgrid collaborative optimization dispatching system is characterized in that the distribution network intelligent body is set in the following manner: the distribution network is virtually divided into a number of autonomous areas, a continuous intelligent body is set for each autonomous area, and a separate intelligent body is set for each independent and multi-level discrete device in the distribution network, and the distribution network intelligent body is obtained by combining them with each other; each microgrid intelligent body is set in the following manner: an autonomous continuous intelligent body is set for each microgrid to obtain each microgrid intelligent body.

10. The distribution and microgrid coordinated optimization dispatching system according to claim 7, characterized in that: The objective function of the microgrid optimization scheduling model is: Among them, F MG,i is the total production cost of microgrid i; T is the scheduling duration; is the number of micro gas turbines in microgrid i; is the power generation cost of the jth micro gas turbine in period t; is the number of energy storage units in microgrid i; is the power generation cost of the j-th energy storage unit in period t; The cost of microgrid i purchasing and selling electricity from the distribution network during period t; The constraints of the microgrid optimization scheduling model include: Renewable energy unit output constraints, micro gas turbine output constraints, energy storage system constraints, microgrid internal power flow balance constraints and distribution microgrid transmission power constraints; The objective function of the distribution network optimization dispatching model is: Among them, F ADN is the total operating cost of the distribution network; and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and are the power generation costs of the jth micro gas turbine and energy storage unit in the distribution network during period t respectively; and are the operating costs of the on-load tap-changing transformer and the j-th switching capacitor bank in the distribution network during period t respectively; is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, is the cost of electricity purchased and sold from the distribution network to the main grid during period t; The constraints of the distribution network optimization dispatching model include: Constraints on the number of operations of on-load tap-changing transformers, the number of operations of switching capacitor banks, the output constraints of static VAR compensators, the power flow balance constraints of distribution networks, the node voltage constraints and the current carrying capacity constraints.

11. The distribution and microgrid coordinated optimization dispatching system according to claim 7, characterized in that: The design of the distribution microgrid collaborative optimization model as a reinforcement learning model in a multi-agent environment includes: The microgrid optimization dispatch model and the distribution network optimization dispatch model are designed as reinforcement learning models according to the observation space, action space and reward function; Observation Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for: in, and are the observation data of the energy storage system and the micro gas turbine of microgrid i in period t, and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the boundary information from the distribution network intelligent agent in period t-1; Action Space of Reinforcement Learning Model for Microgrid Optimal Dispatch Model for: in, is the reactive power prediction value of the renewable energy unit in microgrid i during period t, and are the active power and reactive power of the micro gas turbine in microgrid i during period t, is the charging and discharging active power of the energy storage system in microgrid i during period t; Reward function of the reinforcement learning model for microgrid optimal dispatch model for: in, and are the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t respectively; The cost of microgrid i purchasing and selling electricity from the distribution network during period t; is the penalty function for exceeding the transmission power constraint between distribution microgrids; 1,2 is the penalty weight for exceeding the limit; δ i is the willingness coefficient of microgrid i to support the safe operation of distribution network; Constraints for over-limit penalty functions for safe operation of distribution networks; Observation Space of Reinforcement Learning Model for Distribution Network Optimal Dispatch Model for: in, and They are the observation data of the on-load tap-changing transformer and the observation data of the switched capacitor bank, and are the observation data of the static VAR compensator, the observation data of the energy storage system and the observation data of the micro gas turbine of microgrid i, respectively. and are the observed data and active load of the renewable energy unit of microgrid i with the normal distribution error superimposed in period t; is the electricity purchase and sales price set of the distribution network in period t; is the boundary information from the microgrid agent i; The action space of the reinforcement learning model of the distribution network optimization dispatching model is: in, and is the discrete action space and continuous action space in region i of the distribution network, and They are the gear positions of the on-load tap-changing transformer and the gear positions of the switching capacitor bank in area i of the distribution network during period t; is the reactive power prediction value of the renewable energy unit i in microgrid during period t, and are the active power and reactive power of the micro gas turbine in microgrid i at time period t, is the charging and discharging active power of the energy storage system of microgrid i during period t, is the reactive power output by the static VAR compensator of microgrid i during period t; Reward function of reinforcement learning model for distribution network optimal dispatch model for: in, and are the operating cost of the on-load tap-changing transformer in period t and the operating cost of the j-th switching capacitor bank, respectively. and N MG are the number of micro gas turbines, energy storage systems, switched capacitor banks, and microgrids in the distribution network, respectively; and is the power generation cost of the j-th micro gas turbine and the power generation cost of the j-th energy storage unit in period t, is the cost of electricity purchase and sale from the distribution network to the main grid, is the cost of microgrid i purchasing and selling electricity from the distribution network during period t, It is the penalty function for exceeding the limit of distribution network voltage and branch power flow constraints.

12. The distribution and microgrid coordinated optimization dispatching system according to claim 7, characterized in that: The hybrid multi-agent soft actor critic algorithm is based on a pre-trained strategy network and solves the preset distribution-microgrid collaborative optimization model according to the state observation data of the distribution network agent and each microgrid agent. The distribution-microgrid collaborative optimization model adopts a double-layer architecture of the distribution network layer and the microgrid layer; Among them, the distribution network optimization dispatching model at the distribution network layer is set as a discrete-continuous intelligent agent including a critic network, a discrete actor network and a continuous actor network, and the critic network is used to guide the update of the discrete actor network and the continuous actor network; the optimization dispatching model of each microgrid at the microgrid layer is set as a continuous intelligent agent including a dual critic network and a continuous actor network, and the dual critic network is used to guide the update of the continuous actor network; The Critic network of the microgrid optimal scheduling model updates parameters by minimizing the Bellman residual, and the Actor network and temperature parameters are updated using the policy gradient with maximum entropy. The Critic network of the distribution network optimal scheduling model updates parameters through a centralized joint regression loss function, and introduces OU noise into the continuous Actor network of the distribution network optimal scheduling model. In addition, a multi-head attention mechanism is introduced into the Critic network of the distribution network optimal scheduling model to selectively focus on the information from each microgrid optimal scheduling model.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the distribution microgrid collaborative optimization scheduling method as described in any one of claims 1 to 6 are implemented.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for coordinated optimization and scheduling of distribution and microgrids as claimed in any one of claims 1 to 6 are implemented.

Images