Active power distribution network cloud-side collaborative day-ahead scheduling method, device, equipment and medium

Abstract

The invention relates to an active power distribution network cloud-side collaborative day-ahead scheduling method, device, equipment and medium, and the method comprises the steps: building an ADN distributed day-ahead economic scheduling model with the optimal daily operation cost of an active power distribution network as a target under the condition of considering the carbon emission and the interaction cost of a main power grid; under the constraint condition of the ADN distributed day-ahead economic dispatching model, establishing a multi-agent distribution deep reinforcement learning model based on an MADDPG algorithm; the multi-agent distribution deep reinforcement learning model is trained in the cloud server, and the trained multi-agent distribution deep reinforcement learning model is sent to each agent edge server; and in each control period, each agent edge server adopts the received trained multi-agent distribution deep reinforcement learning model to perform active power distribution network dispatching control of the sub-region according to real-time local observation information. The method is suitable for an ADN operation scene with small edge calculation amount and low communication requirement.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology for distribution networks, and in particular to a method, device, equipment and medium for active distribution network cloud-edge collaborative day-ahead scheduling. Background Art

[0002] In recent years, the power Internet of Things (Power IoT) and cloud computing / edge computing have developed rapidly, providing strong support for active distribution networks (ADNs) and significantly improving the overall operation performance of ADNs. Cloud computing has great potential in terms of high efficiency, massive data processing, and resource optimization and scheduling, but it also faces problems such as long latency and network dependence, data security and privacy risks, and high computing costs. Edge computing has the advantages of low latency, low bandwidth usage, strong security and privacy, and real-time decision-making and response. If the two are combined and applied to ADN operations, their respective advantages can be fully utilized to provide a comprehensive solution with low latency, high efficiency, enhanced security and privacy protection, and strong economy.

[0003] Although centralized control can effectively improve the operating performance of the distribution network, it also has disadvantages such as extended cloud remote control time, high network communication bandwidth and data security requirements, and high construction and use costs, which affect its application in scenarios with limited distribution network infrastructure resources such as weak networks and low computing power. Compared with centralized control, distributed control of distribution networks has advantages in time delay, privacy security, bandwidth occupancy, reliability and many other aspects. At present, most distributed controls require partial communication between intelligent agents. When the communication bandwidth is affected or even fails, the control performance will be greatly affected. Summary of the invention

[0004] The present invention provides an active distribution network cloud-edge collaborative day-ahead scheduling method, device, equipment and medium to address the problems of insufficient edge computing capability and limited communication resources in the distribution network.

[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide an active distribution network cloud-edge collaborative day-ahead scheduling method, including the following steps:

[0006] Taking carbon emissions and the interaction costs of the main power grid into consideration, a distributed day-ahead economic dispatch model for active distribution networks is established with the goal of optimizing the daily operating cost of the active distribution network.

[0007] Under the constraints of the distributed day-ahead economic dispatch model of the active distribution network, a multi-agent distributed deep reinforcement learning model is established based on a multi-agent deep deterministic policy gradient algorithm;

[0008] The multi-agent distributed deep reinforcement learning model is trained on the cloud server, and after the training is completed, the trained multi-agent distributed deep reinforcement learning model is sent to each agent edge server;

[0009] In each control cycle, each intelligent edge server uses the received trained multi-agent distributed deep reinforcement learning model to perform active distribution network dispatching and control in the sub-region based on real-time local observation information.

[0010] The objective function of the distributed day-ahead economic dispatch model of the active distribution network is: Among them, C ADN is the daily operating cost of the active distribution network, C ADN,n is the daily operating cost of agent n, N agent is the total number of agents in the active distribution network, C grid (t) is the main grid cost at time t, C es,i (t) is the operating cost of energy storage system i at time t, C gt,j (t) is the operating cost of gas generator j at time t, C loss,k (t) is the network loss cost of branch k at time t.

[0011] The main grid cost at time t is expressed as: The operating cost of the energy storage system i at time t is expressed as: C es,i (t)=∑ i c 1coe ·abs(P es,i (t))·T d ; The operating cost of gas generator j at time t is expressed as: The network loss cost of branch k at time t is expressed as: C loss,k (t) = ∑ k c g (k) I k (t) 2 ·r k ·T d ; Among them, c g (k) is the unit time-of-use electricity price of the main power grid, P pcc (k) is the main grid power, T d To control the cycle, is the carbon emission cost of the main power grid, expressed as: c co2 is the unit carbon emission cost, E g (k) is the carbon emission per unit time at the common coupling point, expressed as: E g (k) = α co2 ·P pcc (k)·T d , αco2 is the CO2 emission coefficient per unit energy of the main grid, To track the allowable value of carbon emissions per unit time at the common coupling point, is the penalty cost for interaction between the active distribution network and the main grid, expressed as: c penal is the unit penalty cost of interaction energy, abs() is the absolute value function; c 1coe is the unit energy conversion cost of energy storage, P es,i (t) is the power of energy storage system i at time t; c gt (t) is the unit operating cost of the gas generator at time t, P gt,j (t) is the power of gas generator j at time t, is the carbon emission cost of gas generator j at time t, expressed as: E gt,j (t) is the carbon emission of the gas generator at time t, is the allowable value of carbon emissions per unit time of the gas generator, I k (t) is the branch current at time t, r k is the branch resistance.

[0012] The constraints of the distributed day-ahead economic dispatch model of the active distribution network include the power flow constraints of the distribution network, the transmission power constraints of the active distribution network master node tracking the common coupling point, the power constraints of the energy storage system, the energy constraints of the energy storage system, the output constraints of the gas generator, the reactive power output constraints of new energy, the voltage constraints of each node in the intelligent agent n, and the power constraints of each branch in the intelligent agent n.

[0013] The state space of the multi-agent distributed deep reinforcement learning model includes the state space s of the active power distribution network at time t t and the observation space of each agent n Among them, the state space s of the active distribution network at time t t Expressed as: s t ={t,c g ,SOC,P g ,Q g ,P rew ,P L ,Q L ,U t ,S t}, where t is time, c g is the unit time-of-use electricity price of the main power grid, SOC is the charge state set of the energy storage in the active distribution network at time t, P g is the active power interacting between the common coupling point and the upper level, Q g is the reactive power of the common coupling point interacting with the upstream, P rewis the active power set of renewable energy in the active distribution network; P L and Q L are respectively the active and reactive sets of each node load; U t and S t are the node voltage set and branch power set respectively; the observation space of each agent n It is expressed as: Among them, SOC is the state of charge set of the energy stored in agent n at time t, is the active power of the interaction between the common coupling point in the nth agent and the superior, is the reactive power of the interaction between the common coupling point in the nth agent and the upper level, is the set of new energy active power in intelligent agent n; and are respectively the active set and reactive set of each node load in the intelligent agent n; and are the node voltage set and branch power set in agent n respectively; the action space of the multi-agent distributed deep reinforcement learning model It is expressed as: in, is the active output of the energy storage system in agent n, is the reactive output of the energy storage system in agent n, is the active output of the gas generator in agent n, is the reactive power output of the new energy source in agent n; the reward function r of the multi-agent distributed deep reinforcement learning model n Expressed as: r n =-C ADN,n , where C ADN,n is the daily operating cost of agent n.

[0014] The constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model include action space constraints and state space constraints, wherein the action space constraints include the power constraints of the energy storage system of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system, the output constraints of the gas generator and the reactive output constraints of new energy; the state space constraints include the transmission power constraints at the common coupling point tracked by the main node of the active distribution network of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system at the end of each day, the voltage constraints of each node within the agent n and the power constraints of each branch within the agent n.

[0015] The active distribution network cloud-edge collaborative day-ahead scheduling method further includes: a step of processing the constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model, specifically:

[0016] The state of charge constraint of the energy storage system and the reactive output constraint of the new energy are mapped to the output range of the intelligent agent neural network respectively by means of coefficient linear mapping; if the inequality constraint conditions of the rated capacity of the energy storage converter and the rated capacity of the new energy converter are not satisfied after mapping, the active power and reactive power of the energy storage system and the new energy are designed to be nonlinearly transformed;

[0017] Modify the reward function r of each agent n n A penalty term is added to obtain a new reward function r′ for each agent n. The new reward function r′ increases the penalty intensity when the agent approaches the constraint boundary in a nonlinear way for the state space of the multi-agent distributed deep reinforcement learning model.

[0018] The new reward function r' of each agent n is expressed as: r' = -(C ADN,n +C penal ), where C penal is the penalty term, expressed as: in, is the unit penalty cost of energy exchange with the upper grid, is the unit penalty cost when the state of charge of the energy storage exceeds the allowed range at the end of the day, is the unit penalty cost for each node voltage exceeding the allowable range, The unit penalty cost for each branch transmission power exceeding the allowed range, SOC es,i (24) is the state of charge at the 24th hour of the energy storage day, U k is the voltage at node k, S ij is the power of branch i to j, f() is the barrier function, which is expressed as: x min and x max are the lower and upper limits of the variable x, respectively, and ε a is the penalty intensity coefficient.

[0019] The technical solution adopted by the present invention to solve the technical problem is: to provide an active distribution network cloud-edge collaborative day-ahead dispatching device, comprising:

[0020] The first building module is used to establish a distributed day-ahead economic dispatch model for the active distribution network with the goal of optimizing the daily operation cost of the active distribution network, taking into account carbon emissions and the interaction cost of the main power grid;

[0021] A second establishment module is used to establish a multi-agent distributed deep reinforcement learning model based on a multi-agent deep deterministic policy gradient algorithm under the constraints of the distributed day-ahead economic dispatch model of the active distribution network;

[0022] A training module is used to train the multi-agent distributed deep reinforcement learning model on the cloud server, and after the training is completed, the trained multi-agent distributed deep reinforcement learning model is sent to each agent edge server;

[0023] The scheduling module is used to perform active distribution network scheduling and control in the sub-region in each control cycle, based on the real-time local observation information and the received trained multi-agent distributed deep reinforcement learning model by each intelligent edge server.

[0024] The objective function of the distributed day-ahead economic dispatch model of the active distribution network established by the first establishment module is: Among them, C ADN is the daily operating cost of the active distribution network, C ADN,n is the daily operating cost of agent n, N agent is the total number of agents in the active distribution network, C grid (t) is the main grid cost at time t, C es,i (t) is the operating cost of energy storage system i at time t, C gt,j (t) is the operating cost of gas generator j at time t, C loss,k (t) is the network loss cost of branch k at time t.

[0025] The main grid cost at time t is expressed as: The operating cost of the energy storage system i at time t is expressed as: C es,i (t) = ∑ i c 1coe ·abs(P es,i (t))·T d ; The operating cost of gas generator j at time t is expressed as: The network loss cost of branch k at time t is expressed as: C loss,k (t) = ∑ k c g (k) I k (t) 2 ·r k ·T d ; Among them, c g (k) is the unit time-of-use electricity price of the main power grid, P pcc (k) is the main grid power, T d To control the cycle, is the carbon emission cost of the main power grid, expressed as: c co2 is the unit carbon emission cost, E g (k) is the carbon emission per unit time at the common coupling point, expressed as: E g (k) = α co2 ·P pcc (k)·Td , α co2 is the CO2 emission coefficient per unit energy of the main grid, To track the allowable value of carbon emissions per unit time at the common coupling point, is the penalty cost for interaction between the active distribution network and the main grid, expressed as: c penal is the unit penalty cost of interaction energy, abs() is the absolute value function; c 1coe is the unit energy conversion cost of energy storage, P es,i (t) is the power of energy storage system i at time t; c gt (t) is the unit operating cost of the gas generator at time t, P gt,j (t) is the power of gas generator j at time t, is the carbon emission cost of gas generator j at time t, expressed as: E gt,j (t) is the carbon emission of the gas generator at time t, is the allowable value of carbon emissions per unit time of the gas generator, I k (t) is the branch current at time t, r k is the branch resistance.

[0026] The constraints of the distributed day-ahead economic dispatch model of the active distribution network established by the first establishment module include the power flow constraints of the distribution network, the transmission power constraints of the active distribution network master node tracking the common coupling point, the power constraints of the energy storage system, the energy constraints of the energy storage system, the output constraints of the gas generator, the reactive output constraints of new energy, the voltage constraints of each node within the intelligent body n, and the power constraints of each branch within the intelligent body n.

[0027] The state space of the multi-agent distributed deep reinforcement learning model established by the second establishment module includes the state space s of the active distribution network at time t t and the observation space of each agent n Among them, the state space s of the active distribution network at time t t Expressed as: s t ={t,c g ,SOC,P g ,Q g ,P rew ,P L ,Q L ,U t ,S t}, where t is time, c g is the unit time-of-use electricity price of the main power grid, SOC is the charge state set of the energy storage in the active distribution network at time t, P g is the active power interacting between the common coupling point and the upper level, Q gis the reactive power of the common coupling point interacting with the upstream, P rew is the active power set of renewable energy in the active distribution network; P L and Q L are respectively the active and reactive sets of each node load; U t and S t are the node voltage set and branch power set respectively; the observation space of each agent n It is expressed as: Among them, SOC is the state of charge set of the energy stored in agent n at time t, is the active power of the interaction between the common coupling point in the nth agent and the superior, is the reactive power of the interaction between the common coupling point in the nth agent and the upper level, is the set of new energy active power in intelligent agent n; and are respectively the active set and reactive set of each node load in the intelligent agent n; and are the node voltage set and branch power set in agent n respectively; the action space of the multi-agent distributed deep reinforcement learning model established by the second establishment module It is expressed as: in, is the active output of the energy storage system in agent n, is the reactive output of the energy storage system in agent n, is the active output of the gas generator in agent n, is the reactive power output of the new energy source in agent n; the reward function r of the multi-agent distributed deep reinforcement learning model established by the second establishment module n Expressed as: r n =-C ADN,n , where C ADN,n is the daily operating cost of agent n.

[0028] The constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model established by the second establishment module include action space constraints and state space constraints, wherein the action space constraints include the power constraints of the energy storage system of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system, the output constraints of the gas generator and the reactive output constraints of new energy; the state space constraints include the transmission power constraints at the common coupling point tracked by the main node of the active distribution network of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system at the end of each day, the voltage constraints of each node within the intelligent agent n and the power constraints of each branch within the intelligent agent n.

[0029] The second establishing module comprises:

[0030] The first constraint processing unit is used to map the state of charge constraint of the energy storage system and the reactive output constraint of the new energy to the output range of the intelligent agent neural network respectively by means of coefficient linear mapping; if the inequality constraint conditions of the rated capacity of the energy storage converter and the rated capacity of the new energy converter are not satisfied after mapping, a nonlinear transformation is designed for the active power and reactive power of the energy storage system and the new energy;

[0031] The second constraint processing unit is used to modify the reward function r of each agent n n A penalty term is added to obtain a new reward function r′ for each agent n. The new reward function r′ increases the penalty intensity when the agent approaches the constraint boundary in a nonlinear way for the state space of the multi-agent distributed deep reinforcement learning model.

[0032] The new reward function r' of each agent n is expressed as: r' = -(C ADN,n +C penal ), where C penal is the penalty term, expressed as: in, is the unit penalty cost of energy exchange with the upper grid, is the unit penalty cost when the state of charge of the energy storage exceeds the allowed range at the end of the day, is the unit penalty cost for each node voltage exceeding the allowable range, The unit penalty cost for each branch transmission power exceeding the allowed range, SOC es,i (24) is the state of charge at the 24th hour of the energy storage day, U k is the voltage at node k, S ij is the power of branch i to j, f() is the barrier function, which is expressed as: x min and x max are the lower and upper limits of the variable x, respectively, and ε a is the penalty intensity coefficient.

[0033] The technical solution adopted by the present invention to solve its technical problem is: to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned active distribution network cloud-edge collaborative day-ahead scheduling method when executing the computer program.

[0034] The technical solution adopted by the present invention to solve its technical problem is: providing a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned active distribution network cloud-edge collaborative day-ahead scheduling method are implemented.

[0035] Beneficial Effects

[0036] Due to the adoption of the above-mentioned technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention establishes an ADN distributed day-ahead economic dispatch model that takes into account carbon emissions and the interaction costs of the main power grid and takes the optimal daily operating cost of ADN as the goal. At the same time, a multi-agent distributed deep reinforcement learning model is established based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm. Finally, a cloud-edge collaborative day-ahead dispatching method for the distribution network based on MADDPG is given. This method does not require communication with other agents, which can not only effectively protect the privacy of each agent, but also is not affected by the communication bandwidth. It has obvious advantages in edge computing amount, computing time, communication volume, data privacy protection, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the cloud-edge collaborative architecture of the active distribution network;

[0038] Figure 2 It is a flow chart of the cloud-edge collaborative day-ahead scheduling method for active distribution network according to the first embodiment of the present invention;

[0039] Figure 3 is a training flow chart of a multi-agent distributed deep reinforcement learning model in the first embodiment of the present invention;

[0040] Figure 4 is a reward value curve diagram of the MADDPG training process in an embodiment of the present invention;

[0041] Figure 5 is a result diagram obtained by using the method of the first implementation mode in an embodiment of the present invention;

[0042] Figure 6 1 is a comparison chart of results obtained by using different methods in the embodiments of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0044] Taking the improved IEEE 33 node as an example, the ADN edge-cloud collaborative architecture is as follows: Figure 1 As shown, ADN is divided into multiple regions, each region is an intelligent agent.

[0045] The first embodiment of the present invention relates to a cloud-edge collaborative day-ahead scheduling method for an active distribution network, which performs model training through a cloud server (cloud server, CS), and then performs real-time scheduling and control on each server (edge ​​server, ES). In the deep reinforcement learning (DRL) model training phase, CS relies on its computing and storage advantages to accelerate the model training process, and can be dynamically adjusted according to demand to optimize the model training effect and performance. In the DRL model execution segment, the good DRL model in the cloud is distributed to the ES of each intelligent agent. In each control cycle, each intelligent agent ES performs ADN control of the sub-region based on local information, thereby realizing collaborative control of ADN on multiple sides. In the operation control phase, each intelligent runs independently and does not need to interact with information, so as to cope with the problems of security and privacy protection and occupation of a large amount of communication resources brought about by ADN energy management under centralized reinforcement learning. For example Figure 2 As shown, the specific steps include:

[0046] Step 1: Taking carbon emissions and the interaction costs of the main power grid into consideration, a distributed day-ahead economic dispatch model for the active distribution network is established with the goal of optimizing the daily operation cost of the active distribution network.

[0047] This implementation method uses multiple intelligent devices to control the safe operation of ADN. Since the flowmeters have multiple reactive capabilities, ADN can be divided into multiple intelligent entities, each of which is measured by the active and reactive power of its energy system, and the reactive power of photovoltaic (PV) and wind power (WT) (i.e., new energy) is used as a measure to reduce the daily operation cost of ADN and avoid over-limit.

[0048] The ADN day-ahead dispatching goal is to minimize the daily operating cost. The operating cost of each intelligent agent includes the main grid cost, energy storage operating cost, gas turbine (GT) operating cost, network loss cost, etc. (if the intelligent agent does not have this equipment, its cost is 0). Therefore, the objective function of the ADN distributed day-ahead economic dispatching model established in this step is as follows:

[0049]

[0050] Among them, C ADN is the daily operating cost of the active distribution network, C ADN,n is the daily operating cost of agent n, N agent is the total number of agents in the active distribution network, C grid (t) is the main grid cost at time t, C es,i (t) is the operating cost of energy storage system i at time t, C gt,j (t) is the operating cost of gas generator j at time t, C loss,k (t) is the network loss cost of branch k at time t. Among them, the various costs are as follows:

[0051] If agent n is connected to the main grid by tracking the point of common coupling (PCC), the agent includes the main grid cost, otherwise the term is 0. The main grid cost includes the main grid power purchase cost, carbon emission cost and power penalty cost, which can be expressed as:

[0052]

[0053] Among them, c g (k) is the unit time-of-use electricity price of the main power grid (￥ / kWh), P pcc (k) is the main grid power (in this implementation mode, the main grid injects ADN power as positive), T d To control the cycle, is the carbon emission cost of the main power grid, if the carbon emission per unit time at PCC is E g (k) Exceeds its allowable value The carbon emission cost is:

[0054]

[0055] Among them, c co2 is the unit carbon emission cost (￥ / kg), To track the allowable value of carbon emissions per unit time at the common coupling point, E g (k) is to track the carbon emissions per unit time at the common coupling point, which can be expressed as:

[0056] E g (k) = α co2 ·P pcc (k)·T d (4)

[0057] In the formula, α co2 CO2 emission coefficient per unit energy of the main grid (kg / kWh).

[0058] Furthermore, in order to promote the consumption of distributed renewable energy, a penalty cost for interaction between ADN and the main grid is set. To improve the utilization rate of new energy, namely:

[0059]

[0060] Among them, c penal is the unit penalty cost of interactive energy (￥ / kWh), and abs() is the absolute value function.

[0061] Energy storage operation cost C in agent n es,i(t) Calculated by the levelized cost of energy (LCOE) model, namely:

[0062] C es,i (t) = ∑ i c 1coe ·abs(P es,i (t))·T d (6)

[0063] Among them, c 1coe is the unit energy conversion cost of energy storage (￥ / kWh).

[0064] GT operation cost C within agent n gt,j (t) Mainly includes gas cost and carbon emission cost, namely:

[0065]

[0066] Among them, c gt (t) is the unit operating cost of the gas generator at time t (￥ / kWh), P gt,j (t) is the power of gas generator j at time t, is the carbon emission cost of gas generator j at time t. Similarly, if the carbon emission per unit time of the gas turbine is E gt,j (t) exceeds its allowable value The carbon emission cost is:

[0067]

[0068] Among them, GT carbon emissions can be obtained by fitting the GT power generation through a quadratic polynomial, that is:

[0069]

[0070] Among them, α0(kg / kW 2 h), α1(kg / kWh), α2(kg / h) are fitting coefficients.

[0071] The network loss cost C of all branches in agent n loss,k (t) is:

[0072] C loss,k (t) = ∑ k c g (k) I k (t) 2 ·r k ·T d (10)

[0073] Among them, I k (t) is the branch current at time t, rk is the branch resistance.

[0074] In each control cycle of day-ahead dispatch, the objective function of each intelligent agent should satisfy certain constraints. The constraints of the distributed day-ahead economic dispatch model of the active distribution network in this implementation include the power flow constraints of the distribution network, the transmission power constraints at the common coupling point tracked by the main node of the active distribution network, the power constraints of the energy storage system, the energy constraints of the energy storage system, the output constraints of the gas generator, the reactive power output constraints of new energy, the voltage constraints of each node in the intelligent agent n, and the power constraints of each branch in the intelligent agent n.

[0075] The power flow constraints of the entire distribution network are:

[0076]

[0077] In the formula, Q pcc (t) is the reactive power of the master at time t; P pv , P wt , Q pv , Q wt are the active and reactive power of photovoltaic and wind power of each intelligent agent respectively; P L and Q L is the load of each node; Q es,i (t) is the reactive power of energy storage system i at time t; U i is the voltage at node i, θ ij (t) is the phase difference between nodes i and j at time t; G ij and B ij The present embodiment provides that the output power of each power supply is positive.

[0078] Transmission power constraints at the ADN master node PCC:

[0079]

[0080] in, They are respectively the upper and lower limits of active power and reactive power allowed by the main point.

[0081] Power constraints of energy storage systems. The energy storage in each agent n must meet the output power constraints, namely:

[0082]

[0083] in, They are respectively the upper and lower limits of active output and reactive output of the energy storage system. es,i is the rated capacity of the energy storage converter.

[0084] Energy constraints of the energy storage system. The energy storage system in each agent n must satisfy the state of charge (SOC) constraints, namely:

[0085]

[0086] In addition, the SOC of energy storage must be within a reasonable range at the end of each day, namely:

[0087] β0·SOC es,i (0)≤SOC es,i (24)≤β 24 ·SOC es,i (0) (15)

[0088] Among them, β0, β 24 are constraint coefficients respectively, and their value range is [0.9~1.1].

[0089] The output constraint of GT, the output power constraint of GT is:

[0090]

[0091] in, They are the upper and lower limits of GT active output respectively.

[0092] Reactive power output constraints of new energy sources. Similar to energy storage, the reactive power output of new energy sources of agent n also needs to meet the converter capacity constraints, namely:

[0093]

[0094] Where x is new energy (PV or WT), are the upper and lower limits of reactive power output of new energy, S x,i is the rated capacity of the new energy converter.

[0095] The voltage constraints of each node in agent n are:

[0096]

[0097] in, are the lower and upper limits of the voltage amplitude required for the safety of node k.

[0098] The power constraints of each branch in agent n are:

[0099]

[0100] in, are the lower and upper limits of the transmission power of each branch respectively.

[0101] Step 2: Under the constraints of the distributed day-ahead economic dispatch model of the active distribution network, a multi-agent distributed deep reinforcement learning model is established based on a multi-agent deep deterministic policy gradient algorithm.

[0102] The multi-agent deep deterministic policy gradient algorithm is a multi-agent deep reinforcement learning algorithm that is particularly suitable for dealing with multiple agent collaborative control problems. The MADDPG algorithm can achieve centralized training and distributed operation, thereby protecting the operation privacy of each area of ​​the ADN and reducing the communication burden between agents. It is an effective method to achieve ADN cloud-edge collaboration.

[0103] In a multi-agent environment, states, actions, and rewards directly affect the training and control performance of MADDPG.

[0104] The state space of the multi-agent distributed deep reinforcement learning model established in this step includes the state space s of the active distribution network at time t t and the observation space of each agent n

[0105] The state space s of the active distribution network at time t t The design is as follows:

[0106] s t ={t,c g ,SOC,P g ,Q g ,P rew ,P L ,Q L ,U t ,S t} (20)

[0107] Where t is time, c g is the unit time-of-use electricity price of the main power grid, SOC is the charge state set of the energy storage in the active distribution network at time t, P g is the active power interacting between the common coupling point and the upper level, Q g is the reactive power of the common coupling point interacting with the upstream, P rew is the active power set of renewable energy in the active distribution network; P L and Q L are respectively the active and reactive sets of each node load; U t and S t are the node voltage set and branch power set respectively.

[0108] The observation space of each agent n For itself and part of its surrounding information, it is a subset of the system state space, as follows:

[0109]

[0110] Among them, SOC is the state of charge set of the energy stored in agent n at time t, is the active power of the interaction between the common coupling point in the nth agent and the superior, is the reactive power of the common coupling point in the nth agent interacting with the upper level. If the agent has no common coupling point with the upper level power grid, then the observation space has no and is the set of new energy active power in intelligent agent n; and are respectively the active set and reactive set of each node load in the intelligent agent n; and are the node voltage set and branch power set in agent n respectively.

[0111] The action space of the multi-agent distributed deep reinforcement learning model established in this embodiment Including the controllable devices in its area, it can be expressed as:

[0112]

[0113] in, is the active output of the energy storage system in agent n, is the reactive output of the energy storage system in agent n, is the active output of the gas generator in agent n, is the reactive power output of the new energy within the intelligent agent n.

[0114] The reward function r of the multi-agent distributed deep reinforcement learning model established in this embodiment is n is the operating cost of each agent (since the goal is to minimize the cost, the design reward is negative), that is:

[0115] r n =-C ADN,n (twenty three)

[0116] In MADDPG modeling, the action space and rewards of multiple agents also need to be modified to meet the ADN operation objectives and constraints.

[0117] The constraints of multi-agents can be divided into two categories: one is the constraints of the action space, namely equations (13)~(14), (16)~(17), and the other is the constraints of the state space, namely equations (12), (15), (18)~(19).

[0118] Constraint processing of action space. Constraints are set in the output of the neural network of each agent. Since the output range of each agent neural network is (-1, 1), the state of charge constraint of the energy storage system and the reactive power output constraint of new energy can be mapped to the output range of the agent neural network by means of coefficient linear mapping. For the energy storage system, the SOC constraint of the energy storage system (Equation (14)) can be converted into the following:

[0119]

[0120] If the inequality constraint of the rated capacity of the energy storage converter is not satisfied after mapping, that is, Constraints, design the following nonlinear transformations to constrain:

[0121]

[0122] Similarly, the reactive power output constraints of renewable energy can be obtained.

[0123] Constraint processing of state space. Constraints are performed in the form of penalty functions, that is, the reward function of each agent n is modified and a penalty term is added to constrain it to avoid crossing the limit. The reward function of multiple agents is modified as follows:

[0124] r′=-(C ADN,n +C penal ) (26)

[0125] Among them, C penal is the penalty term, expressed as:

[0126]

[0127] in, is the unit penalty cost of energy exchange with the upper grid, is the unit penalty cost when the state of charge of the energy storage exceeds the allowed range at the end of the day, is the unit penalty cost for each node voltage exceeding the allowable range, The unit penalty cost for each branch transmission power exceeding the allowed range, SOC es,i (24) is the state of charge (SOC) at the 24th hour of the energy storage day, U k is the voltage at node k, S ij is the power of branch i to j.

[0128] f() is the barrier function, which is expressed as:

[0129]

[0130] x min and x max are the lower and upper limits of the variable x, respectively, and εa is the penalty intensity coefficient. The state space increases the penalty intensity when the agent approaches the constraint boundary in a nonlinear way to prevent the state quantity from approaching or exceeding the boundary, thereby achieving the constraint.

[0131] Step 3: Train the multi-agent distributed deep reinforcement learning model on the cloud server. After the training is completed, send the trained multi-agent distributed deep reinforcement learning model to each agent edge server.

[0132] The multi-agent distributed scheduling model and the multi-agent distributed deep reinforcement learning model established in this embodiment are centrally trained on the cloud server. The training process is as follows: Figure 3 As shown in the figure. During the training phase, although each agent only controls its own actions, the evaluation network judges each intelligent action based on global information, so that each agent can perceive the impact of other agents' actions on itself and learn the collaborative control strategy between agents. After the training is completed, the cloud-trained model is sent to each ES.

[0133] Step 4: In each control cycle, each intelligent agent edge server uses the received trained multi-agent distributed deep reinforcement learning model to perform active distribution network dispatching control in the sub-region based on real-time local observation information, that is, to control the active and reactive power of energy storage in each intelligent agent. There is no need for each intelligent agent to exchange information during the execution phase, which not only protects the data privacy of the intelligent agent, but also reduces the communication burden of the ADN.

[0134] In order to verify the effectiveness of the method in this implementation, a case analysis is conducted based on the improved IEEE 33-node distribution network system. All examples are implemented in PyCharm and MATLAB environments, and the proposed method completes model training and operation based on the PyTorch framework.

[0135] Improved IEEE 33-node system such as Figure 1 As shown in Figure 1. Among them, three ESs control three intelligent entities respectively. The distribution of photovoltaic (PV), wind power (WT) and other equipment is as follows: Figure 1 Some parameter settings are shown in Table 1. The pypower library is used for power flow calculation, and other hyper parameters of MADDPG such as learning rate l r , and τ use the default values.

[0136] Table 1 Simulation parameter setting table

[0137]

[0138]

[0139] The episode reward value (episode reward) of the ADN model training using the method proposed in this embodiment is as follows: Figure 4 As shown in the figure. The reward for this round is the sum of the rewards of multiple agents, that is, the reward of the entire ADN. At the beginning of sampling, CS randomly samples the ADN model and stores it in the experience replay pool until 5000 steps (i.e. 209 episodes) before training begins. As the number of training rounds increases, the reward value gradually increases and converges. After 3000 rounds, the reward value has stabilized. In this example, the process takes 127 minutes to complete the training.

[0140] The model trained by the cloud server is used to distribute the ADN control. The results are as follows: Figure 5 As shown. Further, in order to verify the effectiveness of the proposed method, several methods are used to control ADN: (1) MADDPG method proposed in this embodiment. (2) Centralized method (COA), centralized control of ADN is performed through MATLAB and OPTI optimization toolbox to compare the performance of distributed control and centralized control. (3) No control method (No control), that is, the output of energy storage and the reactive output of new energy are always 0. The results under different methods are shown in Figure 6 As shown in Table 2 (the operating cost does not include the penalty cost for the constraints).

[0141] Table 2 Results of different methods

[0142]

[0143]

[0144] Depend on Figure 5 , Figure 6 As can be seen from the results in Table 2. In terms of operating costs, the total daily operating cost of ADN under the method proposed in this embodiment is 65,600 yuan, slightly higher than 16,400 yuan under the COA algorithm, but much lower than the total cost under the uncontrolled method (206,800 yuan). The operating goal of ADN in this embodiment is to improve the operating economy and reduce the energy interaction between ADN and the main power grid. The ADN interaction energy without control is 35.16MWh, while the method proposed in this embodiment is 3.84MWh, and the COA algorithm is even lower. This reduces the total operating cost of ADN, while also reducing the disturbance to the main power grid and the level of carbon emissions. The control performance of the proposed algorithm is close to that of the COA algorithm, which also shows the effectiveness of the proposed method.

[0145] In terms of node voltage variation, the method proposed in this embodiment can ensure that all node voltages of ADN are within the safety domain (0.95pu~1.05pu). The nodes with large ADN node voltage variations are nodes 17, 19, and 32. The voltages of multiple nodes of ADN are out of limit without control. The lowest node voltage is node 17, with a value of 0.83pu, while the node voltage value under the proposed method is 0.97pu.

[0146] In terms of optimizing the computing time, the MADDPG method of this embodiment only takes 0.015s, while COA takes 172.3s. MADDPG greatly reduces the amount of computation and time. Furthermore, the proposed method does not need to communicate with other agents during execution, which not only effectively protects the privacy of each agent, but also is not affected by the communication bandwidth. If the communication bandwidth is affected or even fails, the performance of traditional distributed control will be greatly affected. In summary, the proposed method has obvious advantages in terms of edge computing amount, computing time, communication volume, data privacy protection, etc. The proposed method is more suitable for ADN operation scenarios with small edge computing amount and low communication requirements.

[0147] A second embodiment of the present invention relates to an active distribution network cloud-edge collaborative day-ahead scheduling device, comprising:

[0148] The first building module is used to establish a distributed day-ahead economic dispatch model for the active distribution network with the goal of optimizing the daily operation cost of the active distribution network, taking into account carbon emissions and the interaction cost of the main power grid;

[0149] A second establishment module is used to establish a multi-agent distributed deep reinforcement learning model based on a multi-agent deep deterministic policy gradient algorithm under the constraints of the distributed day-ahead economic dispatch model of the active distribution network;

[0150] A training module is used to train the multi-agent distributed deep reinforcement learning model on the cloud server, and after the training is completed, the trained multi-agent distributed deep reinforcement learning model is sent to each agent edge server;

[0151] The scheduling module is used to perform active distribution network scheduling and control in the sub-region in each control cycle, based on the real-time local observation information and the received trained multi-agent distributed deep reinforcement learning model by each intelligent edge server.

[0152] The objective function of the distributed day-ahead economic dispatch model of the active distribution network established by the first establishment module is: Among them, C ADN is the daily operating cost of the active distribution network, C ADN,n is the daily operating cost of agent n, N agent is the total number of agents in the active distribution network, Cgrid (t) is the main grid cost at time t, C es,i (t) is the operating cost of energy storage system i at time t, C gt,j (t) is the operating cost of gas generator j at time t, C loss,k (t) is the network loss cost of branch k at time t.

[0153] The main grid cost at time t is expressed as: The operating cost of the energy storage system i at time t is expressed as: C es,i (t) = ∑ i c 1coe ·abs(P es,i (t))·T d ; The operating cost of gas generator j at time t is expressed as: The network loss cost of branch k at time t is expressed as: C loss,k (t) = ∑ k c g (k) I k (t) 2 ·r k ·T d ; Among them, c g (k) is the unit time-of-use electricity price of the main power grid, P pcc (k) is the main grid power, T d To control the cycle, is the carbon emission cost of the main power grid, expressed as: c co2 is the unit carbon emission cost, E g (k) is the carbon emission per unit time at the common coupling point, expressed as: E g (k) = α co2 ·P pcc (k)·T d , α co2 is the CO2 emission coefficient per unit energy of the main grid, To track the allowable value of carbon emissions per unit time at the common coupling point, is the penalty cost for interaction between the active distribution network and the main grid, expressed as: c penal is the unit penalty cost of interaction energy, abs() is the absolute value function; c 1coe is the unit energy conversion cost of energy storage, P es,i (t) is the power of energy storage system i at time t; c gt (t) is the unit operating cost of the gas generator at time t, P gt,j (t) is the power of gas generator j at time t, is the carbon emission cost of gas generator j at time t, expressed as: E gt,j (t) is the carbon emission of the gas generator at time t, is the allowable value of carbon emissions per unit time of the gas generator, I k (t) is the branch current at time t, r k is the branch resistance.

[0154] The constraints of the distributed day-ahead economic dispatch model of the active distribution network established by the first establishment module include the power flow constraints of the distribution network, the transmission power constraints of the active distribution network master node tracking the common coupling point, the power constraints of the energy storage system, the energy constraints of the energy storage system, the output constraints of the gas generator, the reactive output constraints of new energy, the voltage constraints of each node within the intelligent body n, and the power constraints of each branch within the intelligent body n.

[0155] The state space of the multi-agent distributed deep reinforcement learning model established by the second establishment module includes the state space s of the active distribution network at time t t and the observation space of each agent n Among them, the state space s of the active distribution network at time t t Expressed as: s t ={t,c g ,SOC,P g ,Q g ,P rew ,P L ,Q L ,U t ,S t}, where t is time, c g is the unit time-of-use electricity price of the main power grid, SOC is the charge state set of the energy storage in the active distribution network at time t, P g is the active power interacting between the common coupling point and the upper level, Q g is the reactive power of the common coupling point interacting with the upstream, P rew is the active power set of renewable energy in the active distribution network; P L and Q L are respectively the active and reactive sets of each node load; U t and S t are the node voltage set and branch power set respectively; the observation space of each agent n It is expressed as: Among them, SOC is the state of charge set of the energy stored in agent n at time t, is the active power of the interaction between the common coupling point in the nth agent and the superior, is the reactive power of the interaction between the common coupling point in the nth agent and the upper level, is the set of new energy active power in intelligent agent n; and are respectively the active set and reactive set of each node load in the intelligent agent n; and are the node voltage set and branch power set in agent n respectively; the action space of the multi-agent distributed deep reinforcement learning model established by the second establishment module It is expressed as: in, is the active output of the energy storage system in agent n, is the reactive output of the energy storage system in agent n, is the active output of the gas generator in agent n, is the reactive power output of the new energy source in agent n; the reward function r of the multi-agent distributed deep reinforcement learning model established by the second establishment module n Expressed as: r n =-C ADN,n , where C ADN,n is the daily operating cost of agent n.

[0156] The constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model established by the second establishment module include action space constraints and state space constraints, wherein the action space constraints include the power constraints of the energy storage system of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system, the output constraints of the gas generator and the reactive output constraints of new energy; the state space constraints include the transmission power constraints at the common coupling point tracked by the main node of the active distribution network of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system at the end of each day, the voltage constraints of each node within the intelligent agent n and the power constraints of each branch within the intelligent agent n.

[0157] The second establishing module comprises:

[0158] The first constraint processing unit is used to map the state of charge constraint of the energy storage system and the reactive output constraint of the new energy to the output range of the intelligent agent neural network respectively by means of coefficient linear mapping; if the inequality constraint conditions of the rated capacity of the energy storage converter and the rated capacity of the new energy converter are not satisfied after mapping, a nonlinear transformation is designed for the active power and reactive power of the energy storage system and the new energy;

[0159] The second constraint processing unit is used to modify the reward function r of each agent n n A penalty term is added to obtain a new reward function r′ for each agent n. The new reward function r′ increases the penalty intensity when the agent approaches the constraint boundary in a nonlinear way for the state space of the multi-agent distributed deep reinforcement learning model.

[0160] The new reward function r' of each agent n is expressed as: r' = -(CADN,n +C penal ), where C penal is the penalty term, expressed as: in, is the unit penalty cost of energy exchange with the upper grid, is the unit penalty cost when the state of charge of the energy storage exceeds the allowed range at the end of the day, is the unit penalty cost for each node voltage exceeding the allowable range, The unit penalty cost for each branch transmission power exceeding the allowed range, SOC es,i (24) is the state of charge at the 24th hour of the energy storage day, U k is the voltage at node k, S ij is the power of branch i to j, f() is the barrier function, which is expressed as: x min and x max are the lower and upper limits of the variable x, respectively, and ε a is the penalty intensity coefficient.

[0161] A third embodiment of the present invention relates to an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned active distribution network cloud-edge collaborative day-ahead scheduling method when executing the computer program.

[0162] A fourth embodiment of the present invention relates to a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned active distribution network cloud-edge collaborative day-ahead scheduling method.

[0163] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0164] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction method, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0167] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for day-ahead scheduling of active distribution network cloud-edge collaboration, characterized in that: The following steps are involved: Taking carbon emissions and the interaction costs of the main power grid into consideration, a distributed day-ahead economic dispatch model for active distribution networks is established with the goal of optimizing the daily operating cost of the active distribution network. Under the constraints of the distributed day-ahead economic dispatch model of the active distribution network, a multi-agent distributed deep reinforcement learning model is established based on a multi-agent deep deterministic policy gradient algorithm; The multi-agent distributed deep reinforcement learning model is trained on the cloud server, and after the training is completed, the trained multi-agent distributed deep reinforcement learning model is sent to each agent edge server; In each control cycle, each intelligent edge server uses the received trained multi-agent distributed deep reinforcement learning model to perform active distribution network dispatching and control in the sub-region based on real-time local observation information.

2. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 1 is characterized in that: The objective function of the distributed day-ahead economic dispatch model of the active distribution network is: Among them, C ADN is the daily operating cost of the active distribution network, C ADN,n is the daily operating cost of agent n, N agent is the total number of agents in the active distribution network, C grid (t) is the main grid cost at time t, C es,i (t) is the operating cost of energy storage system i at time t, C gt,j (t) is the operating cost of gas generator j at time t, C loss,k (t) is the network loss cost of branch k at time t.

3. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 2 is characterized in that: The main grid cost at time t is expressed as: The operating cost of the energy storage system i at time t is expressed as: C es,i (t)=∑ i c 1coe ·abs(P es,i (t))·T d ; The operating cost of gas generator j at time t is expressed as: The network loss cost of branch k at time t is expressed as: C loss,k (t)=∑ k c g (k) I k (t) 2 ·r k ·T d ; Among them, c g (k) is the unit time-of-use electricity price of the main power grid, P pcc (k) is the main grid power, T d To control the cycle, is the carbon emission cost of the main power grid, expressed as: c co2 is the unit carbon emission cost, E g (k) is the carbon emission per unit time at the common coupling point, expressed as: E g (k) = α co2 ·P pcc (k)·T d , α co2 is the CO2 emission coefficient per unit energy of the main grid, To track the allowable value of carbon emissions per unit time at the common coupling point, is the penalty cost for interaction between the active distribution network and the main grid, expressed as: c penal is the unit penalty cost of interaction energy, abs() is the absolute value function; c 1coe is the unit energy conversion cost of energy storage, P es,i (t) is the power of energy storage system i at time t; c gt (t) is the unit operating cost of the gas generator at time t, P gt,j (t) is the power of gas generator j at time t, is the carbon emission cost of gas generator j at time t, expressed as: E gt,j (t) is the carbon emission of the gas generator at time t, is the allowable value of carbon emissions per unit time of the gas generator, I k (t) is the branch current at time t, r k is the branch resistance.

4. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 1 is characterized in that: The constraints of the distributed day-ahead economic dispatch model of the active distribution network include the power flow constraints of the distribution network, the transmission power constraints of the active distribution network master node tracking the common coupling point, the power constraints of the energy storage system, the energy constraints of the energy storage system, the output constraints of the gas generator, the reactive power output constraints of new energy, the voltage constraints of each node in the intelligent agent n, and the power constraints of each branch in the intelligent agent n.

5. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 1 is characterized in that: The state space of the multi-agent distributed deep reinforcement learning model includes the state space s of the active power distribution network at time t t and the observation space of each agent n Among them, the state space s of the active distribution network at time t t Expressed as: s t ={t,c g ,SOC,P g ,Q g ,P rew ,P L ,Q L ,U t ,S t }, where t is time, c g is the unit time-of-use electricity price of the main power grid, SOC is the charge state set of the energy storage in the active distribution network at time t, P g is the active power interacting between the common coupling point and the upper level, Q g is the reactive power interacting between the common coupling point and the upstream, P rew is the active power set of renewable energy in the active distribution network; P L and Q L are respectively the active and reactive sets of each node load; U t and S t are the node voltage set and branch power set respectively; the observation space of each agent n It is expressed as: Among them, SOC is the state of charge set of the energy stored in agent n at time t, is the active power of the interaction between the common coupling point in the nth agent and the superior, is the reactive power of the interaction between the common coupling point in the nth agent and the upper level, is the set of new energy active power in intelligent agent n; and are respectively the active set and reactive set of each node load in the intelligent agent n; and are the node voltage set and branch power set in agent n respectively; the action space of the multi-agent distributed deep reinforcement learning model It is expressed as: in, is the active output of the energy storage system in agent n, is the reactive output of the energy storage system in agent n, is the active output of the gas generator in agent n, is the reactive power output of the new energy source in agent n; the reward function r of the multi-agent distributed deep reinforcement learning model n Expressed as: r n =-C ADN,n , where C ADN,n is the daily operating cost of agent n.

6. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 5 is characterized in that: The constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model include action space constraints and state space constraints, wherein the action space constraints include the power constraints of the energy storage system of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system, the output constraints of the gas generator and the reactive output constraints of new energy; the state space constraints include the transmission power constraints at the common coupling point tracked by the main node of the active distribution network of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system at the end of each day, the voltage constraints of each node within the agent n and the power constraints of each branch within the agent n.

7. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 6 is characterized in that: Also includes: The steps of processing the constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model are specifically as follows: mapping the state of charge constraints of the energy storage system and the reactive output constraints of the new energy to the output range of the agent neural network respectively by means of coefficient linear mapping; if the inequality constraints of the rated capacity of the energy storage converter and the rated capacity of the new energy converter are not satisfied after mapping, then designing nonlinear transformations for the active power and reactive power of the energy storage system and the new energy; Modify the reward function r of each agent n n A penalty term is added to obtain a new reward function r′ for each agent n. The new reward function r′ increases the penalty intensity when the agent approaches the constraint boundary in a nonlinear way for the state space of the multi-agent distributed deep reinforcement learning model.

8. The active distribution network cloud-edge collaborative day-ahead scheduling method according to claim 7 is characterized in that: The new reward function r' of each agent n is expressed as: r' = -(C ADN,n +C penal ), where C penal is the penalty term, expressed as: middle, is the unit penalty cost of energy exchange with the upper grid, is the unit penalty cost when the state of charge of the energy storage exceeds the allowed range at the end of the day, is the unit penalty cost for each node voltage exceeding the allowable range, The unit penalty cost for each branch transmission power exceeding the allowed range, SOC es,i (24) is the state of charge at the 24th hour of the energy storage day, U k is the voltage at node k, S ij is the power of branch i to j, f( ) is the barrier function, which is expressed as: x min and x max are the lower and upper limits of the variable x, respectively, and ε a is the penalty intensity coefficient.

9. An active distribution network cloud-edge collaborative day-ahead dispatching device, characterized in that: include: The first building module is used to establish a distributed day-ahead economic dispatch model for the active distribution network with the goal of optimizing the daily operation cost of the active distribution network, taking into account carbon emissions and the interaction cost of the main power grid; A second establishment module is used to establish a multi-agent distributed deep reinforcement learning model based on a multi-agent deep deterministic policy gradient algorithm under the constraints of the distributed day-ahead economic dispatch model of the active distribution network; A training module is used to train the multi-agent distributed deep reinforcement learning model on the cloud server, and after the training is completed, the trained multi-agent distributed deep reinforcement learning model is sent to each agent edge server; The scheduling module is used to perform active distribution network scheduling and control in the sub-region in each control cycle, based on the real-time local observation information and the received trained multi-agent distributed deep reinforcement learning model by each intelligent edge server.

10. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 9, characterized in that: The objective function of the distributed day-ahead economic dispatch model of the active distribution network established by the first establishment module is: Among them, C ADN is the daily operating cost of the active distribution network, C ADN,n is the daily operating cost of agent n, N agent is the total number of agents in the active distribution network, C grid (t) is the main grid cost at time t, C es,i (t) is the operating cost of energy storage system i at time t, C gt,j (t) is the operating cost of gas generator j at time t, C loss,k (t) is the network loss cost of branch k at time t.

11. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 10, characterized in that: The main grid cost at time t is expressed as: The operating cost of the energy storage system i at time t is expressed as: C es,i (t)=∑ i c 1coe ·abs(P es,i (t))·T d ; The operating cost of gas generator j at time t is expressed as: The network loss cost of branch k at time t is expressed as: C loss,k (t)=∑ k c g (k) I k (t) 2 ·r k ·T d ; Among them, c g (k) is the unit time-of-use electricity price of the main power grid, P pcc (k) is the main grid power, T d To control the cycle, is the carbon emission cost of the main power grid, expressed as: c co2 is the unit carbon emission cost, E g (k) is the carbon emission per unit time at the common coupling point, expressed as: E g (k) = α co2 ·P pcc (k)·T d , α co2 is the CO2 emission coefficient per unit energy of the main grid, To track the allowable value of carbon emissions per unit time at the common coupling point, is the penalty cost for interaction between the active distribution network and the main grid, expressed as: c penal is the unit penalty cost of interaction energy, abs() is the absolute value function; c 1coe is the unit energy conversion cost of energy storage, P es,i (t) is the power of energy storage system i at time t; c gt (t) is the unit operating cost of the gas generator at time t, P gt,j (t) is the power of gas generator j at time t, is the carbon emission cost of gas generator j at time t, expressed as: E gt,j (t) is the carbon emission of the gas generator at time t, is the allowable value of carbon emissions per unit time of the gas generator, I k (t) is the branch current at time t, r k is the branch resistance.

12. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 9, characterized in that: The constraints of the distributed day-ahead economic dispatch model of the active distribution network established by the first establishment module include the power flow constraints of the distribution network, the transmission power constraints of the active distribution network master node tracking the common coupling point, the power constraints of the energy storage system, the energy constraints of the energy storage system, the output constraints of the gas generator, the reactive output constraints of new energy, the voltage constraints of each node within the intelligent body n, and the power constraints of each branch within the intelligent body n.

13. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 9, characterized in that: The state space of the multi-agent distributed deep reinforcement learning model established by the second establishment module includes the state space s of the active distribution network at time t t and the observation space of each agent n Among them, the state space s of the active distribution network at time t t Expressed as: s t ={t,c g ,SOC,P g ,Q g ,P rew ,P L ,Q L ,U t ,S t }, where t is time, c g is the unit time-of-use electricity price of the main power grid, SOC is the charge state set of the energy storage in the active distribution network at time t, P g is the active power interacting between the common coupling point and the upper level, Q g is the reactive power of the common coupling point interacting with the upstream, P rew is the active power set of renewable energy in the active distribution network; P L and Q L are respectively the active and reactive sets of each node load; U t and S t are the node voltage set and branch power set respectively; the observation space of each agent n It is expressed as: Among them, SOC is the state of charge set of the energy stored in agent n at time t, is the active power of the interaction between the common coupling point in the nth agent and the superior, is the reactive power of the interaction between the common coupling point in the nth agent and the upper level, is the set of new energy active power in intelligent agent n; and are respectively the active set and reactive set of each node load in the intelligent agent n; and are the node voltage set and branch power set in agent n respectively; the action space of the multi-agent distributed deep reinforcement learning model established by the second establishment module It is expressed as: in, is the active output of the energy storage system in agent n, is the reactive output of the energy storage system in agent n, is the active output of the gas generator in agent n, is the reactive power output of the new energy source in agent n; the reward function r of the multi-agent distributed deep reinforcement learning model established by the second establishment module n Expressed as: r n =-C ADN,n , where C ADN,n is the daily operating cost of agent n.

14. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 13, characterized in that: The constraints of the multi-agent in the multi-agent distributed deep reinforcement learning model established by the second establishment module include action space constraints and state space constraints, wherein the action space constraints include the power constraints of the energy storage system of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system, the output constraints of the gas generator and the reactive output constraints of new energy; the state space constraints include the transmission power constraints at the common coupling point tracked by the main node of the active distribution network of the distributed day-ahead economic dispatch model of the active distribution network, the state of charge constraints of the energy storage system at the end of each day, the voltage constraints of each node within the intelligent agent n and the power constraints of each branch within the intelligent agent n.

15. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 14, characterized in that: The second establishing module comprises: The first constraint processing unit is used to map the state of charge constraint of the energy storage system and the reactive output constraint of the new energy to the output range of the intelligent agent neural network respectively by means of coefficient linear mapping; if the inequality constraint conditions of the rated capacity of the energy storage converter and the rated capacity of the new energy converter are not satisfied after mapping, a nonlinear transformation is designed for the active power and reactive power of the energy storage system and the new energy; The second constraint processing unit is used to modify the reward function r of each agent n n A penalty term is added to obtain a new reward function r′ for each agent n. The new reward function r′ increases the penalty intensity when the agent approaches the constraint boundary in a nonlinear way for the state space of the multi-agent distributed deep reinforcement learning model.

16. The active distribution network cloud-edge collaborative day-ahead dispatching device according to claim 15, characterized in that: The new reward function r' of each agent n is expressed as: r' = -(C ADN,n +C penal ), where C penal is the penalty term, expressed as: in, is the unit penalty cost of energy exchange with the upper grid, is the unit penalty cost when the state of charge of the energy storage exceeds the allowed range at the end of the day, is the unit penalty cost for each node voltage exceeding the allowable range, The unit penalty cost for each branch transmission power exceeding the allowed range, SOC es,i (24) is the state of charge at the 24th hour of the energy storage day, U k is the voltage at node k, S ij is the power of branch i to j, f() is the barrier function, which is expressed as: x min and x max are the lower and upper limits of the variable x, respectively, and ε a is the penalty intensity coefficient.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the active distribution network cloud-edge collaborative day-ahead scheduling method as described in any one of claims 1-8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the active distribution network cloud-edge collaborative day-ahead scheduling method as described in any one of claims 1 to 8 are implemented.

Images