Virtual power plant dynamic regulation and control method based on deep reinforcement learning

Abstract

The invention provides a virtual power plant dynamic regulation and control method based on deep reinforcement learning. The method comprises the following steps: constructing a virtual power plant dynamic model; the virtual power plant dynamic model comprises a network dynamic model and an equipment dynamic model; constructing a wind power output uncertainty model according to the wind power historical data; constructing a virtual power plant scheduling model based on the virtual power plant dynamic model and the wind power output uncertainty model; the virtual power plant scheduling model is divided into two stages of day-ahead optimization and intra-day adjustment; and optimizing the virtual power plant scheduling model based on a deep reinforcement learning optimization strategy so as to improve the economical efficiency and the real-time performance of scheduling by learning the dynamic characteristics and the scheduling strategy of the equipment. According to the method, the virtual power plant dynamic model containing the wind turbine generator, the combined heat and power generation unit, the heat pump and the electric energy storage unit is established, the accuracy of the model is improved, cooperative regulation and control of all devices in the virtual power plant are achieved, and real-time requirements and economical efficiency are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual power plant control, and in particular to a virtual power plant dynamic control method based on deep reinforcement learning. Background Art

[0002] At present, the green and low-carbon transformation of the energy structure has become an inevitable trend. Renewable energy such as wind power and solar energy have occupied an increasingly important position in the energy consumption structure due to their clean and efficient characteristics. However, the randomness and volatility of these energy sources have brought significant uncertainty to the power system and posed a huge challenge to the security and supply and demand balance of the power grid. As a new energy management model, Virtual Power Plant (VPP) can effectively alleviate the impact of renewable energy uncertainty by aggregating a large number of distributed energy and demand-side resources to achieve flexible resource regulation, and provide a reliable path for the safe and stable operation of the power grid.

[0003] However, as a renewable energy source, wind power presents great uncertainty in its power generation process, which makes the optimal scheduling of VPPs face challenges. Therefore, in order to achieve efficient scheduling of VPPs, it is urgent to develop intelligent algorithms that can handle the uncertainty of wind power output. Traditional solution algorithms have limitations when dealing with complex and uncertain environments: on the one hand, they are difficult to effectively search and optimize in multidimensional environments; on the other hand, traditional algorithms have limited ability to adapt to dynamic environments and deal with uncertainties, lack the ability to learn gradually and adjust in real time, and are prone to fall into local optimal solutions. At the same time, in some observable environments, they are difficult to infer implicit states from historical data, and their effective decision-making capabilities are insufficient. In addition, in multi-objective optimization, traditional algorithms have poor generalization capabilities, are significantly insufficient in flexibility to cope with environmental changes, and their computational efficiency often cannot meet the needs of real-time applications. These defects make it difficult for them to meet the high standards of modern intelligent decision-making systems. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a dynamic control method of a virtual power plant based on deep reinforcement learning, which realizes the coordinated control of various equipment in the virtual power plant and ensures real-time demand and economy.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A virtual power plant dynamic control method based on deep reinforcement learning, comprising:

[0007] Constructing a virtual power plant dynamic model; the virtual power plant dynamic model includes a network dynamic model and an equipment dynamic model;

[0008] Construct wind power output uncertainty model based on historical wind power data;

[0009] A virtual power plant dispatching model is constructed based on the virtual power plant dynamic model and the wind power output uncertainty model; the virtual power plant dispatching model is divided into two stages: day-ahead optimization and intraday adjustment;

[0010] The virtual power plant scheduling model is optimized based on deep reinforcement learning optimization strategy to improve the economy and real-time performance of scheduling by learning the dynamic characteristics and scheduling strategies of the equipment.

[0011] Preferably, the network dynamic model includes a power grid dynamic model and a heating network dynamic model; and the method for establishing the power grid dynamic model includes:

[0012] The distribution of voltage u(x, ω) and current i(x, ω) is described by a group of partial differential equations; the formula of the group of partial differential equations is: Where Z(ω) and Y(ω) represent the impedance and admittance of the line respectively; the complex forms of impedance Z(ω) and admittance Y(ω) are defined as: R(ω) and L(ω) represent the resistance and inductance of the line, respectively, indicating the resistance encountered when the current flows through the conductor and the self-inductance effect caused by the change of current; G(ω) and C(ω) represent the conductance and capacitance, respectively, ω is the frequency, x is the position coordinate of the transmission line; k is the imaginary unit;

[0013] The expression of the dynamic model of the heating network is: Among them, M i represents the mass of mass point i; h i represents the specific enthalpy of mass point i; dh i / dt represents the rate of change of specific enthalpy with time, which represents the rate of change of energy of mass point i; D ij Indicates the degree of coupling between mass points i and j, which is related to the distance between the two points; G ij represents the flow conductivity of the fluid from mass point j to i; h j The specific enthalpy of other mass points i coupled with mass point i; Q i represents the heat received by mass point i; N is the total number of nodes.

[0014] Preferably, the equipment dynamic model includes a wind turbine dynamic model, a cogeneration unit dynamic model, an electric energy storage dynamic model and a heat pump dynamic model;

[0015] The steps of establishing the dynamic model of the wind turbine generator set are:

[0016] Construct an expression for the rotor torque; the rotor torque T ω' The expression is: Among them, P BLADE is the power captured by the blade; C p(λ,β) is the power coefficient, λ is the tip speed ratio, β is the pitch angle, g is the acceleration due to gravity, ρ is the air density, A is the rotor swept area, v is the wind speed, ω' is the rotor angular velocity, and H is the rotor moment of inertia;

[0017] Construct an expression of electromagnetic torque; the electromagnetic torque T e The expression is: T e =K e i f θ e ; Among them, K e is the electromagnetic torque constant; i f is the excitation current; θ e is the electromagnetic angle;

[0018] A differential equation of the rotor torque is constructed; the differential equation of the rotor torque is expressed as: in, is the rate of change of the rotor angular velocity; T ω is the torque generated by the wind; T e is the electromagnetic torque; represents the feedback effect of the motor control system on the rotor speed.

[0019] Construct an expression for the generator power; the generator power P gen The expression is: Where L is the inductance of the generator; R is the resistance; i is the current; k e is the electromotive force constant;

[0020] The steps of establishing the dynamic model of the cogeneration unit are:

[0021] Construct a boiler mass flow rate equation; the boiler mass flow rate equation is: M =u B (t-τ); where r M is the adjusted boiler mass flow rate; u B is the fuel flow rate; τ is the time delay of the fuel reaching the burner;

[0022] Construct a dynamic equation of the grinding mill; the expression of the dynamic equation of the grinding mill is: Among them, r B is the boiler pressure response rate; r M is the boiler mass flow rate; T f is the coal mill time constant;

[0023] Construct a boiler energy balance equation; the boiler energy balance equation is: Among them, P a is the throttle valve pressure; C b is the boiler capacity; K1 and K3 are proportional constants; u Tis the opening of the turbine input regulating valve;

[0024] Construct a throttle valve pressure equation; the throttle valve pressure equation is: P a =P b -K 2 (K 1 r B ) 1.5 ; Among them, P b Indicates the drum pressure; K1 and K2 are both proportional constants.

[0025] Construct a turbine energy balance equation; the turbine energy balance equation is: Where: T t is the turbine time constant; P MT is the power generation capacity of the thermal power unit; P e is the heat rejection pressure; K 3 , K 4 and K 5 are all related constants; u H It is the opening degree of the heat regulating butterfly valve;

[0026] Construct a formula for the heating power of a cogeneration unit; the formula for the heating power of the cogeneration unit is: Q MT =σP MT ; where σ is the thermoelectric conversion coefficient; Q MT It is the heating power of the cogeneration unit.

[0027] The expression of the electric energy storage dynamic model is: Among them, Q cap is the energy storage capacity; z is the storage state of the energy storage; η e is the round-trip energy efficiency factor of electric energy storage; p + and p - Represent the charging power and discharging power of the battery respectively.

[0028] The steps of establishing the heat pump dynamic model include:

[0029] Construct a relationship equation between compressor power and outlet temperature; the relationship equation between compressor power and outlet temperature is: Among them, T out,co is the compressor inlet temperature; T in,co is the compressor outlet temperature; η co is the isentropic efficiency; β co is the compression ratio; κ is the air isentropic index; is the mass flow rate; c p,co is the specific heat at constant pressure;

[0030] A dynamic equation of the compressor outlet pressure is constructed; the dynamic equation of the compressor outlet pressure is: Where: P out,co is the compressor outlet pressure; R g is the gas constant; V is the volume of the corresponding chamber of the compressor; and are the mass flow rates entering and leaving the cavity, respectively;

[0031] Build the output power W of the expander ex and outlet temperature T out,ex Formula: The output power W of the expander ex and outlet temperature T out,ex The formula is: Among them, T in,ex is the expander inlet temperature; T out,ex is the expander outlet temperature; η ex is the isentropic efficiency; β ex is the expansion ratio; is the mass flow rate; c p,ex is the specific heat at constant pressure;

[0032] The kinetic equation of the expander outlet pressure is constructed; the kinetic equation of the expander outlet pressure is: Among them, P out,ex is the expander outlet pressure, U is the volume of the corresponding chamber of the expander, and are the mass flow rates entering and leaving the cavity, respectively;

[0033] Construct the validity equation of the outlet temperature of the cold and hot fluids of the heat exchanger; the validity equation of the outlet temperature of the cold and hot fluids of the heat exchanger is: Among them, T in,h and T in,c are the inlet temperatures of the hot and cold fluids, respectively, T out,h and T out,c are the outlet temperatures, respectively, and λ is the effectiveness coefficient of the heat exchanger;

[0034] Construct the condenser thermal fluid thermal power Q c Formula; the condenser thermal fluid thermal power Q c The formula is: Q c t=c p m w (T out,h -T in,h ), where c p is the constant pressure heat capacity of water; m w is the mass flow rate of the thermal fluid; t is the time;

[0035] Construct a heat pump power conversion formula; the heat pump power conversion formula is: P HP =λ cop Qc ; Among them, λ cop is the heating coefficient of the heat pump, P HP The heat pump consumes electrical power.

[0036] Preferably, a wind power output uncertainty model is constructed based on historical wind power data, including:

[0037] Unsupervised clustering of wind power historical data using Dirichlet process mixture model;

[0038] Dynamically adjust the number of clusters and divide multiple clusters according to data complexity;

[0039] The posterior distribution is approximately solved by the variational inference algorithm, and the lower bound of evidence is optimized to approach the true distribution and obtain the clustering result;

[0040] Using the clustering results, the wind power data is divided by season to generate fuzzy sets with different labels;

[0041] Calculate the probability of each cluster and obtain the probability distribution of each cluster through maximum likelihood estimation;

[0042] An uncertainty fuzzy set formula is constructed, and the uncertainty of wind power output is described in combination with auxiliary coefficients and proportional factors to obtain the wind power output uncertainty model.

[0043] Preferably, the uncertainty fuzzy set formula includes:

[0044] Π m =Π m,1 ∪Π m,2 ∪Π m,k …∪Π m,K

[0045]

[0046]

[0047] Among them, κ m,i is an auxiliary coefficient; Π s is the mth type of data-driven uncertainty fuzzy set, Π m,k is the uncertainty fuzzy set of the kth cluster in the mth class, k is the number of clusters in the mth class; Λ m,i is the proportionality factor; d is the correlation coefficient between wind power output data, μ m,i ,ω m,i , ψ m,t is the inference result of the i-th component obtained from data class m, φ m,i It is uncertainty.

[0048] Preferably, in the stage of day-ahead optimization, the method for constructing the virtual power plant scheduling model includes:

[0049] The day-ahead optimization objective is to minimize the operating cost of the virtual power plant; the expression for minimizing the operating cost of the virtual power plant is: in, is the electricity purchase price at time t; is the amount of electricity purchased by the virtual power plant from the upper power grid at time t; is the electricity selling price at time t; is the amount of electricity sold by the virtual power plant to the upper power grid at time t; is the operating cost of the CHP unit at the ith node at time t; The electric energy storage operation cost of the i-th node at time t; The operating cost of the heat pump unit at the i-th node at time t, N is the total number of nodes; T is the total time;

[0050] Calculate the cost of each unit at the i-th node at the t-th time; the calculation formula of the unit cost includes:

[0051] C MT =λ 1 P MT +λ 2 Q MT

[0052] C ES =λ 3 P ES

[0053] C HP =λ 4 P HP

[0054] Among them, λ 1 is the cost coefficient of electric power conversion of the cogeneration unit; 2 is the thermal power conversion cost coefficient of the cogeneration unit; 3 is the cost coefficient of electric power conversion from electric energy storage; 4 is the cost coefficient of electric power consumed by the heat pump unit; P HP The electrical power consumed by the heat pump, P MT P is the power generation of the thermal power unit; ES is the output power of the electric energy storage; Q MT Produces heat power for thermal power units.

[0055] Construct constraints to form the virtual power plant dispatch model in the stage of day-ahead optimization; the virtual power plant dispatch model includes electric power balance constraints, thermal power constraints, virtual power plant power purchase and sales constraints, cogeneration unit output power constraints, cogeneration unit ramp power limit constraints, heat pump constraints and electric energy storage capacity constraints.

[0056] Preferably, the expression of the electric power balance constraint is: in, is the wind power output power of the ith node at the tth moment; is the electrical load power demand of the ith node at time t; is the electrical output power of the CHP unit at the ith node at time t; and are the electrical output and input power of the electrical energy storage; is the electrical power consumed by the heat pump at the i-th node at time t;

[0057] The expression of the thermal power balance constraint is: in, is the heat load power demand of the ith node at time t; is the thermal output power of the CHP unit at the ith node at time t; is the heat pump output power of the ith node at time t

[0058] The expression of the power purchase and sale constraint of the virtual power plant is: in, is the maximum power purchase and sale power of the virtual power plant, θ t It is a Boolean variable, 1 indicates the electricity selling state, and 0 indicates the electricity purchasing state;

[0059] The expression of the output power constraint of the cogeneration unit is: in, is the maximum output power of the CHP unit; is the electrical output power of the cogeneration unit at the i-th node at the t-th moment.

[0060] The expression of the ramp power limit constraint of the cogeneration unit is: in, and They are the rising and falling ramp power limits of the cogeneration unit respectively; is the electrical output power of the cogeneration unit at the i-th node at the t-1th moment.

[0061] The expression of the heat pump constraint is: in, and are the maximum and minimum heat load output powers of the heat pump respectively; is the heat production power of the heat pump unit at the tth moment.

[0062] The expression of the electric energy storage capacity constraint is: min ≤z(t)≤z max ; Among them, z minand z max are the minimum and maximum energy storage levels of the electric energy storage respectively; z(t) is the energy storage level of the electric energy storage at the tth moment.

[0063] Preferably, during the intraday adjustment stage, the virtual power plant dispatching model includes a two-layer rolling optimization model of a fast control sublayer and a slow control sublayer.

[0064] Preferably, the deep reinforcement learning optimization strategy is a communication-independent dual-delay deep deterministic policy gradient algorithm; the goal of the communication-independent dual-delay deep deterministic policy gradient algorithm is to optimize the strategy of each virtual power plant device to minimize operating costs and meet grid stability requirements.

[0065] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0066] The present invention provides a method for dynamic control of virtual power plants based on deep reinforcement learning, including: constructing a virtual power plant dynamic model; the virtual power plant dynamic model includes a network dynamic model and an equipment dynamic model; constructing a wind power output uncertainty model based on wind power historical data; constructing a virtual power plant scheduling model based on the virtual power plant dynamic model and the wind power output uncertainty model; the virtual power plant scheduling model is divided into two stages: day-ahead optimization and intraday adjustment; optimizing the virtual power plant scheduling model based on a deep reinforcement learning optimization strategy, so as to improve the economy and real-time performance of scheduling by learning the dynamic characteristics and scheduling strategies of the equipment. The present invention establishes a dynamic model of a virtual power plant containing wind turbines, cogeneration units, heat pumps, and electric energy storage, improves the accuracy of the model, and realizes the coordinated control of various equipment in the virtual power plant to ensure real-time demand and economy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0068] Figure 1 A flow chart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] The purpose of the present invention is to provide a dynamic control method for a virtual power plant based on deep reinforcement learning, establish a dynamic model of a virtual power plant including wind turbines, cogeneration units, heat pumps, and electric energy storage, improve the accuracy of the model, and realize the coordinated control of various equipment in the virtual power plant to ensure real-time demand and economy.

[0071] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0072] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a virtual power plant dynamic control method based on deep reinforcement learning, comprising:

[0073] Step 100: construct a virtual power plant dynamic model; the virtual power plant dynamic model includes a network dynamic model and an equipment dynamic model;

[0074] Step 200: constructing a wind power output uncertainty model based on historical wind power data;

[0075] Step 300: constructing a virtual power plant dispatching model based on the virtual power plant dynamic model and the wind power output uncertainty model; the virtual power plant dispatching model is divided into two stages: day-ahead optimization and intraday adjustment;

[0076] Step 400: Optimize the virtual power plant scheduling model based on a deep reinforcement learning optimization strategy to improve the economy and real-time performance of scheduling by learning the dynamic characteristics and scheduling strategies of the equipment.

[0077] This embodiment first constructs a virtual power plant dynamic model. The virtual power plant dynamic model described in the present invention is divided into a network dynamic model and an equipment dynamic model. The network dynamic model includes a power grid and a heat network dynamic model. The specific steps are as follows:

[0078] Establishing a dynamic model of the power grid:

[0079] The distribution of voltage u(x,ω) and current i(x,ω) can be described by the following set of partial differential equations:

[0080]

[0081]

[0082] Where: Z(ω) and Y(ω) represent the impedance and admittance of the line, respectively, which change with the frequency ω. The complex forms of impedance Z(ω) and admittance Y(ω) are defined as:

[0083] Z(ω)=R(ω)+kL(ω)

[0084] Y(ω)=G(ω)+kC(ω)

[0085] In the formula: R(ω) and L(ω) represent the resistance and inductance of the line, which represent the resistance encountered when the current flows through the conductor and the self-inductance effect caused by the change of current. G(ω) and C(ω) represent conductance and capacitance, respectively. Conductance represents the amount of current that can pass through a unit length of line under unit voltage, while capacitance describes the ability of the line to store charge.

[0086] The dynamic model of the heat network is established as follows:

[0087]

[0088] Where: M i represents the mass of mass point i; h i represents the specific enthalpy of mass point i, that is, the thermal enthalpy per unit mass; dh i / dt represents the rate of change of specific enthalpy with time, which represents the rate of change of energy of mass point i; D ij Indicates the degree of coupling between mass points i and j, which is related to the distance between the two points; G ij represents the flow conductivity of the fluid from mass point j to i; h j The specific enthalpy of other mass points i coupled with mass point i; Q i represents the heat received by mass point i; N is the total number of nodes.

[0089] The equipment dynamic model includes wind turbines, cogeneration units, electric energy storage and heat pump dynamic models.

[0090] Establish a dynamic model of wind turbines:

[0091] Rotor torque T w for:

[0092]

[0093] Where: P BLADE is the power captured by the blade; C p(λ,β) is the power coefficient, which reflects the influence of blade design and operating conditions (such as tip speed ratio λ and pitch angle β) on energy capture efficiency; g is the acceleration due to gravity; ρ is the air density; A is the rotor swept area; v is the wind speed; ω is the angular velocity of the rotor; and H is the moment of inertia of the rotor.

[0094] Electromagnetic torque equation T e for:

[0095] T e =K e i f θ e

[0096] Where: K e is the electromagnetic torque constant; i f is the excitation current; θ e is the electromagnetic angle.

[0097] The rotor torque differential equation is:

[0098]

[0099] Where: is the rate of change of the rotor angular velocity; T ω is the torque generated by the wind; T e is the electromagnetic torque; represents the feedback effect of the motor control system on the rotor speed.

[0100] Generator power P gen for:

[0101]

[0102] Where: L is the inductance of the generator; R is the resistance; i is the current; k e is the electromotive force constant.

[0103] The process of establishing the dynamic model of the cogeneration unit is as follows:

[0104] Boiler mass flow rate equation:

[0105] r M =u B (t-τ)

[0106] Where: r M is the adjusted boiler mass flow rate; u B is the fuel flow rate; τ is the time delay of the fuel reaching the burner.

[0107] Dynamic equation of grinding mill:

[0108]

[0109] Where: r Bis the boiler pressure response rate; r M is the boiler mass flow rate; T f is the coal mill time constant.

[0110] Boiler energy balance equation:

[0111]

[0112] Where: P a is the throttle valve pressure; C b is the boiler capacity; K1 and K3 are proportional constants; u T is the opening of the turbine input regulating valve.

[0113] Throttle valve pressure equation:

[0114] P a =P b -K 2 (K 1 r B ) 1.5

[0115] Where: P b Indicates the drum pressure; K1, K2 are proportional constants.

[0116] Turbine energy balance equation:

[0117]

[0118] Where: T t is the turbine time constant; P MT is the power generation capacity of the thermal power unit; P e is the heat removal pressure; K3, K4, K5 are related constants; u H It is the opening of the heat regulating butterfly valve.

[0119] Heating power of cogeneration unit:

[0120] Q MT =σP MT

[0121] Where: σ is the thermoelectric conversion coefficient; Q E It is the heating power of the cogeneration unit.

[0122] Electric energy storage dynamic model:

[0123]

[0124] Where: Q cap is the energy storage capacity; z is the storage state of the energy storage; η e It is the round-trip energy efficiency factor of electric energy storage, which describes the proportion of energy loss during the charging and discharging process of the battery, usually less than 1; p+ and p - Represent the charging power and discharging power of the battery respectively.

[0125] Heat pump dynamic model:

[0126] The relationship equation between compressor power and outlet temperature is:

[0127]

[0128]

[0129] Where: T out,co is the compressor inlet temperature; T in,co is the compressor outlet temperature; η co is the isentropic efficiency; β co is the compression ratio; κ is the air isentropic index; is the mass flow rate; c p,co is the specific heat at constant pressure.

[0130] The dynamic equation for the compressor outlet pressure is:

[0131]

[0132] Where: P out,co is the compressor outlet pressure; R g is the gas constant; V is the volume of the corresponding chamber of the compressor; and are the mass flow rates entering and leaving the cavity, respectively.

[0133] Output power of expander W ex and outlet temperature T out,ex formula:

[0134]

[0135]

[0136] Where: T in,ex is the expander inlet temperature; T out,ex is the expander outlet temperature; η ex is the isentropic efficiency; β ex is the expansion ratio; Mass flow rate; c p,ex is the specific heat at constant pressure.

[0137] The dynamic equation of the expander outlet pressure is:

[0138]

[0139] Where: P out,exis the expander outlet pressure, U is the volume of the corresponding chamber of the expander, and are the mass flow rates entering and leaving the cavity, respectively.

[0140] Validity equation for the outlet temperature of the hot and cold fluids in the heat exchanger:

[0141] T out,h =λT in,c +(1-λ)T in,h

[0142] T out,c =λT in,h +(1-λ)T in,c

[0143] Where: T in,h and T in,c are the inlet temperatures of the hot and cold fluids, respectively, T out,h and T out,c are the outlet temperatures, and λ is the effectiveness coefficient of the heat exchanger.

[0144] Condenser thermal fluid thermal power Q c :

[0145] Q c t=c p m w (T out,h -T in,h )

[0146] Where: c p is the constant pressure heat capacity of water; m w is the mass flow rate of the thermal fluid; t is the time.

[0147] Heat pump power conversion formula:

[0148] P HP =λ cop Q c

[0149] Where: cop is the heating coefficient of the heat pump, P HP The heat pump consumes electrical power.

[0150] Optionally, this embodiment performs wind power output uncertainty modeling, which is specifically described as follows:

[0151] As a parametric Bayesian clustering method, the Dirichlet process mixture model is often used to cluster wind power output uncertainty data. The Dirichlet mixture model can automatically adjust the number of clusters according to the data, which is suitable for data clustering needs in uncertain environments.

[0152] Given the Dirichlet process mixture model definition:

[0153] (D(H 1 ),...,D(H r ))~Dir(aD 0 (H 1 ),...,aD 0 (H r ))

[0154] Where: D is a generalization of the Dirichlet distribution; for any finite partition H 1 ,...,H r , if (D(H 1 ),...,D(H r )) is subject to a parameter aD 0 (H 1 ), then D is a Dirichlet distribution with concentration parameter a and underlying distribution D 0 Dirichlet process, namely D~DP(a,D 0 ). 0 It represents the mean of the distribution, and a determines the degree of central deviation of D from the distribution.

[0155] Posterior distribution:

[0156] (D(H 1 ),...,D(H r ))|q 1 ,...,q n ~Dir(aD 0 (H 1 )+n 1 ,...,aD 0 (H r )+n r )

[0157] If a set of samples h is observed 1 ,...,h n , where n i Indicates that the sample falls on Q i Given these observations, the posterior distribution of D becomes a Dirichlet process with a deviation degree of a+n, which transforms D 0 The weighted average of the empirical distribution is used as its basic distribution. This shows that the Dirichlet process has good updability as a prior distribution. As the data increases, the posterior distribution will become closer and closer to the actual distribution of the data.

[0158] Dirichlet process mixture model (DPMM):

[0159]

[0160] Where: π k are weights generated from the β distribution, θ k is from the base distribution D 0 The process of folding sticks shows how to construct a mixture model with an infinite number of components that can dynamically adjust the number of clusters according to the complexity of the data. This property makes the dynamic probabilistic mixture model (DPMM) very suitable for clustering tasks with uncertainty and unsupervised requirements.

[0161] It is very complicated to directly solve the posterior distribution of the estimated Dirichlet process mixture model, so the variational inference algorithm is used to approximate it, which is explained by the following three formulas:

[0162] Marginal log-likelihood decomposition:

[0163] logp(P)=ELBO(q)+KL(q||p)

[0164] Where: ELBO (Evidence Lower Bound) is the evidence lower bound, which is used to approximate the posterior distribution, and KL divergence indicates the distance between the approximate distribution q(H) and the true posterior distribution p(H|P). The optimization goal is to make q(H) as close to p(H|P) as possible by maximizing ELBO.

[0165] KL divergence formula:

[0166]

[0167] The KL divergence describes the distance between the variational distribution q(H) and the posterior distribution p(H|P). In variational inference, the KL divergence is minimized by maximizing the ELBO so that q(H) can accurately approximate p(H|P).

[0168] ELBO formula:

[0169]

[0170] This formula combines the joint distribution p(P,H) with the variational distribution q(H) to give an optimizable objective function so that q(H) better approximates the true distribution.

[0171] Based on the variational inference algorithm and DPMM model, a data-driven fuzzy set of wind power output uncertainty is constructed. Since wind power generation is affected by the season, it can be divided into four categories according to the season, and the data of different seasons correspond to different labels. The Dirichlet process mixture model is used to perform unsupervised clustering on the data set with the same label, divide it into multiple clusters, and calculate the probability of each cluster by maximum likelihood estimation. The data in each cluster is accompanied by a label and a result based on variational inference, so the fuzzy set of wind power output uncertainty is:

[0172] Π m =Π m,1 ∪Π m,2 ∪Π m,k …∪Π m,K

[0173]

[0174]

[0175] Where: κ m,i is an auxiliary coefficient, calculated as shown in the formula; Π s is the mth type of data-driven uncertainty fuzzy set, Π m,k is the uncertainty fuzzy set of the kth cluster in the mth class, k is the number of clusters in the mth class; Λ m,i is the scaling factor; κ m,i is an auxiliary coefficient, calculated as shown in the formula; d is the correlation coefficient between wind power output data, μ m,i ,ω m,i , ψ m,t is the inference result of the i-th component obtained from data class m, φ m,i It is uncertainty.

[0176] Using the information on the uncertain label data, the probability of data with different labels is extracted. The probability of uncertain data with different labels is described by multinomial distribution, and the probability of each data cluster can be obtained by maximum likelihood estimation:

[0177]

[0178] Where: P m,k is the probability of the kth cluster in the mth class, is the label of the i-th wind power output. N is the total number of wind power output data

[0179] Where σ(*) is an indicator function defined as follows:

[0180]

[0181] Furthermore, the dynamic control strategy of this embodiment is described as follows:

[0182] The dispatch model of the virtual power plant includes two stages: day-ahead optimization and intraday adjustment. Day-ahead optimization uses a 24-hour dispatch cycle, but it has the shortcomings of open-loop dispatch in terms of anti-interference ability and robustness. Therefore, the intraday adjustment method is used to make smaller load adjustments based on real-time load changes. The specific adjustment strategies for the two stages are introduced below.

[0183] Phase 1:

[0184] The optimization goal of the day-ahead is to minimize the operating cost of the virtual power plant, including the electricity purchase cost, electricity sales revenue, combined heat and power unit operating cost, electric energy storage operating cost, and heat pump operating cost. Its expression is:

[0185]

[0186] Where: is the electricity purchase price at time t; is the amount of electricity purchased by the virtual power plant from the upper power grid at time t; is the electricity selling price at time t; is the amount of electricity sold by the virtual power plant to the upper power grid at time t; is the operating cost of the CHP unit at the ith node at time t; The electric energy storage operation cost of the i-th node at time t; The operating cost of the heat pump unit at the i-th node at the t-th moment. Since the operating cost of wind power is relatively low, the present invention does not consider its cost; N is the total number of nodes; T is the total time.

[0187] At the ith node at the tth moment, the cost calculation formula for each unit is:

[0188] C MT =λ 1 P MT +λ 2 Q MT

[0189] C ES =λ 3 P ES

[0190] C HP =λ 4 P HP

[0191] Where: 1 is the cost coefficient of electric power conversion of the cogeneration unit; 2 is the thermal power conversion cost coefficient of the cogeneration unit; 3 is the cost coefficient of electric power conversion from electric energy storage; 4It is the cost coefficient of electrical power consumed by the heat pump unit.

[0192] The constraints include electric power balance constraints, thermal power constraints, virtual power plant power purchase and sales constraints, cogeneration unit output power and ramp constraints, and electric energy storage capacity constraints.

[0193] Electric power balance constraints:

[0194]

[0195] Where: is the wind power output power of the ith node at the tth moment; is the electrical load power demand of the ith node at time t; is the electrical output power of the CHP unit at the ith node at time t; and are the electrical output and input power of the electrical energy storage; is the electrical power consumed by the heat pump at the i-th node at the t-th moment.

[0196] Thermal power balance constraints:

[0197]

[0198] Where: is the heat load power demand of the ith node at time t; is the thermal output power of the CHP unit at the ith node at time t; is the heat pump output power of the ith node at time t

[0199] Virtual power plant power purchase and sales constraints:

[0200]

[0201]

[0202] Where: is the maximum power purchase and sale power of the virtual power plant, θ t It is a Boolean variable, 1 indicates the electricity selling state, and 0 indicates the electricity purchasing state.

[0203] Output power constraints of combined heat and power units:

[0204]

[0205] Where: It is the maximum output power of the cogeneration unit.

[0206] Climbing power limit of cogeneration unit:

[0207]

[0208]

[0209] Where: and They are the rising and falling climbing power limits of the cogeneration unit respectively.

[0210] Heat pump constraints:

[0211]

[0212] Where: and are the maximum and minimum heat load output powers of the heat pump respectively.

[0213] Electric energy storage capacity constraints:

[0214] z min ≤z(t)≤z max

[0215] Where: z min and z max are the minimum and maximum energy storage levels of electrical energy storage, respectively.

[0216] Phase 2:

[0217] Intraday adjustment introduces improved model predictive control (MPC), and establishes a two-layer optimization model that takes dynamic time intervals into consideration based on the dynamic characteristics of the equipment.

[0218] According to the fast and slow characteristics of different energy time scales, a two-layer rolling optimization model is proposed, which divides the scheduling model into a fast control sublayer and a slow control sublayer. The two-layer model is introduced as follows:

[0219] Slow control sublayer (upper layer): uses MPC rolling optimization technology and dynamic time interval algorithms to optimize the system scheduling plan in the prediction domain and control domain. Based on these optimization results, the scheduling instructions in the control domain are executed and passed to the fast control layer, and then wait for feedback from it to complete the scheduling task. After completing the scheduling of the current time interval, the prediction domain and control domain will roll forward to the next time interval and repeat the above process.

[0220] Minimize slow tier costs:

[0221]

[0222] Where: W upper is the slow layer cost; t1 is the upper layer scheduling time interval. ΔP MT (t) is the power adjustment of the cogeneration unit; ΔP HP(t) is the power adjustment of the heat pump unit; μ MT is the unit adjustment cost coefficient of the cogeneration unit; μ HP It is the unit adjustment cost coefficient of the heat pump unit.

[0223] Fast control sublayer (lower layer) model: Wait for the slow control sublayer control domain scheduling plan instruction, and determine the optimization scheduling strategy of the fast dynamic power system of this layer based on it. The fast control sublayer scheduling time window is short. After dynamic rolling optimization of multiple time windows, when the lower layer scheduling time window overlaps with the end time of the upper layer time window, the rolling optimization is stopped, and the scheduling end instruction is sent to the upper layer to enter the next time window of the slow control layer to repeat the scheduling and execution.

[0224] Fast layer cost minimization:

[0225]

[0226] Where: W lower,e is the adjustment cost of the power interaction with the virtual power plant; W B The adjustment cost of charging and discharging electric energy storage; 2 The scheduling interval for the lower layer.

[0227] Adjustment cost of virtual power plant interactive power:

[0228] W lower,e =μ grid ΔP grid (t)t 2

[0229] Where: ΔP grid (t) is the adjustment amount of the power interaction with the power grid; μ grid The unit adjustment cost factor for the tie line.

[0230] Adjustment costs of charging and discharging electric energy storage:

[0231] W B =μ B ΔQ in / out (t)t 2

[0232] Where: μ B is the unit adjustment cost coefficient of electric energy storage; ΔQ in / out (t) is the charge and discharge adjustment power of the energy storage device at time t.

[0233] Prediction model description formula:

[0234] P(t+1|t)=G[P(t),u(t+1|t)]

[0235] Where: P(t) is the load value actually measured at time t; u(t+1|t) is the control quantity at time t acting on time t+1; P(t+1|t) is the load forecast value at time t+1 obtained by the forecast model at time t.

[0236] Load forecast values ​​for all sampling points:

[0237]

[0238] Where: p(t+i|t) is the load forecast value at time t+1 obtained by the forecast model at time t; N p is the prediction step length.

[0239] Dynamic time interval decision index μ t :

[0240]

[0241] Where: W r,t is the predicted trajectory cost; W w,t is the reference trajectory cost; W t is the global cost of the reference trajectory at time t.

[0242] The key to the dynamic time interval method is to set the threshold μ′ of the decision indicator. If the time interval is too large, the error cannot be accurately measured, which leads to an increase in trajectory deviation; if the time interval is too small, the system will be too sensitive to the detection of state error changes, increasing the scheduling frequency and adjustment cost, and weakening the advantages of dynamic time interval scheduling.

[0243] In order to better evaluate the deviation between the reference trajectory and the scheduling trajectory under different thresholds and the corresponding adjustment cost, the performance cost ξ is defined to describe it:

[0244]

[0245] Where: is the adjustment amount of resource k at time t; is the predicted output trajectory; w(t) is the final scheduling trajectory when different time interval decision thresholds are selected; τ p is the trajectory deviation weight coefficient; τ k is the adjustment weight coefficient of the k-th resource

[0246] Furthermore, the deep reinforcement learning algorithm of this embodiment is described as follows. This embodiment solves the above problem based on the optimization algorithm of deep reinforcement learning.

[0247] Deep reinforcement learning (DRL) combines deep learning and reinforcement learning, and uses deep neural networks (DNN) to process and optimize complex high-dimensional state and action spaces. The agent in deep reinforcement learning uses a neural network to fit the value function or policy function, allowing the agent to extract features from the state space and make decisions in a high-dimensional environment.

[0248] Deep reinforcement learning includes five key elements: state S, action A, strategy π, reward R, and value function V:

[0249] In the process of deep reinforcement learning, the agent can handle more complex state and action spaces and optimize the cumulative reward in the long term by using neural networks to approximate the policy or value function. Therefore, the agent network can be used to learn the optimal control strategy to achieve the balance of power supply and demand and optimize the system operation cost and equipment life.

[0250] There are many types of gradient algorithms in deep reinforcement learning algorithms. This paper proposes a twin delayed deep deterministic policy gradient algorithm (TD3) based on communication independence. The algorithm is described as follows:

[0251] The goal of the dual-delayed deep deterministic policy gradient algorithm is to optimize the policy of each virtual power plant device to minimize operating costs and meet grid stability requirements. To this end, a dual-criticism network is used to reduce the Q-value overestimation problem in the deep deterministic policy gradient (DDPG) algorithm. TD3 ensures a more accurate target value by selecting the minimum Q value from the two critic networks.

[0252] TD3Q value update formula:

[0253]

[0254] Where: Q(s,a) represents the Q value of the state-action pair, r is the immediate reward, γ is the discount factor, s′ is the next state, and a′ is the next action. The formula shows that TD3 selects the minimum Q value in the two review networks as the target value for the next step to reduce the error caused by overestimation.

[0255] The loss function of the review network is:

[0256]

[0257] Where: B is a small batch dataset, Qθ (s,a) is the current Q value estimate, and Q is the target Q value. This loss function is used to update the parameters of the review network to make it closer to the target Q value.

[0258] Policy gradient update:

[0259]

[0260] Where: π φ (s) is the action selection based on the policy, and φ is the parameter of the policy network. Through the policy gradient method, TD3 can continuously optimize the policy network so that each VPP agent can better select the optimal action.

[0261] Since each device in VPP acts independently, the environment appears non-stationary to each device. Therefore, Communication-based Independent TD3 (C-ITD3) introduces a neighbor communication strategy so that neighboring devices can exchange selected information to stabilize the training process.

[0262] The independent TD3 algorithm based on communication is introduced as follows:

[0263] State space construction:

[0264]

[0265] Where: S i,t Represents the state space of device i at time t, including the local state and historical information from neighbors With this expansion of the state space, each device can make more comprehensive decisions based on its surroundings.

[0266] Low dimensional fingerprint:

[0267]

[0268] Where: x -i Representing low-dimensional fingerprint information of neighbors,In order to reduce the communication burden, C-ITD3 uses low-dimensional fingerprints to represent neighbor information, so that devices can use the status information of other devices without leaking privacy, thereby converging better under incomplete information.

[0269] Target value update:

[0270]

[0271] By introducing two review networks and selecting the smallest Q value, C-ITD3 further enhances the stability and reduces the impact of non-stationarity on the model.

[0272] The TD3 algorithm defines the strategy update method for VPP devices, solving the overestimation problem in traditional DDPG. Then the module introduces the communication-based strategy C-ITD3, which enables the agent to use neighbor information to stabilize strategy updates in a non-stationary environment, autonomously learn to make optimal decisions, and ultimately achieve coordinated control of various devices in the virtual power plant to ensure real-time demand and economy.

[0273] The beneficial effects of the present invention are as follows:

[0274] (1) The present invention establishes a dynamic model of a virtual power plant including a wind turbine, a cogeneration unit, a heat pump, and electric energy storage, thereby improving the accuracy of the model.

[0275] (2) The present invention proposes a two-stage dynamic control method for a virtual power plant based on improved model predictive control, which exploits the differences in equipment and network characteristics and reduces adjustment costs.

[0276] (3) The present invention proposes a communication-independent dual-delay deep deterministic policy gradient reinforcement learning algorithm, which enhances the processing capability of high-dimensional state and action space, has real-time adaptive optimization capability, and improves solution accuracy and decision-making efficiency.

[0277] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0278] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A virtual power plant dynamic control method based on deep reinforcement learning, characterized in that: include: Construct a dynamic model of a virtual power plant; The virtual power plant dynamic model includes a network dynamic model and an equipment dynamic model; Construct wind power output uncertainty model based on historical wind power data; A virtual power plant dispatching model is constructed based on the virtual power plant dynamic model and the wind power output uncertainty model; the virtual power plant dispatching model is divided into two stages: day-ahead optimization and intraday adjustment; The virtual power plant scheduling model is optimized based on deep reinforcement learning optimization strategy to improve the economy and real-time performance of scheduling by learning the dynamic characteristics and scheduling strategies of the equipment.

2. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 1 is characterized in that: The network dynamic model includes a power grid dynamic model and a heat network dynamic model; the method for establishing the power grid dynamic model includes: The distribution of voltage u(x, ω) and current i(x, ω) is described by a group of partial differential equations; the formula of the group of partial differential equations is: Where Z(ω) and Y(ω) represent the impedance and admittance of the line respectively; the complex forms of impedance Z(ω) and admittance Y(ω) are defined as: R(ω) and L(ω) represent the resistance and inductance of the line, respectively, indicating the resistance encountered when the current flows through the conductor and the self-inductance effect caused by the change of current; G(ω) and C(ω) represent the conductance and capacitance, respectively, ω is the frequency, x is the position coordinate of the line; k is the imaginary unit; The expression of the dynamic model of the heating network is: Among them, M i represents the mass of mass point i; h i represents the specific enthalpy of mass point i; dh i / dt represents the rate of change of specific enthalpy with time, which represents the rate of change of energy of mass point i; D ij Indicates the degree of coupling between mass points i and j, which is related to the distance between the two points; G ij represents the flow conductivity of the fluid from mass point j to i; h j The specific enthalpy of other mass points i coupled with mass point i; Q i represents the heat received by mass point i; N is the total number of nodes.

3. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 1 is characterized in that: The equipment dynamic model includes a wind turbine dynamic model, a cogeneration unit dynamic model, an electric energy storage dynamic model and a heat pump dynamic model; The steps of establishing the dynamic model of the wind turbine generator set are: Construct an expression for the rotor torque; the rotor torque T ω' The expression is: Among them, P BLADE is the power captured by the blade; C p (λ,β) is the power coefficient, λ is the tip speed ratio, β is the pitch angle, g is the acceleration due to gravity, ρ is the air density, A is the rotor swept area, v is the wind speed, ω' is the rotor angular velocity, and H is the rotor moment of inertia; Construct an expression of electromagnetic torque; the electromagnetic torque T e The expression is: T e =K e i f θ e ; Among them, K e is the electromagnetic torque constant; i f is the excitation current; θ e is the electromagnetic angle; A differential equation of the rotor torque is constructed; the differential equation of the rotor torque is expressed as: in, is the rate of change of the rotor angular velocity; T ω is the torque generated by the wind; T e is the electromagnetic torque; it represents the feedback effect of the motor control system on the rotor speed. Construct an expression for the generator power; the generator power P gen The expression is: Where L is the inductance of the generator; R is the resistance; i is the current; k e is the electromotive force constant; The steps of establishing the dynamic model of the cogeneration unit are: Construct a boiler mass flow rate equation; the boiler mass flow rate equation is: M =u B (t-τ); where r M is the adjusted boiler mass flow rate; u B is the fuel flow rate; τ is the time delay of the fuel reaching the burner; Construct a dynamic equation of the grinding mill; the expression of the dynamic equation of the grinding mill is: Among them, r B is the boiler pressure response rate; r M is the boiler mass flow rate; T f is the coal mill time constant; Construct a boiler energy balance equation; the boiler energy balance equation is: Among them, P a is the throttle valve pressure; C b is the boiler capacity; K1 and K3 are proportional constants; u T is the opening of the turbine input regulating valve; Construct a throttle valve pressure equation; the throttle valve pressure equation is: P a =P b -K2(K1r B ) 1.5 ; Among them, P b Indicates the drum pressure; K1 and K2 are both proportional constants. Construct a turbine energy balance equation; the turbine energy balance equation is: Where: T t is the turbine time constant; P MT is the power generation capacity of the thermal power unit; P e is the heat removal pressure; K3, K4 and K5 are related constants; u H It is the opening degree of the heat regulating butterfly valve; Construct a formula for the heating power of a cogeneration unit; the formula for the heating power of the cogeneration unit is: Q MT =σP MT ; where σ is the thermoelectric conversion coefficient; Q MT is the heat production power of the cogeneration unit. The expression of the electric energy storage dynamic model is: Among them, Q cap is the energy storage capacity; z is the storage state of the energy storage; η e is the round-trip energy efficiency factor of the electric energy storage; p + and p - Represent the charging power and discharging power of the battery respectively. The steps of establishing the heat pump dynamic model include: Construct a relationship equation between compressor power and outlet temperature; the relationship equation between compressor power and outlet temperature is: Among them, T out,co is the compressor inlet temperature; T in,co is the compressor outlet temperature; η co is the isentropic efficiency; β co is the compression ratio; κ is the air isentropic index; is the mass flow rate; c p,co is the specific heat at constant pressure; A dynamic equation of the compressor outlet pressure is constructed; the dynamic equation of the compressor outlet pressure is: Where: P out,co is the compressor outlet pressure; R g is the gas constant; V is the volume of the corresponding chamber of the compressor; and are the mass flow rates entering and leaving the cavity, respectively; Build the output power W of the expander ex and outlet temperature T out,ex Formula: The output power W of the expander ex and outlet temperature T out,ex The formula is: Among them, T in,ex is the expander inlet temperature; T out,ex is the expander outlet temperature; η ex is the isentropic efficiency; β ex is the expansion ratio; is the mass flow rate; c p,ex is the specific heat at constant pressure; The kinetic equation of the expander outlet pressure is constructed; the kinetic equation of the expander outlet pressure is: Among them, P out,ex is the expander outlet pressure, U is the volume of the corresponding chamber of the expander, and are the mass flow rates entering and leaving the cavity, respectively; Construct the validity equation of the outlet temperature of the cold and hot fluids of the heat exchanger; the validity equation of the outlet temperature of the cold and hot fluids of the heat exchanger is: Among them, T in,h and T in,c are the inlet temperatures of the hot and cold fluids, respectively, T out,h and T out,c are the outlet temperatures, respectively, and λ is the effectiveness coefficient of the heat exchanger; Construct the condenser thermal fluid thermal power Q c Formula; the condenser thermal fluid thermal power Q c The formula is: Q c t=c p m w (T out,h -T in,h ), where c p is the constant pressure heat capacity of water; m w is the mass flow rate of the thermal fluid; t is the time; Construct a heat pump power conversion formula; the heat pump power conversion formula is: P HP =λ cop Q c ; Among them, λ cop is the heating coefficient of the heat pump, P HP The heat pump consumes electrical power.

4. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 1 is characterized in that: A wind power output uncertainty model is constructed based on historical wind power data, including: Unsupervised clustering of wind power historical data using Dirichlet process mixture model; Dynamically adjust the number of clusters and divide multiple clusters according to data complexity; The posterior distribution is approximately solved by the variational inference algorithm, and the lower bound of evidence is optimized to approach the true distribution and obtain the clustering result; Using the clustering results, the wind power data is divided by season to generate fuzzy sets with different labels; Calculate the probability of each cluster and obtain the probability distribution of each cluster through maximum likelihood estimation; An uncertainty fuzzy set formula is constructed, and the uncertainty of wind power output is described in combination with auxiliary coefficients and proportional factors to obtain the wind power output uncertainty model.

5. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 4 is characterized in that: The uncertainty fuzzy set formula include: P m =P m,1 ∪Π m,2 ∪Π m,k …∪P m,K Among them, κ m,i is an auxiliary coefficient; Π s is the mth type of data-driven uncertainty fuzzy set, Π m,k is the uncertainty fuzzy set of the kth cluster in the mth class, k is the number of clusters in the mth class; Λ m,i is the proportionality factor; d is the correlation coefficient between wind power output data, μ m,i ,ω m,i , ψ m,t is the inference result of the i-th component obtained from data class m, φ m,i It is uncertainty.

6. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 1 is characterized in that: In the day-ahead optimization stage, the method for constructing the virtual power plant scheduling model includes: The day-ahead optimization objective is to minimize the operating cost of the virtual power plant; the expression for minimizing the operating cost of the virtual power plant is: in, is the electricity purchase price at time t; P t VPP,b is the amount of electricity purchased by the virtual power plant from the upper power grid at time t; is the electricity price at time t; P t VPP,s is the amount of electricity sold by the virtual power plant to the upper power grid at time t; is the operating cost of the CHP unit at the ith node at time t; The electric energy storage operation cost of the i-th node at time t; The operating cost of the heat pump unit at the i-th node at time t, N is the total number of nodes; T is the total time; Calculate the cost of each unit at the i-th node at the t-th time; the calculation formula of the unit cost includes: C MT =λ1P MT +λ2Q MT C ES =λ3P ES C HP =λ4P HP Among them, λ1 is the cost coefficient of electric power conversion of the cogeneration unit; λ2 is the cost coefficient of thermal power conversion of the cogeneration unit; λ3 is the cost coefficient of electric power conversion of electric energy storage; λ4 is the cost coefficient of electric power consumption of the heat pump unit; P HP The electrical power consumed by the heat pump, P MT P is the power generation capacity of the combined heat and power unit; ES is the output power of the electric energy storage; Q MT The heat production power of the combined heat and power unit; Construct constraints to form the virtual power plant dispatch model in the stage of day-ahead optimization; the virtual power plant dispatch model includes electric power balance constraints, thermal power constraints, virtual power plant power purchase and sales constraints, cogeneration unit output power constraints, cogeneration unit ramp power limit constraints, heat pump constraints and electric energy storage capacity constraints.

7. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 6 is characterized in that: The expression of the electric power balance constraint is: in, is the wind power output power of the ith node at the tth moment; is the electrical load power demand of the ith node at time t; is the electrical output power of the CHP unit at the ith node at time t; and are the electrical output and input power of the electrical energy storage; is the electrical power consumed by the heat pump at the i-th node at time t; The expression of the thermal power balance constraint is: in, is the heat load power demand of the ith node at time t; is the thermal output power of the CHP unit at the ith node at time t; is the heat pump output power of the ith node at time t The expression of the power purchase and sale constraint of the virtual power plant is: in, is the maximum power purchase and sale power of the virtual power plant, θ t It is a Boolean variable, 1 indicates the electricity selling state, and 0 indicates the electricity purchasing state; The expression of the output power constraint of the cogeneration unit is: in, is the maximum output power of the CHP unit; is the electrical output power of the cogeneration unit at the i-th node at the t-th moment; The expression of the ramp power limit constraint of the cogeneration unit is: in, and They are the rising and falling ramp power limits of the cogeneration unit respectively; is the electrical output power of the cogeneration unit at the i-th node at time t-1; The expression of the heat pump constraint is: in, and are the maximum and minimum heat load output powers of the heat pump respectively; is the heat production power of the heat pump unit at the tth moment; The expression of the electric energy storage capacity constraint is: min ≤z(t)≤z max ; Among them, z min and z max are the minimum and maximum energy storage levels of the electric energy storage respectively; z(t) is the energy storage level of the electric energy storage at the tth moment.

8. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 1 is characterized in that: During the intraday adjustment stage, the virtual power plant scheduling model includes a two-layer rolling optimization model of a fast control sublayer and a slow control sublayer.

9. The method for dynamic control of virtual power plants based on deep reinforcement learning according to claim 1 is characterized in that: The deep reinforcement learning optimization strategy is a communication-independent dual-delay deep deterministic policy gradient algorithm; the goal of the communication-independent dual-delay deep deterministic policy gradient algorithm is to optimize the strategy of each virtual power plant device to minimize operating costs and meet grid stability requirements.

Images