Automatic power generation control method and system for large-scale new energy grid connection, medium and processor

Abstract

The invention relates to the technical field of automatic power generation control, in particular to an automatic power generation control method and system for large-scale new energy grid connection, a medium and a processor. A distributed power grid control mechanism is established, a multi-region interconnected power grid model is merged into a power grid, the power distribution proportion of a thermal power generating unit, a hydroelectric generating unit and an electric vehicle cluster is calculated, the region control deviation of each region is calculated, a distributed deep reinforcement learning controller takes the region control deviation of the corresponding region as input, and the distributed deep reinforcement learning control method is applied to the power grid. Iterative updating is carried out through an internal control method of a deep reinforcement learning controller, total control power of a corresponding area is output, then power instructions of a thermal power generating unit, a hydroelectric generating unit and an electric vehicle cluster of the corresponding area are obtained, and the thermal power generating unit, the hydroelectric generating unit and the electric vehicle cluster execute the corresponding power instructions to achieve system closed loop. And interacting with a power grid. According to the invention, CPS index qualification and frequency stability of the multi-regional power grid can be realized.

Description

Technical Field

[0001] The present invention relates to the field of automatic power generation control technology, and in particular to an automatic power generation control method, system, medium and processor for large-scale new energy grid connection. Background Art

[0002] With the increasingly severe environmental problems such as the tight supply of fossil energy and the greenhouse effect, the transformation to renewable energy is an inevitable trend in China and even the world's energy development. In order to achieve the goal of renewable energy transformation, it is necessary to build a clean, low-carbon, safe and efficient energy system, implement renewable energy substitution actions, deepen the reform of the power system, and build a new power system with new energy as the main body. In the context of building a new power system, the proportion of new energy access to the power system continues to increase, and its power intermittent, random and unpredictable nature has brought many problems to the power grid. Among them, the volatility of new energy is converted to a short time scale of seconds and minutes, and the power grid is in urgent need of frequency modulation resources.

[0003] With the rapid development of smart grids, the installed capacity continues to expand, and new energy and distributed energy are continuously connected. The traditional centralized automatic generation control (AGC) mode is difficult to meet the development and operation conditions of the power grid, and the distributed AGC control mode is imperative.

[0004] From the perspective of distributed energy utilization, due to the limitations of grid structure, power plant capacity and unit regulation rate, different types of power plants have huge differences in power distribution and utilization. The power of power plants controlled by centralized AGC can only be distributed through provincial regulation, and it is difficult for distributed energy power plants and new energy power plants to achieve coordinated control. These small-capacity power plants without fixed power generation indicators are difficult to achieve efficient utilization under the centralized AGC mode. Obviously, centralized AGC lacks economy and scientificity, and it is difficult to meet the development model of the power grid. From the opposite perspective, it shows that the study of distributed AGC is of great significance.

[0005] At present, most studies mainly use traditional control methods (such as Q learning) as frequency modulation control strategies. In the face of new power systems dominated by renewable energy, discrete Q learning and its derived control algorithms have poor adaptability. When the power system changes, it is impossible to quickly achieve system control of each generator set, and the control accuracy is poor. Therefore, this field urgently needs to study a reinforcement learning control strategy or method for distributed AGC. Summary of the invention

[0006] In view of the problems in the prior art, the present invention provides an automatic power generation control method, system, medium and processor for large-scale new energy grid connection. The specific technical solutions are as follows:

[0007] An automatic power generation control method for large-scale renewable energy grid connection includes the following steps:

[0008] Build a multi-region interconnected power grid model, initialize model parameters, and integrate the multi-region interconnected power grid model into the power grid; the multi-region interconnected power grid model includes deep reinforcement learning controllers, wind turbines, photovoltaic power stations, thermal power units, hydropower units, electric vehicle clusters, and loads in each region; the deep reinforcement learning controller is connected to the thermal power units, hydropower units, and electric vehicle clusters respectively;

[0009] Calculate the power allocation ratio of thermal power units, hydropower units and electric vehicle clusters based on the capacities of thermal power units, hydropower units and electric vehicle clusters in each region;

[0010] According to the output power of wind turbines, photovoltaic power stations, thermal power units, hydropower units, the power of electric vehicle clusters participating in frequency modulation, and loads in each region, the real-time frequency deviation of the power grid in each region and the exchange power between adjacent regions are calculated, and the regional control deviation of each region is calculated;

[0011] The regional control deviation of each area is input into the deep reinforcement learning controller of the corresponding area, and the internal control method of the deep reinforcement learning controller is used to iteratively update and output the total control power of the corresponding area;

[0012] The allocated power of the thermal power units, hydropower units and electric vehicle clusters is obtained according to the total control power of the corresponding area and the power allocation ratio of the thermal power units, hydropower units and electric vehicle clusters, and then the power instructions of the thermal power units, hydropower units and electric vehicle clusters in the corresponding area are obtained;

[0013] Thermal power units, hydropower units, and electric vehicle clusters execute corresponding power instructions and interact with the power grid.

[0014] Preferably, the specific calculation method of the power distribution ratio of the thermal power unit, the hydropower unit, and the electric vehicle cluster is as follows:

[0015] Where: a f 、a h 、a ev are the power allocation ratios of thermal power units, hydropower units, and electric vehicle clusters, respectively. f +a h +a ev =1 and a f 、a h 、a ev ∈[0,1]; E f 、E hare the rated capacities of thermal power units and hydropower units, E ev,all is the available capacity of the electric vehicle cluster.

[0016] Preferably, the calculation method of the regional control deviation of each region is as follows:

[0017] Δf k =L -1 {(P g,k +P pv,k +P h,k +P f,k +P ev,k -P L,k -P tie,kj )×K p / (1+sT p )};

[0018] P tie,kj =L -1 {(Δf k -Δf j )×(2πT kj / s)};

[0019] ACE k =ΔP tie,kj +β k Δf k ;

[0020] Where Δf k is the real-time frequency deviation of the power grid in region k, L -1 is the inverse Laplace transform, P g,k is the wind turbine output power in region k, P pv,k is the output power of the photovoltaic power station in area k, P h,k is the output power of the hydropower unit in area k, P f,k is the output power of thermal power units in region k, P ev,k is the power of the electric vehicle cluster in area k when participating in frequency regulation, P L,k is the load of area k, P tie,kj is the interconnection line exchange power between area k and area j, where area k is adjacent to area j, K p is the frequency response function coefficient, T p is the time constant of the frequency response function, s is the Laplace operator, Δf j is the real-time frequency deviation of the power grid in region j, T kj is the time constant of the tie line between area k and area j; ACE k is the regional control deviation of region k, β k is the frequency deviation coefficient of region k.

[0021] Preferably, the power P of the electric vehicle cluster participating in frequency modulation in area k is ev,k The details are as follows:

[0022] Build a dynamic change model of the number of controllable electric vehicles, as follows:

[0023] Let the starting time be t 0 , the time when electric vehicles in area k are connected to the grid is t 1 , assuming that the charging time of the electric vehicle is T 2 , from which we can get the time t when the electric vehicle enters the frequency modulation operation 3 for:

[0024] t 3 =t 1 +T 2 ;

[0025] The electric vehicle determines whether it can participate in the frequency modulation service based on its own state of charge. The calculation of the electric vehicle's own state of charge is as follows;

[0026]

[0027] Where: SOC 0 is the current state of charge of a single electric vehicle; E ev is the existing capacity of a single electric vehicle; E ev,max is the maximum capacity of a single electric vehicle;

[0028] Electric vehicle charging time T 2 Calculated by the formula:

[0029]

[0030] in, is the charging power of a single electric vehicle; η ev Charging efficiency of electric vehicles; SOC min The minimum state of charge required for electric vehicles to participate in frequency regulation;

[0031] The Monte Carlo method is used to randomly sample the electric vehicle entry / exit controllable timetable for statistical analysis to obtain the number of electric vehicles in the controllable state N at each time in the area. in '(t i ) and the number of electric vehicles in controllable state N out '(t i ), the cumulative number of electric vehicles N that enter the controllable state at time t is calculated by the following formulas: in (t) and the cumulative number of electric vehicles N that are out of controllable state out (t):

[0032]

[0033]

[0034] Assume t 0 The initial number of controllable electric vehicles at the time is N 0 , the number of controllable electric vehicles N at time t c (t) is shown as follows: N c (t) = N 0 +N in (t)-N out (t);

[0035] So the capacity of the electric vehicle cluster available at time t is for:

[0036]

[0037] When participating in frequency regulation, the total available capacity of electric vehicles needs to be subject to certain restrictions. When the existing capacity is less than the minimum capacity limit, the available capacity of electric vehicles is zero, then:

[0038] The lower limit of the capacity of the electric vehicle cluster at time t for:

[0039]

[0040] The upper limit of the capacity of the electric vehicle cluster at time t for:

[0041]

[0042] The available capacity E of the electric vehicle cluster at time t ev,all The calculation formula is as follows:

[0043]

[0044] Calculate a from the existing capacity ev , the power P of the electric vehicle cluster participating in frequency regulation in area k ev,k The calculation method is as follows:

[0045] P ev,k =P t,k ·a ev ;

[0046] Among them, P t,k is the total power command output by the deep reinforcement learning controller in region k at time t.

[0047] Preferably, the deep reinforcement learning controller includes two policy networks: and 2 evaluation networks

[0048] The strategy network includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, and an output layer. It is used to output action a under state S, where state S is the regional control deviation of the corresponding area, and action a is to output the total control power deviation of the corresponding area; the mathematical expression of the strategy network is as follows:

[0049]

[0050] Where: y u is the input of the policy network; y 1 ,y 2 are the outputs of the first hidden layer and the second hidden layer respectively; μ and logσ are the outputs of the third hidden layer and the fourth hidden layer respectively; W μ , W logσ are the weight matrices of the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer respectively; b μ 、b logσ is the bias vector of the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer; the output action a follows the normal distribution N(μ,σ), and the mean μ and standard deviation σ are the mean and standard deviation of the normal distribution followed by the output action a respectively; tanh(x)=(e x -e -x ) / (e x +e -x ) is the activation function, A is the maximum action value;

[0051] The evaluation network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. It is used to judge the output value Q of the output action a under state S. The mathematical expression of the evaluation network is as follows:

[0052]

[0053] Among them, x u is the input of the evaluation network, x 1 、x 2 are the outputs of the first hidden layer and the second hidden layer respectively; Q is the output of the output layer; are the weight matrices of the first hidden layer, the second hidden layer, and the output layer respectively; is the bias vector of the first hidden layer, the second hidden layer, and the output layer; relu(x)=max(0,x) is the activation function, which ensures that the output value is not negative.

[0054] Preferably, the regional control deviation of each region is input into the deep reinforcement learning controller of the corresponding region, and the internal control method of the deep reinforcement learning controller is used to iteratively update and output the total control power of the corresponding region, which specifically includes the following steps:

[0055] Initialize 2 evaluation networks and 2 policy networks Set the maximum action value, learning rate α, discount factor γ, entropy regularization coefficient τ 1 、Environmental experience buffer pool B 1 and expert experience buffer pool B 2 and its upper limit B 2max , random action N, optimal action proportion coefficient ρ, expert experience proportion ξ;

[0056] Input the regional control deviation ACE value of the corresponding area at time t and use it as the environmental state S at time t t ;

[0057] Judgment Expert Experience Pool B 2 Whether the amount of stored information has reached the upper limit B 2max ; If the expert experience pool B 2 The amount of stored information has reached the upper limit B 2max , then through the strategy network at time t Output the environmental state S at time t t The next action a t ; If the expert experience pool B 2 The amount of stored information has not reached the upper limit B 2max , then the environmental state S at time t is selected based on expert experience t Next action

[0058] Calculate real-time reward r t ;

[0059] Store the environment information in the environment experience buffer pool B 1 , store the expert information in the expert experience buffer pool B 2 , the environmental information includes the environmental state S at the previous moment t-1 , corresponding real-time strategy Action t-1 , the reward r at the previous moment t-1 , the environmental state at this moment S t ; Expert information includes the environment state S at the last moment t-1 , corresponding actions selected by expert experience Rewards from the last moment The environmental state at this moment is S t ;

[0060] According to the expert experience proportion ξ, in the expert experience buffer pool B 2 With the same probability, randomly select data, and then from the environment experience buffer pool B 1 With the same probability data, and the two together constitute the training data, where G is the number of training samples set for the small batch size reduction;

[0061] Use the training data to update the parameters of the evaluation network and the strategy network, and update the expert experience ratio ξ;

[0062] Output power action a, calculate the total power command at time t in the corresponding area.

[0063] Preferably, the total power instruction at the corresponding area at time t is calculated as follows:

[0064] If the expert experience buffer pool B 2 The stored information does not exceed the storage limit:

[0065]

[0066] If the expert experience buffer pool B 2 The stored information exceeds the storage limit:

[0067] P t =P t-1 +a t ;

[0068] Among them, P t is the total power command at time t in the corresponding area, P t-1 is the total power command of the corresponding area at time t-1.

[0069] An automatic power generation control system for large-scale renewable energy grid connection, applied to the method described, comprising:

[0070] A model building module is used to build a multi-region interconnected power grid model, initialize model parameters, and integrate the multi-region interconnected power grid model into the power grid; the multi-region interconnected power grid model includes a deep reinforcement learning controller, a wind turbine, a photovoltaic power station, a thermal power unit, a hydropower unit, an electric vehicle cluster, and a load in each region; the deep reinforcement learning controller is connected to the thermal power unit, the hydropower unit, and the electric vehicle cluster respectively;

[0071] A power allocation ratio calculation module is used to calculate the power allocation ratio of thermal power units, hydropower units and electric vehicle clusters according to the capacities of thermal power units, hydropower units and electric vehicle clusters in each area;

[0072] The regional control deviation calculation module is used to calculate the real-time frequency deviation of the power grid in each region and the exchange power between adjacent regions based on the output power of the wind turbines in each region, the output power of the photovoltaic power station, the output power of the thermal power units, the output power of the hydropower units, the power of the electric vehicle cluster participating in the frequency modulation, and the load, and calculate the regional control deviation of each region; the deep reinforcement learning controller is used to input the regional control deviation of each region into the deep reinforcement learning controller of the corresponding region, iteratively update through the internal control method of the deep reinforcement learning controller, and output the total control power of the corresponding region;

[0073] A total power command calculation module is used to obtain the allocated power of the thermal power unit, the hydropower unit, and the electric vehicle cluster according to the total control power of the corresponding area and the power allocation ratio of the thermal power unit, the hydropower unit, and the electric vehicle cluster, and then obtain the power command of the thermal power unit, the hydropower unit, and the electric vehicle cluster in the corresponding area;

[0074] The grid interaction module is used for thermal power units, hydropower units, and electric vehicle clusters to execute corresponding power instructions and interact with the grid.

[0075] A computer-readable storage medium includes a stored program, wherein when the program is run, the device where the computer-readable storage medium is located is controlled to execute the automatic power generation control method for large-scale new energy grid connection.

[0076] A processor is used to run a program, wherein the program executes the automatic power generation control method for large-scale new energy grid connection when running.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] The present invention establishes a distributed power grid control mechanism, incorporates a multi-region interconnected power grid model into the power grid, calculates the power allocation ratio of thermal power units, hydropower units, and electric vehicle clusters, calculates the regional control deviation of each region, and the distributed deep reinforcement learning controller takes the regional control deviation of the corresponding region as input, iterates and updates through the internal control method of the deep reinforcement learning controller and outputs the total control power of the corresponding region, thereby obtaining the power instructions of the thermal power units, hydropower units, and electric vehicle clusters in the corresponding region. The thermal power units, hydropower units, and electric vehicle clusters execute the corresponding power instructions to realize the system closed loop and interact with the power grid. The control method of the present invention can achieve qualified CPS indicators and frequency stability of multi-regional power grids, the absorption of large-scale new energy in new power systems, and the coordinated control of generator sets through the designed controller. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0080] Figure 1 It is a schematic diagram of the process of the present invention;

[0081] Figure 2 A multi-region interconnected power grid model constructed in an embodiment of the present invention;

[0082] Figure 3 It is the principle model of wind-solar-thermal-hydro-electric vehicle in the multi-regional interconnected power grid model;

[0083] Figure 4 Schematic diagram of wind and solar power curve in the embodiment;

[0084] Figure 5 It is a curve chart showing the capacity change of electric vehicles in one day;

[0085] Figure 6 It is a schematic diagram of the structure of the strategy network;

[0086] Figure 7 This is a schematic diagram of the evaluation network structure;

[0087] Figure 8 This is a flow chart of the internal control method of the deep reinforcement learning controller;

[0088] Fig. 9 Schematic diagram of the update of the policy network and the evaluation network;

[0089] Fig.10 This is a comparison curve of the control performance indicators of the two-region power grid under the irregular random disturbance of the present invention compared with different methods, where: Fig.10 (a) Irregular random disturbance waveform, Fig.10 (b) is the frequency curve under irregular random disturbance; Fig.10 (c) is the 10-min ACE average value curve under irregular random disturbance; Fig.10 (d) is the 10-min CPS1 average value curve under irregular random disturbance;

[0090] Fig.11 It is a system principle diagram of the present invention. DETAILED DESCRIPTION

[0091] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0092] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0093] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0094] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0095] Embodiment 1:

[0096] like Figure 1 As shown, this embodiment provides an automatic power generation control method for large-scale renewable energy grid connection, which specifically includes the following steps:

[0097] Step S1, build a multi-regional interconnected power grid model, initialize the model parameters, and integrate the multi-regional interconnected power grid model into the power grid; the multi-regional interconnected power grid model includes deep reinforcement learning controllers, wind turbines, photovoltaic power stations, thermal power units, hydropower units, electric vehicle clusters, and loads in each region; the deep reinforcement learning controller is connected to the thermal power unit, hydropower unit, and electric vehicle cluster respectively. In this embodiment, the present invention is based on the realistic background of large-scale new energy (wind power, photovoltaic and electric vehicle clusters) grid connection, such as Figure 2 As shown in the figure, a two-region interconnected power grid model that conforms to reality is built to verify the collaborative control strategy based on deep reinforcement learning designed by the present invention. The regional power interconnection line is the power AC interconnection line of each region. When the power generation and power consumption in one region are unbalanced, the power of other regions is called through the interconnection line to maintain the stability of the entire power grid. In order to verify the control ability of the controller, the load is a load signal randomly generated by MATLAB / Simulink.

[0098] Step S2, calculating the power allocation ratio of the thermal power units, hydropower units, and electric vehicle clusters according to the capacities of the thermal power units, hydropower units, and electric vehicle clusters in each region; the calculation method for each region is the same, and the specific calculation is as follows:

[0099]

[0100] Where: a f 、a h 、a ev are the power allocation ratios of thermal power units, hydropower units, and electric vehicle clusters, respectively. f +a h +a ev =1 and a f 、a h 、a ev ∈[0,1]; E f 、E h are the rated capacities of thermal power units and hydropower units, respectively, which are two fixed values; E ev,all is the available capacity of the electric vehicle cluster, which is a value that changes in real time.

[0101] Step S3, calculate the real-time frequency deviation of the power grid in each region and the exchange power between adjacent regions based on the output power of the wind turbines in each region, the output power of the photovoltaic power station, the output power of the thermal power units, the output power of the hydropower units, the power of the electric vehicle cluster participating in the frequency modulation, and the load, and calculate the regional control deviation of each region. The calculation method of the regional control deviation of each region is as follows:

[0102] Δf k =L -1 {(P g,k +P pv,k +P h,k +P f,k +P ev,k -P L,k -P tie,kj )×K p / (1+sT p )};(2)

[0103] P tie,kj =L -1 {(Δf k -Δf j )×(2πT kj / s)};(3)

[0104] ACE k =ΔP tie,kj +β k Δf k ; (4)

[0105] Where Δfk is the real-time frequency deviation of the power grid in region k, L -1 is the inverse Laplace transform, P g,k is the wind turbine output power in region k, P pv,k is the output power of the photovoltaic power station in area k, P h,k is the output power of the hydropower unit in area k, P f,k is the output power of thermal power units in region k, P ev,k is the power of the electric vehicle cluster participating in frequency regulation in area k, P L,k is the load of area k, P tie,kj is the interconnection line exchange power between area k and area j, where area k is adjacent to area j, K p is the frequency response function coefficient, T p is the time constant of the frequency response function, s is the Laplace operator, Δf j is the real-time frequency deviation of the power grid in region j, T kj is the time constant of the tie line between area k and area j; ACE k is the regional control deviation of region k, β k is the frequency deviation coefficient of region k.

[0106] like Figure 3 As shown in the figure, the generator sets are thermal power steam turbine units and hydropower units that take into account the generator output rate control (GRC). The thermal power generator set includes a speed governor, GRC, steam turbine and power limiter module, and the hydropower generator set includes a speed governor, GRC, turbine and power limiter module. The above modules are used to simulate the actual working conditions of thermal power and hydropower units. Figure 2 and Figure 3 The output power calculation formula of thermal power units and hydropower units can be obtained:

[0107] The output power P of thermal power units in region k f,k as follows:

[0108] P f,k =L -1 {(P t,k ·a f -Δf / R 1 ) / (1+sT g )(1+sT f )};(5)

[0109] Among them, P t,k is the total power command output by the deep reinforcement learning controller in region k at time t, a f is the power allocation ratio of thermal power units, R 1 is the droop coefficient of the thermal power unit, Tg is the time delay constant of the thermal power unit speed governor, T f is the time constant of the thermal power unit.

[0110] The output power P of the hydropower unit in area k h,k as follows:

[0111] P h,k =L -1 {(P t,k ·a h -Δf / R 2 )·(sT rs +1)(-sT w1s +1) / (1+sT gh )(1+sT rh )(1+sT rh )};(6)

[0112] Among them, a h is the power distribution ratio of the hydropower unit, R 2 is the droop coefficient of the hydropower unit, T rs , T w1s , T rh is the time constant of the hydropower unit; T gh is the time delay constant of the hydropower unit speed governor.

[0113] Wind turbine output power P in region k g,k The calculation of is as follows:

[0114]

[0115] Where: is the rated power of the fan; V w is the actual wind speed; is the rated wind speed; is the cut-in wind speed; To cut out wind speed.

[0116] The output power P of the photovoltaic power station in area k pv,k as follows:

[0117]

[0118] Where: is the rated power generation of the photovoltaic power station; c pv is the photovoltaic temperature conversion power coefficient; T is the current temperature; T ref is the reference value of temperature; s pv is the light intensity at the current moment.

[0119] Wind power has a high uncertainty and has a high output power in the evening; photovoltaic power has an output power during the day and has the highest output power at noon. Figure 4 Using the above wind power and photovoltaic output can effectively simulate the actual wind power and photovoltaic output.

[0120] Electric vehicles participate in the frequency regulation of the power system through vehicle-to-grid (V2G) technology, and achieve rapid response to the grid frequency through centralized control. They discharge when the grid load is high and charge when the load is low, helping the grid to regulate peak and frequency. The power P of the electric vehicle cluster participating in the frequency regulation in area k is ev,k The details are as follows:

[0121] Build a dynamic change model of the number of controllable electric vehicles, as follows:

[0122] Let the starting time be t 0 , the time when electric vehicles in area k are connected to the grid is t 1 , assuming that the charging time of the electric vehicle is T 2 , from which we can get the time t when the electric vehicle enters the frequency modulation operation 3 for:

[0123] t 3 =t 1 +T 2 ; (9)

[0124] Electric vehicles need to determine whether they can participate in frequency modulation services based on their own state of charge (SOC). 0 The calculation of is as follows;

[0125]

[0126] Where: SOC 0 is the current state of charge of a single electric vehicle; E ev is the existing capacity of a single electric vehicle; E ev,max is the maximum capacity of a single electric vehicle;

[0127] Electric vehicle charging time T 2 Calculated by the formula:

[0128]

[0129] in, is the charging power of a single electric vehicle; η ev Charging efficiency of electric vehicles; SOC min The minimum state of charge required for electric vehicles to participate in frequency regulation;

[0130] The Monte Carlo method is used to randomly sample the electric vehicle entry / exit controllable timetable for statistical analysis to obtain the number of electric vehicles in the controllable state N at each time in the area. in '(t i ) and the number of electric vehicles in controllable state N out '(t i ), the following formula can be used to calculate the cumulative number of electric vehicles N that enter the controllable state at time t in (t) and the cumulative number of electric vehicles N that are out of controllable state out (t):

[0131]

[0132]

[0133] Assume t 0 The initial number of controllable electric vehicles at the time is N 0 , the number of controllable electric vehicles N at time t c (t) is shown in the following formula:

[0134] N c (t) = N 0 +N in (t)-N out (t); (14)

[0135] So the capacity of the electric vehicle cluster available at time t is for:

[0136]

[0137] When participating in frequency regulation, the total available capacity of electric vehicles needs to be subject to certain restrictions. When the existing capacity is less than the minimum capacity limit, the available capacity of electric vehicles is zero, then:

[0138] The lower limit of the capacity of the electric vehicle cluster at time t for:

[0139]

[0140] The upper limit of the capacity of the electric vehicle cluster at time t for:

[0141]

[0142] The available capacity E of the electric vehicle cluster at time t ev,all The calculation formula is as follows:

[0143]

[0144] Figure 5The figure shows the change of capacity of an electric vehicle in one day. The electric vehicle assumes SOC min is 0.6, and the time when the electric private car is connected to the grid is t 1 Between 8:00 and 9:00 (assuming t 1 It obeys uniform distribution, that is, t 1 ~U(8,9)), and between 17:30-19:00, or between 21:00-22:30 (the owner goes out for leisure and entertainment after get off work and does not drive home immediately). The moment when the electric private car is disconnected from the grid (i.e., the moment when it is out of the controllable state t 4 ) between 7:30 and 8:30 (assuming t 4 It obeys uniform distribution, that is, t 4 ~U(7.5,8.5)), and between 17:00 and 18:30.

[0145] The capacity calculated according to formula (18) and the a calculated in step S2 ev , thus the power P of the electric vehicle cluster participating in frequency regulation in area k can be obtained ev,k :

[0146] P ev,k =P t,k ·a ev ; (19)

[0147] Among them, P t,k is the total power command output by the deep reinforcement learning controller in region k at time t.

[0148] The present invention adds electric vehicle clusters to simulate the diversity of frequency modulation resources in real power grids, and further verifies the collaborative ability of EL-GAC deep reinforcement learning. In order to effectively solve the problem of accommodating wind power and photovoltaic power with strong randomness, wind power and photovoltaic power are directly integrated into the power grid, and the EL-GAC deep reinforcement learning controller's adaptability and collaborative ability with different generator sets are used to achieve the problem of accommodating new energy.

[0149] Step S4, inputting the regional control deviation of each area into the deep reinforcement learning controller of the corresponding area, iteratively updating through the internal control method of the deep reinforcement learning controller and outputting the total control power of the corresponding area.

[0150] Figure 2 The EL-GAC deep reinforcement learning controller in the invention is an important component of the present invention. It can monitor the frequency and ACE of the power grid, and then output the specific power generation power of the unit to eliminate the impact of large-scale system energy grid connection on the power system, maintain the frequency stability of the power grid, improve the control performance standard (Control Performance Standard, CPS) of the AGC system, and realize the absorption of large-scale new energy.

[0151] The deep reinforcement learning controller is a new generation of artificial intelligence control method based on deep reinforcement learning. The first is to reasonably use expert data to quickly learn the optimal collaborative strategy, with efficient self-update and self-adaptation capabilities. It uses ACE as input and the current state of the system, and the total power instruction of the generator set as the action output to interact with the power system, and then stores the control data obtained from the interaction into the database, and finally uses the corresponding proportion to learn and update the collaborative control strategy. The controller used in the present invention is a continuous type, which replaces the action set of the discrete controller with the action range, and maps the neural network output to [-1,1] through the tanh activation function and then expands it to an appropriate range. The present invention limits the action to [-25,25]MW. ACE is a comprehensive indicator of power deviation and frequency deviation. As long as ACE is maintained within the qualified range, the power grid indicators will also tend to be stable.

[0152] The deep reinforcement learning controller consists of two policy networks and 2 evaluation networks like Figure 6 As shown, the strategy network includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, and an output layer. It is used to output action a under state S, where state S is the regional control deviation of the corresponding area, and action a is to output the total control power deviation of the corresponding area; the mathematical expression of the strategy network is as follows:

[0153]

[0154] Where: y u is the input of the policy network; y 1 ,y 2 are the outputs of the first hidden layer and the second hidden layer respectively; μ and logσ are the outputs of the third hidden layer and the fourth hidden layer respectively; W μ , W logσ are the weight matrices of the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer respectively; b μ 、b logσ is the bias vector of the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer; the output action a follows the normal distribution N(μ,σ), and the mean μ and standard deviation σ are the mean and standard deviation of the normal distribution followed by the output action a respectively; tanh(x)=(e x -e -x ) / (e x +e -x) is the activation function, A is the maximum action value; the policy network will bring the mean and standard deviation of the output into the normal distribution for sampling to obtain the action output.

[0155] like Figure 7 As shown, the evaluation network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. It is used to judge the output value Q of the output action a under state S. The mathematical expression of the evaluation network is as follows:

[0156]

[0157] Among them, x u is the input of the evaluation network, x 1 、x 2 are the outputs of the first hidden layer and the second hidden layer respectively; Q is the output of the output layer; are the weight matrices of the first hidden layer, the second hidden layer, and the output layer respectively; is the bias vector of the first hidden layer, the second hidden layer, and the output layer; relu(x)=max(0,x) is the activation function, which ensures that the output value is not negative.

[0158] Each layer of the evaluation network will have multiple neurons, and the connection of each layer of neurons will be multiplied by a coefficient, such as Figure 3 The connection between the input layer and the first hidden layer, and the matrix composed of these coefficients is the weight matrix. After multiplying the weight matrix, the corresponding bias will be added. These biases can form a bias vector, and then the output is obtained through the activation function. It is connected to the next layer in the same way until the output of value Q is finally obtained.

[0159] like Figure 8 As shown, taking the regional control deviation as input, iteratively updating and outputting the total control power of the region through the internal control method of the deep reinforcement learning controller specifically includes the following steps:

[0160] Step S41, initialize two evaluation networks and 2 policy networks In order to facilitate the stable operation of the algorithm, the maximum action value is set to A = 25; the learning rate α = 0.001; the discount factor γ = 0.99; the entropy regularization coefficient τ 1 =0.2; The size of the experience pool is a key parameter in deep reinforcement learning. It has an important impact on the stability of the learning process, data efficiency, and final performance. Therefore, the environment experience buffer pool B is set 1 and expert experience buffer pool B 2 and its upper limit B 2max , B 2max=200; random actions N = 30; optimal action ratio coefficient ρ = 0.1; expert experience ratio ξ.

[0161] Step S42: input the area control deviation ACE value of the area corresponding to time t and use it as the environmental state S at time t. t .

[0162] Step S43, determine the expert experience pool B 2 Whether the amount of stored information has reached the upper limit B 2max ; If the expert experience pool B 2 The amount of stored information has reached the upper limit B 2max , then through the strategy network at time t Output the environmental state S at time t t The next action a t ; If the expert experience pool B 2 The amount of stored information has not reached the upper limit B 2max , then the environmental state S at time t is selected based on expert experience t Next action

[0163] Step S44, calculate the real-time reward r t ; The present invention considers the reliability and power quality of the new power system, sets the controller's goal to minimize ACE and Δf, and linearly weights ACE and Δf to generate the controller's reward function, which is as follows:

[0164]

[0165] in, is the ratio of the frequency deviation Δf at time t to 0.2, k 1 With k 2 is the weight factor, and the present invention takes 0.5.

[0166] Step S45, storing the environment information in the environment experience buffer pool B 1 , store the expert information in the expert experience buffer pool B 2 , the environmental information includes the environmental state S at the previous moment t-1 , corresponding real-time strategy Action t-1 , the reward r at the previous moment t-1 , the environmental state at this moment S t ; Expert information includes the environment state S at the last moment t-1 , corresponding actions selected by expert experience Rewards from the last moment The environmental state at this moment is S t .

[0167] Step S46: In the expert experience buffer pool B, according to the expert experience proportion ξ 2 With the same probability, randomly select data, and then from the environment experience buffer pool B 1 With the same probability data, which together constitute the training data, where G is the amount of data required for mini-batch reduction, and in this embodiment, G=64.

[0168] Step S47, using the training data to update the parameters of the evaluation network and the policy network, and updating the expert experience ratio ξ; the larger the ξ, the higher the proportion of expert information involved in the update, which can speed up the convergence of the algorithm; the smaller the ξ, the higher the proportion of environmental information involved in the update, which can increase the probability of learning to output the most accurate power command. The expert experience ratio ξ changes with the increase in the number of iterations, and the calculation formula is:

[0169]

[0170] Where: The iteration rate of ξ in table i is set to 200, and ξ decreases by 0.1 every i iterations of the algorithm. When ξ reaches 0.1, ξ stops updating.

[0171] Among them, the evaluation network Network parameters Updated as follows:

[0172]

[0173] θ 2 ←τ 2 ·θ 1 -(1-τ 2 )·θ 2 ; (25)

[0174] Where: α is the learning rate; For the price of θ 1 The gradient of θ 1 Find partial derivative); γ is the discount factor, used to calculate the target value; τ 2 is a constant, specifically a decimal close to 0, used to control the update rate. This method can smoothly update the target network and avoid the target network changing too quickly, which leads to unstable training.

[0175] Policy Network The structure of is the same as that of , as shown in formula (6). It takes the state (ACE) as input, calculates the mean μ and variance σ through three linear layers, then passes through the normal distribution N(μ,σ) and the activation function tanh and multiplies by the maximum action A, and finally outputs the action (power instruction) corresponding to the state. The policy network parameter σ Updates need to be made through According to the status t-1 Sampling action, by Calculate the Q value and sort and filter to obtain the state s t-1 The optimal action set

[0176]

[0177] Where: s t-1 is the state at time t-1, N is the state of the sample network at time s t-1 The total number of actions generated under the optimal action ratio; ρ is the ratio of the optimal action; a 1 to a ρN For the greatest value Actions. Indicates rounding down.

[0178] Then use renew Parameter w 1 and Parameter w 2 During the update process Do not use entropy to avoid policy collapse, and Add a higher level of entropy to encourage exploration. The update formula is as follows:

[0179]

[0180] Where: α is the learning rate; and They are and In status t-1 The probability of output action (power deviation); τ 1 is the regularization parameter, τ 1 The larger the size, the higher the entropy, and the greater the randomness; is the cross entropy, and the calculation formula is as follows:

[0181]

[0182] Step S48, output power action a, calculate the total power instruction for the corresponding area at time t, if B 2 The stored information does not exceed the storage limit:

[0183]

[0184] If B 2 The stored information exceeds the storage limit:

[0185] Pt =P t-1 +a t ; (31)

[0186] Among them, P t is the total power command at time t in the corresponding area, P t-1 is the total power command of the corresponding area at time t-1.

[0187] If the total power command P output by the deep reinforcement learning controller at time t in the calculation area k is t,k , then in B 2 When the stored information does not exceed the storage limit:

[0188]

[0189] In B 2 When the stored information exceeds the storage limit:

[0190] P t,k =P t-1,k +a t,k ; (33)

[0191] Among them, P t-1,k is the total power command output by the deep reinforcement learning controller in region k at time t-1, a t,k is the deep reinforcement learning controller policy network in region k Output of the environmental state S at time t t The next action, The deep reinforcement learning controller in region k selects the environment state S at time t based on expert experience t Next action.

[0192] If the policy network output makes the reward value at a large value and cannot significantly increase the reward value, it means that the policy network training is completed and the algorithm iteration is stopped. If the simulation system stops, the corresponding algorithm will also stop running.

[0193] In the face of new power systems dominated by new energy sources, discrete Q learning and its derived control algorithms have poor adaptability, fail to reasonably store and utilize expert data, and cannot quickly realize system control of each generator set when the power system changes, and the control accuracy is poor. The present invention studies continuous deep reinforcement learning, reasonably stores and utilizes expert data, and can maintain the frequency stability of each region of the distributed power grid under strong random load conditions; improves the ability of distributed power grids to absorb intermittent energy such as wind power and photovoltaics; and improves the coordinated control capability between regions.

[0194] Step S5, obtaining the allocated power of the thermal power units, hydropower units and electric vehicle clusters according to the total control power of the corresponding area in step S2 and the power allocation ratio of the thermal power units, hydropower units and electric vehicle clusters, and then obtaining the power instructions of the thermal power units, hydropower units and electric vehicle clusters in the corresponding area.

[0195] According to the following formula (5), formula (6) and formula (19), the total power instructions of the thermal power unit, hydropower unit and electric vehicle cluster are obtained respectively.

[0196] Step S6, the thermal power unit, the hydropower unit, and the electric vehicle cluster execute corresponding power instructions and interact with the power grid, and then return to step S2 to proceed to the next cycle.

[0197] The following combination Fig.10 Compared with two different methods, the control performance of the proposed controller is superior:

[0198] In the power system, Area Control Error (ACE) is an important indicator that measures the degree of imbalance between power generation and load in a control area. The value of ACE is the difference between actual power and planned power, where actual power is the difference between the total power generation and total load power in the area, and planned power is the power value determined based on the trading plan in the day-ahead market or the real-time market. ACE is crucial to the stable operation of the power system because it directly affects the frequency and voltage levels of the power grid. If the ACE value of an area is large, it means that the power generation and load in the area are unbalanced, which may cause fluctuations in the frequency of the power grid, thereby affecting the stability of the power system. Therefore, the power system operator will adjust the output of the generator set through the automatic generation control (AGC) system to reduce the ACE value and maintain the stability of the power grid.

[0199] The evaluation of regional power grids depends on the CPS index. The controller should try to keep the CPS index and frequency (f / Hz) stable. The specific indicators are as follows:

[0200] 1) If CPS1 ≥ 200% and CPS2 is any value, the CPS index is qualified.

[0201] 2) If 100%≤CPS1<200%, and CPS2≥90%, the CPS index is qualified.

[0202] 3) If CPS1<100%, the CPS index is unqualified.

[0203] 4) If f∈(49.80,50.20)Hz, the frequency index is qualified.

[0204] See also Fig.10(a) In order to simulate the load changes of irregular random disturbances in the actual power grid, the present invention introduces irregular random disturbances with strong disturbance to simulate the real power grid environment, and performs 24h real-time simulation to verify the actual engineering application effect of EL-GAC.

[0205] See also Fig.10 (b) Under irregular random disturbances, the controller proposed based on EL-GAC can always maintain the average value of the absolute value of the regional frequency deviation within 0.0003Hz, the SAC controller is 0.001Hz, and the DQN controller is 0.009Hz. In the actual power grid, the frequency needs to be stabilized at 50Hz, and the frequency deviation is the difference between the actual frequency and 50Hz. The smaller this value is, the better the frequency index is.

[0206] See also Fig.10 (c) Under irregular random disturbances, the controller proposed based on EL-GAC can always maintain the average value of the absolute value of ACE within 1.06MW, the SAC controller is 4.4MW, and the DQN controller is 3.4MW. In actual power grids, the smaller the ACE value, the better the performance.

[0207] See also Fig.10 (d) Under irregular random disturbances, the controller proposed based on EL-GAC can always maintain the 10-min CPS average value above 199.939%, the SAC controller is above 199.984%, and the DQN controller is above 199.977%. In actual power grids, the closer the CPS is to 200%, the better the index is.

[0208] In summary, the present invention uses a deep reinforcement learning controller to ensure that the CPS indicators of multi-regional power grids are qualified and the frequency is stable. Compared with the existing reinforcement learning controllers (DQN, SAC), the deep reinforcement learning controller has a simple structure and self-learning ability. It can use expert data to quickly learn new collaborative control strategies when the power system changes, and maintain good control capabilities.

[0209] The control method of the present invention can achieve qualified CPS indicators and frequency stability of multi-regional power grids, the consumption of large-scale new energy in new power systems, and coordinated control of generator sets through the designed controller.

[0210] Embodiment 2:

[0211] like Fig.11 As shown, based on the same inventive concept as that of Example 1, this embodiment provides an automatic power generation control system for large-scale new energy grid connection, which is applied to the method described, including:

[0212] The model building module is used to build a multi-region interconnected power grid model, initialize model parameters, and integrate the multi-region interconnected power grid model into the power grid; the multi-region interconnected power grid model includes deep reinforcement learning controllers, wind turbines, photovoltaic power stations, thermal power units, hydropower units, electric vehicle clusters, and loads in each region; the deep reinforcement learning controller is connected to the thermal power units, hydropower units, and electric vehicle clusters respectively.

[0213] The power allocation ratio calculation module is used to calculate the power allocation ratio of thermal power units, hydropower units and electric vehicle clusters according to the capacities of thermal power units, hydropower units and electric vehicle clusters in each area.

[0214] The regional control deviation calculation module is used to calculate the real-time frequency deviation of the power grid in each region and the exchange power between adjacent regions based on the output power of wind turbines in each region, the output power of photovoltaic power stations, the output power of thermal power units, the output power of hydropower units, the power of electric vehicle clusters participating in frequency modulation, and the load, and calculate the regional control deviation of each region.

[0215] The deep reinforcement learning controller is used to input the regional control deviation of each area into the deep reinforcement learning controller of the corresponding area, iteratively update through the internal control method of the deep reinforcement learning controller and output the total control power of the corresponding area.

[0216] The total power instruction calculation module is used to obtain the allocated power of the thermal power units, hydropower units and electric vehicle clusters according to the total control power of the corresponding area and the power allocation ratio of the thermal power units, hydropower units and electric vehicle clusters, and then obtain the power instructions of the thermal power units, hydropower units and electric vehicle clusters in the corresponding area.

[0217] The grid interaction module is used for thermal power units, hydropower units, and electric vehicle clusters to execute corresponding power instructions and interact with the grid.

[0218] The specific working principle is described in the above method and will not be repeated here.

[0219] Embodiment 3:

[0220] Based on the same inventive concept as Example 1, this embodiment provides a computer-readable storage medium: the computer-readable storage medium includes a stored program, wherein, when the program is running, the device where the computer-readable storage medium is located is controlled to execute the automatic power generation control method for large-scale new energy grid connection.

[0221] Embodiment 4:

[0222] Based on the same inventive concept as that of Example 1, this embodiment provides a processor, which is used to run a program, wherein the program executes the automatic power generation control method for large-scale new energy grid connection when running.

[0223] Those of ordinary skill in the art will appreciate that the modules of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0224] In the embodiments provided by the present invention, it should be understood that the division of modules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple modules can be combined into one module, one module can be split into multiple modules, or some features can be ignored, etc.

[0225] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0226] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nly Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes. Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An automatic power generation control method for large-scale renewable energy grid connection, characterized in that: The following steps are involved: Build a multi-region interconnected power grid model, initialize model parameters, and integrate the multi-region interconnected power grid model into the power grid; the multi-region interconnected power grid model includes deep reinforcement learning controllers, wind turbines, photovoltaic power stations, thermal power units, hydropower units, electric vehicle clusters, and loads in each region; the deep reinforcement learning controller is connected to the thermal power units, hydropower units, and electric vehicle clusters respectively; Calculate the power allocation ratio of thermal power units, hydropower units and electric vehicle clusters according to the capacities of thermal power units, hydropower units and electric vehicle clusters in each region; According to the output power of wind turbines, photovoltaic power stations, thermal power units, hydropower units, the power of electric vehicle clusters participating in frequency modulation, and loads in each region, the real-time frequency deviation of the power grid in each region and the exchange power between adjacent regions are calculated, and the regional control deviation of each region is calculated; The regional control deviation of each area is input into the deep reinforcement learning controller of the corresponding area, and the internal control method of the deep reinforcement learning controller is used to iteratively update and output the total control power of the corresponding area; The allocated power of the thermal power units, hydropower units and electric vehicle clusters is obtained according to the total control power of the corresponding area and the power allocation ratio of the thermal power units, hydropower units and electric vehicle clusters, and then the power instructions of the thermal power units, hydropower units and electric vehicle clusters in the corresponding area are obtained; Thermal power units, hydropower units, and electric vehicle clusters execute corresponding power instructions and interact with the power grid.

2. The automatic power generation control method for large-scale renewable energy grid connection according to claim 1, characterized in that: The specific calculation method for the power allocation ratio of thermal power units, hydropower units, and electric vehicle clusters is as follows: Where: a f 、a h 、a ev are the power allocation ratios of thermal power units, hydropower units, and electric vehicle clusters, respectively. f +a h +a ev =1 and a f 、a h 、a ev ∈[0,1]; E f 、E h are the rated capacities of thermal power units and hydropower units, E ev,all is the available capacity of the electric vehicle cluster.

3. The automatic power generation control method for large-scale renewable energy grid connection according to claim 1, characterized in that: The calculation method of regional control deviation for each region is as follows: Δf k =L -1 {(P g,k +P pv,k +P h,k +P f,k +P ev,k -P L,k -P tie,kj )×K p / (1+sT p )}; P tie,kj =L -1 {(Δf k -Δf j )×(2πT kj / s)}; ACE k =ΔP tie,kj +b k Δf k ; Where Δf k is the real-time frequency deviation of the power grid in region k, L -1 is the inverse Laplace transform, P g,k is the wind turbine output power in region k, P pv,k is the output power of the photovoltaic power station in area k, P h,k is the output power of the hydropower unit in area k, P f,k is the output power of thermal power units in region k, P ev,k is the power of the electric vehicle cluster in area k when participating in frequency regulation, P L,k is the load of area k, P tie,kj is the interconnection line exchange power between area k and area j, where area k is adjacent to area j, K p is the frequency response function coefficient, T p is the time constant of the frequency response function, s is the Laplace operator, Δf j is the real-time frequency deviation of the power grid in region j, T kj is the time constant of the tie line between area k and area j; ACE k is the regional control deviation of region k, β k is the frequency deviation coefficient of region k.

4. The automatic power generation control method for large-scale renewable energy grid connection according to claim 1, characterized in that: The power P of the electric vehicle cluster participating in frequency regulation in area k ev,k The details are as follows: Build a dynamic change model of the number of controllable electric vehicles, as follows: Assume that the starting time is t0, the time when the electric vehicle in area k is connected to the grid is t1, and the charging time of the electric vehicle is T2. Thus, the time t3 when the electric vehicle enters the operation frequency modulation is: t3 = t1 + T2; The electric vehicle determines whether it can participate in the frequency modulation service based on its own state of charge. The calculation of the electric vehicle's own state of charge is as follows; Where: SOC0 is the current state of charge of a single electric vehicle; E ev is the existing capacity of a single electric vehicle; E ev,max is the maximum capacity of a single electric vehicle; The charging time T2 of electric vehicles is calculated by the formula: in, is the charging power of a single electric vehicle; η ev Charging efficiency of electric vehicles; SOC min The minimum state of charge required for electric vehicles to participate in frequency regulation; The Monte Carlo method is used to randomly sample the electric vehicle entry / exit controllable timetable for statistical analysis to obtain the number of electric vehicles in the controllable state N at each time in the area. in '(t i ) and the number of electric vehicles in controllable state N out '(t i ), the cumulative number of electric vehicles N that enter the controllable state at time t is calculated by the following formulas: in (t) and the cumulative number of electric vehicles N that are out of controllable state out (t): Assume that the initial number of controllable electric vehicles at time t0 is N0, and the number of controllable electric vehicles at time t is N c (t) is shown as follows: N c (t) = N0 + N in (t)-N out (t); So the capacity of the electric vehicle cluster available at time t is for: When participating in frequency regulation, the total available capacity of electric vehicles needs to be subject to certain restrictions. When the existing capacity is less than the minimum capacity limit, the available capacity of electric vehicles is zero, then: The lower limit of the capacity of the electric vehicle cluster at time t for: The upper limit of the capacity of the electric vehicle cluster at time t for: The available capacity E of the electric vehicle cluster at time t ev,all The calculation formula is as follows: Calculate a from the existing capacity ev , the power P of the electric vehicle cluster participating in frequency regulation in area k ev,k The calculation method is as follows: P ev,k =P t,k ·a ev ; Among them, P t,k is the total power command output by the deep reinforcement learning controller in region k at time t.

5. The automatic power generation control method for large-scale renewable energy grid connection according to claim 1, characterized in that: The deep reinforcement learning controller consists of two policy networks and 2 evaluation networks The strategy network includes an input layer, a first hidden layer, a second hidden layer, a third hidden layer, a fourth hidden layer, and an output layer. It is used to output action a under state S, where state S is the regional control deviation of the corresponding area, and action a is to output the total control power deviation of the corresponding area; the mathematical expression of the strategy network is as follows: Where: y u is the input of the strategy network; y1 and y2 are the outputs of the first hidden layer and the second hidden layer respectively; μ and logσ are the outputs of the third hidden layer and the fourth hidden layer respectively; W1 μ , W μ , W logσ are the weight matrices of the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer respectively; b μ 、b logσ is the bias vector of the first hidden layer, the second hidden layer, the third hidden layer, and the fourth hidden layer; the output action a follows the normal distribution N(μ,σ), and the mean μ and standard deviation σ are the mean and standard deviation of the normal distribution followed by the output action a respectively; tanh(x)=(e x -e -x ) / (e x +e -x ) is the activation function, A is the maximum action value; The evaluation network includes an input layer, a first hidden layer, a second hidden layer, and an output layer. It is used to judge the output value Q of the output action a under state S. The mathematical expression of the evaluation network is as follows: Among them, x u is the input of the evaluation network, x1 and x2 are the outputs of the first hidden layer and the second hidden layer respectively; Q is the output of the output layer; W1 Q , are the weight matrices of the first hidden layer, the second hidden layer, and the output layer respectively; is the bias vector of the first hidden layer, the second hidden layer, and the output layer; relu(x)=max(0,x) is the activation function, which ensures that the output value is not negative.

6. The automatic power generation control method for large-scale renewable energy grid connection according to claim 1, characterized in that: Inputting the regional control deviation of each area into the deep reinforcement learning controller of the corresponding area, iteratively updating through the internal control method of the deep reinforcement learning controller and outputting the total control power of the corresponding area specifically includes the following steps: Initialize 2 evaluation networks and 2 policy networks Set the maximum action value, learning rate α, discount factor γ, entropy regularization coefficient τ1, environment experience buffer pool B1 and expert experience buffer pool B2 and their upper limit B 2max , random action N, optimal action proportion coefficient ρ, expert experience proportion ξ; Input the regional control deviation ACE value of the corresponding area at time t and use it as the environmental state S at time t t ; Determine whether the amount of information stored in the expert experience pool B2 has reached the upper limit B 2max ; If the amount of information stored in the expert experience pool B2 reaches the upper limit B 2max , then through the strategy network at time t Output the environmental state S at time t t The next action a t ; If the amount of information stored in the expert experience pool B2 does not reach the upper limit B 2max , then the environmental state S at time t is selected based on expert experience t Next action Calculate real-time reward r t ; The environmental information is stored in the environmental experience buffer pool B1, and the expert information is stored in the expert experience buffer pool B2. The environmental information includes the environmental state S at the previous moment. t-1 , corresponding real-time strategy Action t-1 , the reward r at the previous moment t-1 , the environmental state at this moment S t ; Expert information includes the environment state S at the last moment t-1 , corresponding actions selected by expert experience Rewards from the last moment The environmental state at this moment is S t ; According to the expert experience proportion ξ, randomly select from the expert experience buffer pool B2 with the same probability data, and then extract them from the environmental experience buffer pool B1 with the same probability data, and the two together constitute the training data, where G is the number of training samples set for the small batch size reduction; Use the training data to update the parameters of the evaluation network and the strategy network, and update the expert experience ratio ξ; Output power action a, calculate the total power command at time t in the corresponding area.

7. The automatic power generation control method for large-scale new energy grid connection according to claim 6, characterized in that: The total power command for the corresponding area at time t is calculated as follows: If the information stored in the expert experience buffer pool B2 does not exceed the storage limit: If the information stored in the expert experience buffer pool B2 exceeds the storage limit: P t =P t-1 +a t ; Among them, P t is the total power command at time t in the corresponding area, P t-1 is the total power command of the corresponding area at time t-1.

8. An automatic power generation control system for large-scale renewable energy grid connection, characterized in that: The method applied to any one of claims 1 to 7, comprising: A model building module is used to build a multi-region interconnected power grid model, initialize model parameters, and integrate the multi-region interconnected power grid model into the power grid; the multi-region interconnected power grid model includes a deep reinforcement learning controller, a wind turbine, a photovoltaic power station, a thermal power unit, a hydropower unit, an electric vehicle cluster, and a load in each region; the deep reinforcement learning controller is connected to the thermal power unit, the hydropower unit, and the electric vehicle cluster respectively; A power allocation ratio calculation module is used to calculate the power allocation ratio of thermal power units, hydropower units and electric vehicle clusters according to the capacities of thermal power units, hydropower units and electric vehicle clusters in each area; The regional control deviation calculation module is used to calculate the real-time frequency deviation of the power grid in each region and the exchange power between adjacent regions based on the output power of wind turbines in each region, the output power of photovoltaic power stations, the output power of thermal power units, the output power of hydropower units, the power of electric vehicle clusters participating in frequency modulation, and the load, and calculate the regional control deviation of each region; A deep reinforcement learning controller, used for inputting the regional control deviation of each area into the deep reinforcement learning controller of the corresponding area, iteratively updating through the internal control method of the deep reinforcement learning controller and outputting the total control power of the corresponding area; A total power command calculation module is used to obtain the allocated power of the thermal power unit, the hydropower unit, and the electric vehicle cluster according to the total control power of the corresponding area and the power allocation ratio of the thermal power unit, the hydropower unit, and the electric vehicle cluster, and then obtain the power command of the thermal power unit, the hydropower unit, and the electric vehicle cluster in the corresponding area; The grid interaction module is used for thermal power units, hydropower units, and electric vehicle clusters to execute corresponding power instructions and interact with the grid.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the automatic power generation control method for large-scale new energy grid connection as described in any one of claims 1 to 7.

10. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the automatic power generation control method for large-scale new energy grid connection as described in any one of claims 1 to 7.

Images