A WADC design method for wind power generation system based on reinforcement learning
By designing a WADC control method based on reinforcement learning in a wind power system, using a double-feed wind turbine and reinforcement learning controller, the impact of wind power output power fluctuations on grid stability is solved, and the suppression of low-frequency oscillation and grid stability are achieved. The improvement has improved the economic benefits of wind power enterprises.
Patent Information
- Application Number
- CN202210808271.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-11
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-07-11
AI Technical Summary
When large-scale wind power is connected to the grid, the output power fluctuations of wind power generation affect the stability of the power grid, resulting in low-frequency oscillation. The existing PSS control is difficult to effectively suppress, affecting the safe and stable operation of the power grid.
Design a WADC control method for wind power generation system based on reinforcement learning, adopting a double-feed wind turbine and a WADC controller based on reinforcement learning, including reinforcement learning control, voltage outer ring PI control and current inner ring PI control, through the Actor network and Critic Network updates the weights online to achieve intelligent control of wind turbines.
Effectively suppress low-frequency oscillations of the power system, improve the safety and stability of the power grid, enable the power grid to absorb the electricity emitted by wind power plants in a timely manner, and improve the economic and social benefits of wind power generation enterprises.
Smart Images

Figure CN115276086B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of large-scale wind power grid-connected power generation system control, and in particular to a wind power generation system WADC design method based on reinforcement learning. Background Art
[0002] Wind power generation has the characteristics of being safe, clean, inexhaustible, with a short infrastructure cycle and being environmentally friendly, and has become one of the important sources of renewable energy. Large-scale development and utilization of wind power generation has become the focus of governments and energy companies around the world. With the continuous increase in the penetration rate of wind power generation, wind power generation has brought new challenges to the safety and stability of traditional power systems. Because wind power is intermittent and random, the output power of wind power generation fluctuates due to the influence of wind speed. The transmission of fluctuating wind power to the power grid will have an adverse effect on the stable operation of the power grid. Therefore, how to improve and enhance the stability and safety of wind power grid connection and make the operation of wind power system more stable has become the key to large-scale wind power grid connection.
[0003] If the damping of the power system is close to zero or negative, when the power system is disturbed, the power system will experience low frequency oscillation (LFO) with a frequency of 0.2-2.5Hz. Once the power system has low frequency oscillation, if the system cannot be effectively controlled in time, it may cause relay overcurrent tripping, transformer damage, and even loss of step between units. Therefore, reliably and effectively suppressing the LFO of the power system is of strategic significance for the safe and stable operation of the power grid. Summary of the invention
[0004] In order to solve the problem that conventional PSS control is difficult to adapt to the increasing penetration rate of renewable energy represented by wind power generation, resulting in large fluctuations in voltage and power at the common node between wind power and the power grid, the present invention proposes a WADC design method for a wind power generation system based on reinforcement learning.
[0005] The present application provides a wind power generation system WADC (Widearea damping controller, WADC) design method based on reinforcement learning, comprising the following steps:
[0006] S101: Construct a wind power generation WADC control system based on reinforcement learning; the WADC control system includes: a doubly-fed wind turbine and a WADC controller based on reinforcement learning;
[0007] The doubly-fed wind turbine includes: wind turbine, gearbox, doubly-fed induction generator DFIG, transformer, rotor-side inverter, grid-side inverter and overvoltage protection circuit CrowBar;
[0008] The wind turbine is connected to the gearbox through mechanical transmission; the gearbox is connected to the doubly-fed induction generator through a transmission bearing; the doubly-fed induction generator DFIG is connected to the transformer through electromagnetic coupling and is connected to the AC power grid through the transformer; the doubly-fed induction generator DFIG is electrically connected to the output end of the overvoltage protection circuit CrowBar and the input end of the rotor-side inverter; the output end of the rotor-side inverter is electrically connected to the input end of the grid-side inverter; the output end of the grid-side inverter is electrically connected to one end of the transformer;
[0009] S102: Controlling the doubly-fed wind turbine generator set using a WADC controller based on reinforcement learning;
[0010] The WADC controller based on reinforcement learning includes three control parts: reinforcement learning control, voltage outer loop PI control and current inner loop PI control;
[0011] Reinforcement learning control is used to control the double-fed induction generator DFIG. Its input signal is the angular velocity deviation signal of the double-fed induction generator DFIG, and its output signal is connected to the voltage outer loop PI control;
[0012] The voltage outer loop PI control is used to control the inverter on the grid side;
[0013] The current inner loop PI control is used to control the rotor-side inverter to output the specified active and reactive power, thereby suppressing the power oscillation at the wind turbine grid connection point.
[0014] Furthermore, the reinforcement learning control includes: a state converter, an Actor network and a Critic network; the principle of their individual control is:
[0015] According to the actual situation, the pre-given signal w(t) is subtracted from the output y(t) of the controlled photovoltaic power generation system to generate an error signal e(t); the error signal e(t) is converted into the input state signal x(t) of the reinforcement learning network through the state converter; the state signal x(t) is input to the Actor network to obtain the output signal u n (t); the state signal x(t) and the error reinforcement learning signal r(t) are input into the Critic network together to obtain the output signal n(t); the output signal u n (t) is combined with n(t) to obtain the control input signal u(t) of the controlled photovoltaic power generation system; u(t) acts on the controlled photovoltaic power generation system to obtain the output signal y(t) to form a closed-loop control; the Actor network and the Critic network also use the time difference signal δ TD (t) Update the weight coefficients of the Actor network and the Critic network online.
[0016] Furthermore, the RBF neural network is used to simultaneously complete the strategy function of the Actor network and the value function of the Critic network; the input of the RBF neural network is the state signal x(t)=[x 1 (t),x 2 (t)…,x n (t)] T ;
[0017] , the Gaussian kernel function used in its hidden layer is:
[0018]
[0019] Among them, μ j =[μ j1 ,μ j2 ,μ j3 ] is the center vector of the jth node, σ j is the width function of the jth node, and m is the number of hidden layer nodes.
[0020] Furthermore, the outputs of the Actor network and the Critic network are respectively:
[0021]
[0022]
[0023] where ω nj and ω j They are the weight coefficients of the Actor network and the Critic network respectively.
[0024] Furthermore, the expression of the control input signal u(t) is as follows:
[0025] u(t)=u n (t)+η m (0,ρ(t)) (3)
[0026] Among them, η m () is the Gaussian distribution function; ρ(t)=[1+exp(2n(t))] -1 .
[0027] Furthermore, the timing difference signal δ TD The expression of (t) is as follows:
[0028] δ TD (t) = r(t) + λn(t+1) - n(t) (4)
[0029] Where λ is the discount coefficient, 0<λ<1, and the performance index function is:
[0030] Furthermore, the weight coefficient update formula of the Actor network and the Critic network is:
[0031]
[0032] ω j (t+1)=ω j (t)+η c δ TD h(t) (6)
[0033] Among them, η a and η c is the learning rate of the Actor network and the Critic network.
[0034] Furthermore, the update formulas of the hidden layer node center function c(t) and width function σ(t) are:
[0035]
[0036]
[0037] Among them, β 1 With β 2 is the learning rate.
[0038] Furthermore, the process of reinforcement learning control is as follows:
[0039] S201: Initialize the learning rate η of the RBF network a and ηc, discount factor λ and network weight coefficient ω nj and ω j , total number of iterations N c , iteration time T and error threshold δ 0 ;
[0040] S202: Enter the network loop iteration process, and set the current iteration number i=1;
[0041] S203: Judgment i <N c Is it established? If so, proceed to step S204; otherwise, proceed to step S206;
[0042] S204: Based on x(t) at time t, solve the output u of the Actor and Critic networks at time t according to formulas (1) and (2) n (t) and n(t);
[0043] S205: Calculate the timing difference signal δ according to formula (4) TD The value of , if ,δ 0 >δ TDThen update the weights of the Actor network and the Critic network according to equations (5) and (6); calculate the center function and width function according to equations (7) and (8);
[0044] S206: Obtain the vector x(t+1) at time t+1;
[0045] S207: Calculate the control input signal according to formula (3) and apply it to the controlled system;
[0046] S208: Update the system state vector;
[0047] S209: Determine whether the iteration time T has been reached. If so, the reinforcement learning process ends, otherwise return to step S203.
[0048] Compared with the prior art, the beneficial effects of the present invention include: being able to timely and effectively suppress low-frequency oscillations of the power system, on the one hand improving the safety and stability of the power grid, and on the other hand enabling the power grid to timely and massively absorb the electric energy generated by wind power plants, thereby improving the economic and social benefits of wind power generation enterprises. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 It is a schematic flow diagram of the method of the present invention;
[0050] Figure 2 This is a schematic diagram of the structure of the wind power WADC control system based on reinforcement learning;
[0051] Figure 3 It is a diagram of the control principle of reinforcement learning;
[0052] Figure 4 It is a flowchart of reinforcement learning control;
[0053] Figure 5 It is the basic control block diagram of the rotor-side inverter;
[0054] Figure 6 It is the structure diagram of the inverter on the grid side;
[0055] Figure 7 It is the control block diagram of the inverter on the grid side;
[0056] Figure 8 This is a schematic diagram of the control principle of the WADC controller based on reinforcement learning;
[0057] Fig. 9 It is the principle structure diagram of the 4-machine-11-node power generation system including wind power generation;
[0058] Fig.10 is the relative angular velocity response curve of generators G1 and G3 when a 0.1pu voltage signal disturbance is applied;
[0059] Fig.11 It is the relative power angle response curve of generators G1 and G3 when a 0.1pu voltage signal disturbance is applied;
[0060] Fig.12 is the active power transmission response curve of the tie line 78 when a 0.1 pu voltage signal disturbance is applied;
[0061] Fig.13 Relative angular velocity response curve of generators G1 and G3 in case of three-phase short circuit fault;
[0062] Fig.14 This is the relative power angle response curve of generators G1 and G3 when there is a three-phase short circuit fault;
[0063] Fig.15 It is the active power transmission response curve of the tie line 78 when a three-phase short circuit fault occurs. DETAILED DESCRIPTION
[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0065] Please refer to Figure 1 , Figure 1 It is a schematic flow diagram of the method of the present invention;
[0066] The present invention provides a method for designing an oscillation damping controller for a wind power generation system based on reinforcement learning. The method of the present invention comprises the following steps:
[0067] S101: Construct a wind power generation WADC control system based on reinforcement learning; the WADC control system includes: a doubly-fed wind turbine and a WADC controller based on reinforcement learning;
[0068] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of the wind power WADC control system based on reinforcement learning;
[0069] The doubly-fed wind turbine includes: wind turbine, gearbox, doubly-fed induction generator DFIG, transformer, rotor-side inverter, grid-side inverter and overvoltage protection circuit CrowBar;
[0070] The wind turbine is connected to the gearbox through mechanical transmission; the gearbox is connected to the doubly-fed induction generator through a transmission bearing; the doubly-fed induction generator DFIG is connected to the transformer through electromagnetic coupling and is connected to the AC power grid through the transformer; the doubly-fed induction generator DFIG is electrically connected to the output end of the overvoltage protection circuit CrowBar and the input end of the rotor-side inverter; the output end of the rotor-side inverter is electrically connected to the input end of the grid-side inverter; the output end of the grid-side inverter is electrically connected to one end of the transformer;
[0071] In the structure diagram of the doubly-fed wind turbine, the following models are mainly included, namely the wind turbine model and the doubly-fed induction generator model;
[0072] It should be noted that the wind turbine model mainly completes the conversion from wind energy to mechanical energy, with wind speed, pitch angle and per unit value of generator mechanical speed as input, and the output is the mechanical torque acting on the induction motor rotor.
[0073] According to aerodynamics, the characteristics of wind turbines capturing wind energy can be expressed by the following simplified wind turbine output mechanical power:
[0074]
[0075] In the formula,
[0076] 1)P w is the mechanical power output of the wind turbine (W), R is the blade radius (m), ρ is the air density (kg / m3), v w is the equivalent wind speed (m / s);
[0077] 2) C p is the wind energy utilization coefficient, which is related to λ and β, λ is the tip speed ratio, and β is the blade pitch angle (°).
[0078] ①Tip speed ratio
[0079] Defined as the ratio of the linear velocity at the tip of the wind turbine blade to the wind speed
[0080]
[0081] In the formula, ω m is the mechanical speed of the wind turbine.
[0082]
[0083] In the formula, ω r is the per unit value (pu) of the generator mechanical speed, p is the number of generator pole pairs (pairs), and GR is the gear ratio.
[0084] ②Blade pitch angle
[0085] Refers to the angle between the wind turbine blades and the rotor plane. The smaller the pitch angle, the larger the windward surface of the blade, and therefore the greater the wind energy captured.
[0086] Considering the mechanical efficiency of the wind turbine gearbox, the actual output power of the wind turbine is
[0087] P w_out =η·P w (1.4)
[0088] Where η is the gearbox efficiency.
[0089] The output mechanical torque can be obtained as follows:
[0090]
[0091] It should be noted that the doubly fed induction generator model is as follows:
[0092] The mathematical model of the doubly-fed generator consists of the voltage equation, the flux equation and the torque power equation. According to the motor convention, in the synchronous rotating coordinate system, the stator-rotor voltage equation is:
[0093]
[0094] Where p is the differential operator d / dt, ψ represents the flux, subscripts s and r represent the stator and rotor variables respectively, and subscripts d and q represent the d-axis and q-axis variables in the synchronous coordinate system respectively, the same below. All variables including time are per unit values, where the reference value of time t is 1 / ω b , that is, press ω b The angular frequency is the time it takes to select 1 rad. ωb is the reference angular frequency. For a 50Hz system, ω b =2*πf.
[0095] The normalized stator active and reactive power equations are:
[0096]
[0097] Orient the d-axis of the synchronous rotating coordinate system on the stator flux axis, and the stator and rotor flux equations are:
[0098]
[0099] Ignoring the stator resistance (Rs=0) and the stator flux transient (pψd,q=0), the stator voltage is simplified to:
[0100]
[0101] Where U s is the stator phase voltage amplitude, ω 1 is the angular velocity of the stator electrical quantity.sd =U s / ω 1 Substitute the sum into the power equation and record the stator output power as P out and Q out , that is, P sout =-P s , Q sout =-Q s get:
[0102]
[0103] Therefore, it can be seen from the formula that P sout ,Q sout Can be obtained by rq ,i rd Decoupling control. rq ,i rd The control is ultimately through u rd ,u rq Achieved. rq ,i rd with u rd ,u rq The relationship is derived as follows:
[0104] From the stator flux equation:
[0105]
[0106] I will rq ,i rd Substituting the stator current equation represented by into the rotor flux equation, we get
[0107]
[0108] Where: Substituting the formula into the rotor voltage equation in the formula, we can get i rq ,i rd with u rd ,u rq Relationship
[0109]
[0110] where ω slip =ω 1 -ω r , is the per unit value of the angular velocity deviation signal.
[0111] S102: Controlling the doubly-fed wind turbine generator set using a WADC controller based on reinforcement learning;
[0112] The WADC controller based on reinforcement learning includes three control parts: reinforcement learning control, voltage outer loop PI control and current inner loop PI control;
[0113] Reinforcement learning control is used to control the double-fed induction generator DFIG. Its input signal is the angular velocity deviation signal of the double-fed induction generator DFIG, and its output signal is connected to the voltage outer loop PI control;
[0114] The voltage outer loop PI control is used to control the inverter on the grid side;
[0115] The current inner loop PI control is used to control the rotor-side inverter to output the specified active and reactive power, thereby suppressing the power oscillation at the wind turbine grid connection point.
[0116] Please refer to Figure 3 , Figure 3 It is a schematic diagram of the reinforcement learning control principle; the reinforcement learning control includes: a state converter, an Actor network and a Critic network; the principle of its individual control is:
[0117] According to the actual situation, the pre-given signal w(t) is subtracted from the output y(t) of the controlled photovoltaic power generation system to generate an error signal e(t); the error signal e(t) is converted into the input state signal x(t) of the reinforcement learning network through the state converter; the state signal x(t) is input to the Actor network to obtain the output signal u n (t); the state signal x(t) and the error reinforcement learning signal r(t) are input into the Critic network together to obtain the output signal n(t); the output signal u n (t) is combined with n(t) to obtain the control input signal u(t) of the controlled photovoltaic power generation system; u(t) acts on the controlled photovoltaic power generation system to obtain the output signal y(t) to form a closed-loop control; the Actor network and the Critic network also use the time difference signal δ TD (t) Update the weight coefficients of the Actor network and the Critic network online.
[0118] The RBF neural network is used to simultaneously complete the strategy function of the Actor network and the value function of the Critic network; the input of the RBF neural network is the state signal x(t)=[x 1 (t),x 2 (t)…,x n (t)] T ;
[0119] , the Gaussian kernel function used in its hidden layer is:
[0120]
[0121] Among them, μ j =[μ j1 ,μ j2 ,μ j3] is the center vector of the jth node, σ j is the width function of the jth node, and m is the number of hidden layer nodes.
[0122] The outputs of the Actor network and Critic network are:
[0123]
[0124]
[0125] where ω nj and ω j They are the weight coefficients of the Actor network and the Critic network respectively.
[0126] The expression of the control input signal u(t) is as follows:
[0127] u(t)=u n (t)+η m (0,ρ(t)) (3)
[0128] Among them, η m () is the Gaussian distribution function; ρ(t)=[1+exp(2n(t))] -1 .
[0129] The timing differential signal δ TD The expression of (t) is as follows:
[0130] δ TD (t) = r(t) + λn(t+1) - n(t) (4)
[0131] Where λ is the discount coefficient, 0<λ<1, and the performance index function is:
[0132] The weight coefficient update formula of the Actor network and Critic network is:
[0133]
[0134] ω j (t+1)=ω j (t)+η c δ TD h(t) (6)
[0135] Among them, η a and η c is the learning rate of the Actor network and the Critic network.
[0136] The update formulas of the hidden layer node center function c(t) and width function σ(t) are:
[0137]
[0138]
[0139] Among them, β 1 With β 2 is the learning rate.
[0140] Please refer to Figure 4 , the specific process of reinforcement learning control is:
[0141] S201: Initialize the learning rate η of the RBF network a and η c , discount factor λ and network weight coefficient ω nj and ω j , total number of iterations N c , iteration time T and error threshold δ 0 As an embodiment, the number of hidden layer neurons of the radial basis function neural network (RadicalBasis Function, RBF) is 6, and the Actor network learning rate η a is 0.02 and the Critic network learning rate ηc is 0.02, the initial value of the hidden layer node center function c(t) is [1 1 1 1 1], the initial value of the center function c(t) is [1 1 1 1], and the learning rate β when the center function c(t) is updated 1 is 0.01, and the learning rate β when the width function σ(t) is updated 2 0.01; δ 0 =0.03, ε=0.02;
[0142] S202: Enter the network loop iteration process, and set the current iteration number i=1;
[0143] S203: Judgment i <N c Is it established? If so, proceed to step S204; otherwise, proceed to step S206;
[0144] S204: Based on x(t) at time t, solve the output u of the Actor and Critic networks at time t according to formulas (1) and (2) n (t) and n(t);
[0145] S205: Calculate the timing difference signal δ according to formula (4) TD The value of , if ,δ 0 >δ TD Then update the weights of the Actor network and the Critic network according to equations (5) and (6); calculate the center function and width function according to equations (7) and (8);
[0146] S206: Obtain the vector x(t+1) at time t+1;
[0147] S207: Calculate the control input signal according to formula (3) and apply it to the controlled system;
[0148] S208: Update the system state vector;
[0149] S209: Determine whether the iteration time T has been reached. If so, the reinforcement learning process ends, otherwise return to step S203.
[0150] The voltage outer loop PI control and the current inner loop PI control are used to control the rotor-side inverter to output the specified active and reactive power, thereby suppressing the power oscillation at the wind turbine grid connection point.
[0151] Regarding the control of the rotor-side inverter, the following is a detailed description:
[0152] The DFIG rotor winding is fed into the grid through a power electronic inverter that can control the rotor slip ring voltage. Appropriate decoupling control methods can separately control the active and reactive power generated by the DFIG.
[0153] From the model formula of the double-fed induction generator DFIG recorded above, it can be seen that by changing i qr ,i dr The output power P of the doubly-fed induction generator can be controlled separately sout ,Q sout , while i qr ,i dr The control is ultimately controlled by u qr ,u dr The control block diagram of the rotor-side inverter is shown in Figure 1. Figure 5 It should be noted that the current inner loop PI control is a necessary control method for this application, while the power outer loop PI control can be performed using common PI control methods, which will not be elaborated in detail in this application;
[0154] Regarding the inverter control on the grid side, this application is elaborated in detail as follows:
[0155] The grid-side inverter control is used to maintain the DC link capacitor voltage at a preset constant value, regardless of the direction and size of the rotor power; and to control the reactive power generated by the wind turbine to a set reference value based on the reactive power requirements of the entire wind turbine.
[0156] The positive direction of each electrical quantity of the grid-side inverter is specified as follows: Figure 6 As shown. dc is the DC bus voltage, i dcr is the DC current flowing to the rotor-side inverter, idcg is the DC current output from the grid-side inverter, L g is the filter inductor, R g is the resistance of the filter inductor.
[0157] The voltage equation under any dq rotating coordinates is
[0158]
[0159] where ω c is the grid voltage angular velocity, and all variables including time t are per unit values.
[0160] When the d-axis is oriented to the grid voltage vector, u gq =0, the active and reactive power injected by the grid into the grid-side inverter is:
[0161]
[0162] Therefore, by controlling i gd ,i gq The power P that can be output to the grid g , Q g decoupling control.
[0163] According to the instantaneous power theory, reactive power is only transmitted between the three phases, and the only thing that affects the DC voltage is active power, so it can be measured through the active current i gd Controls the DC voltage.
[0164] Adopting the banner value ABC to dq transformation, under the nominal value, ignoring the DC side loss, P g =u dc i dcg =3 / 2u gd i gd , Substitute into the relationship between AC side voltage and DC voltage (m is the modulation ratio):
[0165]
[0166] get The DC circuit
[0167] get
[0168] will i dcr If it is regarded as a disturbance, the transfer function of the DC voltage controlled by the output active current can be expressed as:
[0169]
[0170] Similar to the control of the rotor-side inverter, the control of the grid-side inverter is ultimately achieved by controlling its voltage, so it is necessary to establish the relationship between the inverter AC voltage and current. According to the formula, the inverter AC side voltage u gcd ,u gcq The expression is:
[0171]
[0172] The control of the inverter on the grid side is as follows Figure 7 shown.
[0173] Please refer to Figure 8 , Figure 8 It is a schematic diagram of the control principle of the WADC controller based on reinforcement learning.
[0174] Among them, I dref It is the d-axis reference value of the inner loop current generated by the active power outer loop, which can control the wind power output active power.
[0175] Output inner loop current q-axis reference value I of reactive power PI control qref , can adjust the wind output reactive power.
[0176] like Figure 8 As shown in FIG. 1 , the wide-area damping controller of the wind power generation system is designed in the voltage outer loop, that is, the wide-area damping control signal is added to the voltage outer loop, thereby adjusting the inner loop current q-axis reference value I qref And the output of wind reactive power can achieve the purpose of suppressing the power oscillation of the wind turbine grid access point.
[0177] As an example, refer to Fig. 9 , Fig. 9 This is the principle structure diagram of a 4-machine-11-node power generation system including wind power generation.
[0178] The 4-machine-11 node power generation system includes 4 generators, of which Generator No. 1 is an equivalent machine, 11 nodes, 4 transformers, and 12 lines. The system base power is 100MVA and the base voltage is 345kV. Please refer to Table 1, which is the electrical parameters of the 4-machine-11 node power generation system branch;
[0179] Table 1 Electrical parameters of the 4-machine-11 node power generation system branch
[0180]
[0181]
[0182] Note: A transformer ratio of 0 indicates that the branch is a transmission line, and a ratio other than 0 indicates a transformer branch.
[0183] Taking the 4-machine-11 node power system with wind power generation as an example, a voltage signal disturbance of 0.1 per unit value is applied to the end of the generator G1. Fig.10 , Fig.11 and Fig.12 is the simulation result of the system response. When a three-phase short circuit fault occurs near 8 between tie lines 7 and 8, Fig.13 , Fig.14 and Fig.15 is the system response curve.
[0184] From the simulation experiment, it can be seen that the use of WADC control based on reinforcement learning can effectively suppress the low-frequency oscillations generated when the power system is disturbed.
[0185] The beneficial effects of the present invention are: it can timely and effectively suppress the low-frequency oscillation of the power system, on the one hand improving the safety and stability of the power grid, and on the other hand enabling the power grid to timely and large-scale absorb the electric energy generated by the wind power plant, thereby improving the economic and social benefits of wind power generation enterprises.
[0186] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0187] The specific implementation of the present invention described above does not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A wind power generation system WADC design method based on reinforcement learning, characterized by: S101: Construct a wind power generation WADC control system based on reinforcement learning; The WADC control system includes: a doubly-fed wind turbine and a WADC controller based on reinforcement learning; The doubly-fed wind turbine includes: wind turbine, gearbox, doubly-fed induction generator DFIG, transformer, rotor-side inverter, grid-side inverter and overvoltage protection circuit CrowBar; The wind turbine is connected to the gearbox through mechanical transmission; the gearbox is connected to the doubly-fed induction generator through a transmission bearing; the doubly-fed induction generator DFIG is connected to the transformer through electromagnetic coupling and is connected to the AC power grid through the transformer; the doubly-fed induction generator DFIG is electrically connected to the output end of the overvoltage protection circuit CrowBar and the input end of the rotor-side inverter; the output end of the rotor-side inverter is electrically connected to the input end of the grid-side inverter; the output end of the grid-side inverter is electrically connected to one end of the transformer; S102: Controlling the doubly-fed wind turbine generator set using a WADC controller based on reinforcement learning; The WADC controller based on reinforcement learning includes three control parts: reinforcement learning control, voltage outer loop PI control and current inner loop PI control; Reinforcement learning control is used to control the double-fed induction generator DFIG. Its input signal is the angular velocity deviation signal of the double-fed induction generator DFIG, and its output signal is connected to the voltage outer loop PI control; The voltage outer loop PI control is used to control the inverter on the grid side; The current inner loop PI control is used to control the rotor-side inverter to output the specified active and reactive power, and to suppress the power oscillation at the wind turbine grid connection point; The reinforcement learning control includes: a state converter, an actor network and a critic network; the principle of individual control is: According to the actual situation, the given signal w(t) is subtracted from the output y(t) to generate an error signal e(t); the error signal e(t) is converted into the input state signal x(t) of the reinforcement learning network through the state converter; the state signal x(t) is input to the Actor network to obtain the output signal u n (t); the state signal x(t) and the error reinforcement learning signal r(t) are input into the Critic network together to obtain the output signal n(t); the output signal u n (t) is combined with n(t) to obtain the control input signal u(t) of the controlled photovoltaic power generation system; u(t) acts on the controlled photovoltaic power generation system to obtain the output signal y(t) to form a closed-loop control; the Actor network and the Critic network also use the time difference signal δ TD (t) Online update of the weight coefficients of the Actor network and the Critic network; The RBF neural network is used to simultaneously complete the strategy function of the Actor network and the value function of the Critic network; the input of the RBF neural network is the state signal x(t) = [x1(t), x2(t)L, x n (t)] T , the Gaussian kernel function used in its hidden layer is: Among them, μ j =[μ j1 ,μ j2 ,μ j3 ] is the center vector of the jth node, σ j is the width function of the jth node, and m is the number of hidden layer nodes; The outputs of the Actor network and Critic network are: where ω nj and ω j are the weight coefficients of the Actor network and the Critic network respectively; The expression of the control input signal u(t) is as follows: u(t)=u n (t)+η m (0,ρ(t))(3) Among them, η m () is the Gaussian distribution function; ρ(t)=[1+exp(2n(t))] -1 .
2. A wind power generation system WADC design method based on reinforcement learning as claimed in claim 1, characterized in that: The timing differential signal δ TD The expression of (t) is as follows: δ TD (t)=r(t)+λn(t+1)-n(t)(4) Where λ is the discount coefficient, 0<λ<1, and the performance index function is:
3. A wind power generation system WADC design method based on reinforcement learning as claimed in claim 2, characterized in that: The weight coefficient update formula of the Actor network and Critic network is: oh j (t+1)=ω j (t)+η c d TD h(t)(6) Among them, η a and η c is the learning rate of the Actor network and the Critic network.
4. A wind power generation system WADC design method based on reinforcement learning as claimed in claim 3, characterized in that: The update formulas of the hidden layer node center function c(t) and width function σ(t) are: Among them, β1 and β2 are learning rates.
5. A wind power generation system WADC design method based on reinforcement learning as claimed in claim 4, characterized in that: The specific process of reinforcement learning control is as follows: S201: Initialize the learning rate η of the RBF network a and η c , discount factor λ and network weight coefficient ω nj and ω j , total number of iterations N c , iteration time T and error threshold δ0; S202: Enter the network loop iteration process, and set the current iteration number i=1; S203: Judgment i <N c Is it established? If so, proceed to step S204; otherwise, proceed to step S206; S204: Based on x(t) at time t, solve the output u of the Actor and Critic networks at time t according to formulas (1) and (2) n (t) and n(t); S205: Calculate the timing difference signal δ according to formula (4) TD If the value of δ0>δ TD Then update the weights of the Actor network and the Critic network according to equations (5) and (6); calculate the center function and width function according to equations (7) and (8); S206: Obtain the vector x(t+1) at time t+1; S207: Calculate the control input signal according to formula (3) and apply it to the controlled system; S208: Update the system state vector; S209: Determine whether the iteration time T has been reached. If so, the reinforcement learning process ends, otherwise return to step S203.
Citation Information
Patent Citations
Double-fed wind generator fuzzy damping control method for improving oscillation of power system
CN112865133A
Multi-device cooperative broadband oscillation suppression method for offshore wind power flexible direct current grid-connected system
CN114512995A
Design method for additional damping controller of photovoltaic power generation system based on reinforcement learning
CN114725953A
Design method for wide-area damping controller of time-delay photovoltaic power generation system
CN114725954A
Real-time robust variable-pitch wind turbine generator control system and method employing reinforcement learning
WO2021073090A1
