Subsynchronous oscillation suppression optimization method for wind storage grid-connected system

Abstract

The invention discloses a subsynchronous oscillation suppression optimization method for a wind storage grid-connected system. The method comprises the following steps: (1) establishing a flexible direct-current power transmission system, a direct-driven wind power plant and a wind power plant-flexible direct-current full-system interconnection model by adopting a flexible direct-current power transmission VSC-HVDC (Voltage Source Converter-High Voltage Direct Current) access power grid topological structure; (2) introducing an energy storage device to construct a topological structure based on a wind storage system; (3) establishing a model-free adaptive controller (MFAC) and an algorithm; (4) solving the decision process by using a depth deterministic strategy gradient algorithm, and obtaining an optimal action strategy through multi-sample depth machine learning in different wind speed operation states; according to the method, the virtual inertia coordination optimization frequency response, the model-free adaptive subsynchronous oscillation controller (MFAC) based on the NMC algorithm and the depth deterministic strategy gradient algorithm are combined together, so that subsynchronous oscillation of different mechanisms generated by wind power integration can be effectively inhibited, and the stability of flexible high-voltage direct-current transmission is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of subsynchronous oscillation suppression, and in particular to a subsynchronous oscillation suppression optimization method for a wind-storage grid-connected system. Background Art

[0002] With the rapid development of the economy, the energy crisis has become more and more serious, and the environment has suffered unprecedented pollution. New energy power generation represented by wind power has developed rapidly in recent years. As a renewable and clean form of energy, it plays an important role in the global energy transformation. Wind power generation has attracted much attention due to its wide distribution, rich resources, and zero emissions. As of the end of February 2022, my country's wind power installed capacity was about 330 million kilowatts. Flexible direct current transmission technology (Voltage Source Converter Based High Voltage Direct Current Transmission, VSC-HVDC) is a new type of transmission technology. It can independently decouple active and reactive power, does not require phase-changing voltage, and has strong fault ride-through capability. It has become an important transmission method for long-distance access to the power grid for wind power generation.

[0003] However, the interaction between the flexible DC converter station and the wind turbine may cause subsynchronous oscillation (SSO) in the system. This type of oscillation has the characteristics of strong time-varying and complex formation mechanism. Once it occurs, it may cause serious problems such as wind turbine shutdown and equipment damage. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a subsynchronous oscillation suppression optimization method for a wind-storage grid-connected system, which combines virtual inertia coordinated optimization frequency response, model-free adaptive subsynchronous oscillation controller (MFAC) and deep deterministic policy gradient algorithm, and can effectively suppress subsynchronous oscillations of different mechanisms generated by wind power grid connection, thereby improving the stability of flexible high-voltage direct current transmission.

[0005] Technical solution: The method for optimizing subsynchronous oscillation suppression of a wind-storage grid-connected system described in the present invention comprises the following steps:

[0006] (1) Using the VSC-HVDC grid access topology, a VSC-HVDC system, a direct-drive wind farm, and a wind farm-VSC system interconnection model are established;

[0007] (2) Introducing energy storage devices to construct a topological structure based on wind storage system;

[0008] (3) Establish model-free adaptive controller MFAC and algorithm;

[0009] (4) The decision-making process is solved using a deep deterministic policy gradient algorithm, and the optimal action strategy is obtained through deep machine learning of multiple samples under different wind speed operating conditions.

[0010] Furthermore, the formula of step (1) is as follows: s△X=A△X

[0011]

[0012] Furthermore, step (2) includes the following steps:

[0013] (21) connecting the energy storage device to the DC line between the wind farm and the main grid through a DC / DC converter;

[0014] (22) Using DC / DC converter for inner loop control;

[0015] (23) Establish a virtual inertia control strategy based on the DC capacitor electric field energy and the differentiated rotor kinetic energy of the wind turbine.

[0016] Further, step (23) comprises the following steps:

[0017] (231) A virtual inertia control strategy based on capacitor energy storage is established, and the formula is as follows: The inertia response process of the synchronous machine is described as:

[0018]

[0019] Where: H is the inertia time constant of the synchronous machine; f is the actual frequency of the AC power grid; P M is the mechanical power; P E is the electromagnetic power.

[0020] Ignoring the active power loss of the line, the change of DC voltage is used to reflect the power imbalance relationship between the rectifier side and the inverter side, as shown in the following formula:

[0021]

[0022] Where: C eq is the equivalent capacitance of MMC; U dc is the measured DC voltage; P wind is the output power of the wind farm; P grid is the active power delivered to the grid side;

[0023] Combining equations (2.3.1) and (2.3.2), we can obtain:

[0024]

[0025] Where: H MMC is the virtual inertia time constant of MMC;

[0026] Integrate both sides of the above equation and simplify to get:

[0027]

[0028] Where: U dc0 is the rated DC voltage in steady state; ΔU dc =U dc -U dc0 is the deviation of DC voltage; f is the actual frequency of AC power grid; f0 is the rated frequency of AC power grid;

[0029] Limit the DC voltage deviation to ±5%, ignore the quadratic term of the voltage deviation, and combine formula (2.3.5) to obtain

[0030]

[0031] Where: △f=f-f0 is the frequency deviation of the AC power grid, k c is the coupling coefficient between the DC voltage deviation and the frequency deviation;

[0032] By introducing the receiving-end grid frequency deviation into the GSMMC DC voltage control link, a coupling relationship between DC voltage and frequency is established, so that it can actively participate in grid frequency regulation and transmit frequency fluctuation information to the DC side, providing response conditions for the wind farm to actively support inertia. The formula is as follows:

[0033] U dcref =U dc0 +k c △f(2.3.7)

[0034] Where: U dcref Set the DC voltage reference value for GSMMC.

[0035] From the above formula, we can see that ΔU dc The coupling coefficient k between Δf c The larger the U dcref The larger the value, the easier it is to reach the DC voltage limit, resulting in the DC voltage being unable to transmit complete receiving-end grid frequency information; according to the maximum allowable DC voltage deviation ΔU dcmax The upper limit of the allowable frequency fluctuation Δf max , design k c The value of

[0036]

[0037] Then the virtual inertia time constant of MMC can be obtained as

[0038]

[0039] (232) A virtual inertia control strategy based on the kinetic energy of the fan rotor is established; the details are as follows:

[0040]

[0041] (233) A primary frequency modulation control strategy based on variable speed control is established; the formula is as follows:

[0042]

[0043] Where: η is the voltage regulation coefficient; kp is the coupling coefficient between the speed regulation and the DC voltage deviation, and its value is

[0044]

[0045] (234) A primary frequency modulation strategy based on pitch angle control is established, and the formula is as follows:

[0046]

[0047] (235) After the wind farm participates in the primary frequency regulation, the grid frequency deviation tends to a constant value. If the wind turbine rotor speed does not reach its limit at this time, after receiving the dispatch instruction, the DC voltage deviation generates an additional pitch angle Δβ through the proportional integral link. The active output of the wind farm is adjusted only by changing the pitch angle, eliminating the frequency deviation that still exists after the primary frequency regulation.

[0048] Furthermore, step (3) includes the following steps:

[0049] (31) The relationship between the system output and input is dynamically linearized by updating the partial derivatives, that is, the equivalent dynamic linearization model of the output at the next moment is established by using the compact dynamic linearization CFDL method; then the control structure of the MFAC is established, including: the pseudo partial derivative estimator, which continuously updates the pseudo partial derivative estimate and the system output estimate at this moment through the I / O data at the current moment, denoted as and A one-step forward predictive controller obtains the unlimited control output u0(k) according to a one-step forward weighted predictive control algorithm suitable for oscillation suppression; an output limiter and a compensator, the limiter constrains the amplitude of the controller output, obtains the limited control signal u(k) and outputs it to the controlled system, and the compensator generates a compensation signal ξ(k) to maintain the boundedness of the tracking error;

[0050] (32) Establish the NMC algorithm design, which is as follows: Pseudo partial derivative estimation algorithm, establish the following system output estimation algorithm based on CFDL format:

[0051]

[0052] Where: is the estimated value of the system output signal y(k); is the estimated error of the output, and γ is the error gain located at (0,1); therefore, the adaptive iterative algorithm for the pseudo partial derivative estimate is:

[0053]

[0054] Where: Γ(k)=η / (△u(k) 2 +μ) represents the weight coefficient of the pseudo partial derivative step change. The larger Γ(k) is, The faster the convergence speed is, the faster the system will converge. If Γ(k) is too large, the system will easily The convergence speed is too fast and it becomes unstable. η is a positive constant representing the step size factor of the pseudo partial derivative estimation algorithm. μ is a positive constant representing the penalty coefficient of the change of the pseudo partial derivative to determine the constraint strength of the pseudo partial derivative; F = 1-γ, located in the interval (0,1); In order to improve the ability to track time-varying parameters during the iteration process, the following algorithm reset mechanism is constructed:

[0055]

[0056] Among them: ε is a sufficiently small positive number, and is proven to be bounded.

[0057] Controller one-step-forward prediction algorithm: First, the MFAC control criterion function is established as:

[0058]

[0059] Where: y * represents the expected output of the controlled system (that is, the reference value of the input signal of the MFAC), λ is a positive number, which represents the penalty coefficient of the control criterion function, so that |u(k)-u(k-1)| is expected to be 0; since the input signal u(k) needs to be reduced to 0 when the subsynchronous oscillation is suppressed and the wind power system returns to steady-state operation, the following new control criterion function is established:

[0060]

[0061] Calculate the partial derivative of (3.2.4) with respect to u(k) and set it equal to 0 to obtain the one-step forward predictive control algorithm of the traditional MFAC:

[0062]

[0063] Among them, for the problem of suppressing subsynchronous oscillations, the expected output y * is 0; λ is a positive number, and u(k) is expected to be 0;

[0064] Substituting algorithm (3.2.1) into function (3.2.5), we get:

[0065]

[0066] For equation (3.2.6), find the partial derivative with respect to u(k), and set the partial derivative to 0, and we get:

[0067]

[0068] The introduction of step factors ρ0, ρ1 (0<ρ0, ρ1≤1) makes the algorithm more flexible:

[0069]

[0070] Control output limit constraint and compensation algorithm: Add constraint u(k)∈[u min ,u max ]:

[0071]

[0072] Where: u0(k) is the original output of the controller before the constraint. In order to ensure the boundedness of the system tracking error, a compensation signal ξ(k) is introduced into u0(k). The iterative algorithm is designed as follows:

[0073]

[0074] Then u0(k) is improved to

[0075]

[0076] (33) Based on stability analysis, the adjustable control parameters λ, ρ0, ρ1, μ, η, γ,

[0077] Furthermore, step (4) is specifically as follows: the subsynchronous oscillation suppression problem is converted into a Markov decision process, the overall framework of the oscillation suppression method, the environmental state set, the feasible action set and the reward function are designed, the deep deterministic policy gradient algorithm is used to solve its decision process, and the optimal action strategy is obtained through deep machine learning of multiple samples under different wind speed operating conditions;

[0078] The network structure includes four deep neural networks: Actor online policy network, Actor target policy network, Critic online Q value network and Critic target Q value network. The online network and the target network have the same architecture.

[0079] During the training process, the update of the Actor online network and Critic online network parameters θ and ω is achieved by gradient descent of the loss function and minimization of the loss function, respectively. The formula is as follows:

[0080]

[0081]

[0082] Where: Q(s i ,a i ,ω) is the current Q value; s i ,a i are the state and action quantities at time i respectively; m is the number of samples sampled from the experience pool each time.

[0083] The present invention provides a subsynchronous oscillation suppression optimization system for a wind-storage grid-connected system based on a deep deterministic policy gradient algorithm, comprising:

[0084] Interconnection module: used to establish interconnection models of flexible direct current transmission system, direct-drive wind farm and wind farm-flexible direct current system by adopting flexible direct current transmission VSC-HVDC access grid topology;

[0085] Energy storage module: used to introduce energy storage devices to build a topological structure based on wind storage system;

[0086] MFAC module: used to establish the model-free adaptive controller MFAC and algorithm;

[0087] Best Action Module: It is used to solve the decision-making process using a deep deterministic policy gradient algorithm and obtain the best action strategy through deep machine learning of multiple samples under different wind speed operating conditions.

[0088] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the methods for optimizing the suppression of subsynchronous oscillations in a wind-storage grid-connected system is implemented.

[0089] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the methods for optimizing subsynchronous oscillation suppression in a wind-storage grid-connected system.

[0090] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: combining the virtual inertia coordinated optimization frequency response, the model-free adaptive subsynchronous oscillation controller (MFAC) based on the NMC algorithm and the deep deterministic policy gradient algorithm, it can effectively suppress the subsynchronous oscillations of different mechanisms generated by wind power grid connection, thereby improving the stability of flexible high-voltage direct current transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 It is the overall flow chart of the present invention;

[0092] Figure 2 This is a diagram of a VSC-HVDC grid-connected wind power system of the present invention;

[0093] Figure 3 A topological diagram of the energy storage device of the present invention in a wind storage system;

[0094] Figure 4 A frequency control strategy diagram of the wind farm active support system of the present invention;

[0095] Figure 5 A schematic diagram of actively providing inertia support for the wind turbine generator system of the present invention;

[0096] Figure 6 It is a flow chart for implementing the frequency active support coordinated control strategy of the present invention;

[0097] Figure 7 It is the internal control structure diagram of MFAC of the present invention;

[0098] Figure 8 is a flow chart of the MFAC optimization method of the present invention;

[0099] Fig. 9 The DDPG algorithm diagram based on the actor-critic algorithm structure of the present invention;

[0100] Fig.10 It is an overall framework diagram of the subsynchronous oscillation suppression method of the wind power flexible direct current grid-connected system based on DDPG of the present invention;

[0101] Fig.11 is a control block diagram of the converter of the present invention;

[0102] Fig.12 This is a simplified control block diagram of the inverter of the present invention. DETAILED DESCRIPTION

[0103] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0104] like Figure 1 As shown, an embodiment of the present invention provides a method for optimizing subsynchronous oscillation suppression in a wind-storage grid-connected system, comprising the following steps:

[0105] Step 1 Based on the research on wind farm grid connection methods, a topological structure for wind farms to access the grid using flexible direct current transmission (VSC-HVDC) is proposed, and a flexible direct current transmission system, direct-drive wind farm, and wind farm-flexible direct current full system interconnection model are established.

[0106] The VSC-HVDC grid-connected wind power system consists of a converter station, a converter transformer, a DC filter, a wind turbine generator set, etc. Figure 2As shown. The converter stations installed on both sides of the VSC-HVDC are mainly voltage sources, and their components include the grid-side converter station and the wind farm-side converter station. The generators installed on the wind turbine generator units are mainly double-fed types, and their stator windings are directly connected to the grid, and are connected to the grid using the AC on the electronic side and the grid side and the DC link in the middle.

[0107] 1.2.1 Field measurement converter modeling

[0108] When the grid angular velocity is ω s When , in the synchronous rotating coordinate system dq, the dynamic equation of the wind farm side voltage source converter (WindFarim Voltage Sourced Converter, WFVSC) is:

[0109]

[0110] Where: u wd 、u wq and i wd 、i wq are the components of the wind farm bus voltage and bus current on the d and q axes respectively; v wd and v wq is the component of the fundamental voltage of WFVSC AC measurement on the d and q axes; R w , L w They represent the equivalent resistance and inductance of transformer T1 on the transmission line between WFVSC and wind farm respectively; ω w is the angular velocity of the wind farm.

[0111] 1.2.2 Grid-side converter modeling

[0112] When the grid angular velocity is ω s When , in the dq synchronous selection coordinate system, the dynamic equation of the grid-side voltage source converter (GSVSC) is:

[0113]

[0114] Where: u sd and u sq is the d-axis and q-axis components of the wind farm bus voltage; i sd and i sq are the d-axis and q-axis components of the electric field busbar current respectively; v sd 、v sq and R s , L s Represents the d-axis and q-axis components of the AC voltage on the GSVSC and the equivalent resistance and inductance on the transmission line between transformer T2 and the wind farm.

[0115] 1.2.3 Modeling of Flexible DC Transmission System

[0116] The rectifier on the wind farm side is directly connected to the wind farm. The control target is to maintain a constant AC bus voltage at the wind farm export node. The inverter adopts constant DC voltage / constant reactive power control to provide reactive power support for the AC system while maintaining the stability of the DC transmission line voltage.

[0117] The line voltage and current equation from the wind farm PCC to the rectifier is as follows:

[0118]

[0119] Where: L c is the rectifier port inductance; C f is the filter capacitor; i pccd 、i pccq are the d-axis and q-axis components of the PCC injected current of the wind farm, and u id 、u iq are the d-axis and q-axis components of the rectifier port voltage respectively; u td 、u tq They are the d-axis and q-axis components of the high-voltage DC transmission line current respectively; ω is the rated angular frequency.

[0120] According to the control system, the outer loop d-axis adopts fixed active power and the q-axis adopts decoupling control of fixed AC voltage to maintain the stability of the PCC AC voltage of the wind farm; the inner loop adopts PI control. The dynamic mathematical model of the rectifier is as follows:

[0121]

[0122] Where: x1-x4 is the integral output state quantity of the rectifier PI control link; u t =u td +ju tq is the wind farm export node; P is the DC transmission active power; k p1 -k p4 , k i1 -k i4 They are proportional and integral control parameters of the DC transmission controller respectively. The rectifier synchronous PLL takes the PCC phase angle as input, and its output phase angle is used for the converter Parker transformation. The mathematical model corresponding to the PLL structure is as follows:

[0123]

[0124] Where: x PLL is the output of the integral control link of PLL; θ t and θ PLL are the input and output phase angles of the PLL respectively; k p PLL, k i PLL are the proportional and integral control parameters of PLL respectively. Assuming that the inverter side is connected to a strong AC grid, existing studies often ignore the impact of the inverter on the rectifier side, which can be equivalent to a constant DC voltage source. The dynamic model of the DC transmission system is as follows:

[0125]

[0126] Where: L dc , R dc They are the inductance and resistance of the DC line; C dc1 is the rectifier side capacitance; i dc is the DC current; u dc1 、u dc are the DC voltages on the rectifier and inverter sides respectively.

[0127] Linearizing the above formula, the state space model of the flexible DC transmission system is as follows:

[0128]

[0129] Where: ΔX d is all state variables of the flexible DC transmission system; △I pcc =-[△i sd △i sq ] T Inject current variable into the wind farm PCC; △U t =[△u td △u tq ] T A is the voltage state variable of the wind farm outlet node; d , B d , C d They are the state matrix, input matrix, and output matrix of the flexible DC transmission system. It can be seen that the flexible DC subsystem takes the wind farm output current as input and the wind farm export node voltage as output.

[0130] 1.2.4 Direct-drive wind farm modeling

[0131] According to the linearized state space model of wind farm wind turbines, the full-order model of wind farm containing N DFIGs is obtained:

[0132]

[0133] Where: △X w =[△X w1 △X w2 ···△X wN ] T is the total state variable of the wind farm; A w , B w , Cw are the state matrix, input matrix and output matrix of the wind farm respectively; △U w =[△U w1 △U w2 ···△U wN ] T is the node voltage variable of the wind farm access system; △I w =[△I w1 △I w2 ···△I wN ] T is the current variable of the wind farm connected to the system.

[0134] The relationship between node voltage and node injection current is shown below:

[0135]

[0136] Where: ΔU pcc is the PCC voltage variable of the wind farm; Y w11 , Y w12 , Y w21 , Y w22 , Y w23 , Y w32 , Y w33 is the wind farm network topology block admittance matrix. According to the above formula, we can get:

[0137]

[0138] Combining the above formula with the full-order model of the wind farm containing N DFIGs, the full-order state space model of the wind power grid-connected system with N DFIGs can be obtained:

[0139]

[0140] Where: C wf is the output matrix. The wind farm subsystem is based on ΔU t is the input, ΔI pcc is the output.

[0141] 1.2.5 Wind farm-flexible DC full system interconnection model

[0142] According to step 1.2.4, the interconnection model of wind farm-flexible DC system can be constructed:

[0143] s△X=A△X

[0144]

[0145] Step 2 In order to stabilize the frequency response of the power grid at the sending end of large-scale wind power transmission and effectively enhance the primary frequency regulation capability of the power grid at the sending end, combined with the above-mentioned wind power grid-connected model, energy storage devices are introduced to improve the frequency characteristics of the main grid, and a topological structure based on the wind storage system is further constructed. The method of virtual inertia coordinated control based on the DC capacitor electric field energy and the differentiated rotor kinetic energy of the wind turbine is studied to improve the frequency response capability. By improving the frequency response, the subsynchronous oscillation is suppressed by optimizing the dynamic characteristics of the system at different frequencies.

[0146] Step 2.1 Topology of energy storage devices in wind energy storage system

[0147] The energy storage device is connected to the DC line between the wind farm and the main grid through a DC / DC converter, as follows Figure 3 shown.

[0148] Step 2.2 DC / DC converter inner loop control

[0149] Since electrochemical energy storage is mainly charged and discharged in the form of direct current, this method only needs to control the DC / DC converter to achieve the regulation and control of the active power of the energy storage system. The inner loop control of the DC / DC converter is completed by controlling the current, and a proportional frequency additional controller based on the AC bus frequency Δf1 of the wind farm is added to its outer loop control to obtain the converter control block diagram, as shown in Fig.11 As shown, U b 、i b are the terminal voltage and current on the DC bus side of the converter respectively.

[0150] Simplifying the inner loop control of the converter into a first-order lag transfer function, we can get the simplified control block diagram of the converter, as shown in Fig.12 shown.

[0151] K in the figure E Additional control proportional coefficient for energy storage frequency; T E is the inner loop control response time constant; ΔP Eref The change in the power setting value sent to the energy storage system; ΔP E The power change sent by the energy storage system. When the frequency of the wind farm decreases, the reference value of the active power output by the energy storage system increases. After the inner loop control of the converter, the active power output by the energy storage system increases, which compensates for and exceeds the power reduction of the DC line. The excess power is used to offset the power shortage transmitted to the main grid through the AC line. Similarly, when the frequency of the wind farm increases, the energy storage system can also compensate for the power imbalance of the main grid and reduce the frequency fluctuation of the main grid. The frequency model transfer function of the energy storage system with additional frequency control is:

[0152] Step 2.3 In this step, a virtual inertia control strategy is designed based on the DC capacitor electric field energy and the differentiated rotor kinetic energy of the wind turbine, which jointly provide inertia support for the system when the receiving grid frequency suddenly changes.

[0153] 2.3.1 Virtual inertia control strategy based on capacitor energy storage

[0154] The inertia response process of the synchronous machine can be described as

[0155]

[0156] Where: H is the inertia time constant of the synchronous machine; f is the actual frequency of the AC power grid; P M is the mechanical power; P E is the electromagnetic power.

[0157] The equivalent capacitor on the DC side of the MMC can use its dynamic characteristics to suppress the change of DC voltage. That is, when the active power loss of the line is ignored, the change of DC voltage is used to reflect the power imbalance relationship between the rectifier side and the inverter side, as shown in the following formula:

[0158]

[0159] Where: C eq is the equivalent capacitance of MMC; U dc is the measured DC voltage; P wind is the output power of the wind farm; P grid is the active power delivered to the grid side.

[0160] In order to make the MMC have the inertia response characteristics of the synchronous machine, analogous to the rotor motion equation of the synchronous machine, Pwind can be regarded as the mechanical power of the synchronous machine. In order to facilitate the design of the control strategy, when only considering the DC capacitor to provide inertia support for the system, Pwind can be assumed to be a constant; Pgrid can be regarded as the electromagnetic power of the synchronous generator, and the change of the DC voltage can be analogous to the change of the synchronous machine speed. The main difference is that the DC capacitor uses its own energy to provide virtual inertia support for the system, thereby suppressing frequency mutations; the synchronous machine uses its rotor kinetic energy to provide inertia support and directly determines the system frequency. Combining equations (2.3.1) and (2.3.2), we can get:

[0161]

[0162] Where: H MMC is the virtual inertia time constant of MMC.

[0163] Integrate both sides of the above equation and simplify to get:

[0164]

[0165] Where: U dc0 is the rated DC voltage in steady state; ΔU dc =U dc -U dc0 is the deviation of DC voltage; f is the actual frequency of AC power grid; f0 is the rated frequency of AC power grid.

[0166] In order to ensure the safe and stable operation of the system, the DC voltage deviation is usually limited to a small range (about ±5%), so the quadratic term of the voltage deviation can be ignored. Combining formula (2.3.5), we can get

[0167]

[0168] Where: △f=f-f0 is the frequency deviation of the AC power grid, k c is the coupling coefficient between the DC voltage deviation and the frequency deviation.

[0169] By introducing the frequency deviation of the receiving grid into the GSMMC DC voltage control link, as shown in formula (2.3.7), a coupling relationship between DC voltage and frequency can be established, so that it can actively participate in the grid frequency regulation while transmitting frequency fluctuation information to the DC side, providing response conditions for the wind farm to actively support the inertia.

[0170] U dcref =U dc0 +k c △f(2.3.7)

[0171] Where: U dcref Set the DC voltage reference value for GSMMC.

[0172] From the above formula, we can see that ΔU dc The coupling coefficient k between Δf c The larger the U dcref The larger the value, the easier it is to reach the DC voltage limit, resulting in the DC voltage being unable to transmit complete frequency information of the receiving grid. dcmax The upper limit of the allowable frequency fluctuation Δf max , design k c The value of

[0173]

[0174] Then the virtual inertia time constant of MMC can be obtained as

[0175]

[0176] 2.3.2 Virtual inertia control strategy based on wind turbine rotor kinetic energy

[0177] Due to the constraints of DC voltage deviation, the energy that DC capacitors can provide is relatively small and the inertial effect is limited. The inertial support effect can be further enhanced by combining the rotor kinetic energy of the wind turbine, so that it can store or release rotor kinetic energy and actively provide inertial support when the grid frequency changes suddenly.

[0178] When the system is disturbed, the established DC voltage and grid frequency coupling relationship is used to transmit the grid frequency fluctuation information to the wind farm side using the DC voltage as the carrier, and the change rate of the DC voltage on the WFMMC side (dU dc / d t ) is introduced into the active power control link of each DFIG (doubly-fed asynchronous wind turbine generator) in the wind farm, such as Figure 4 As shown by the green dotted line, it actively changes its active output to respond to the sudden change of the receiving grid frequency, providing inertia support for the system. In order to enable the offshore wind farm to simultaneously meet the frequency regulation requirements under different frequency changes, a certain amount of power reserve is usually reserved to enable it to operate steadily in the overspeed load reduction mode. Assume that the power tracking curve equation after the wind turbine overspeed load reduction is:

[0179]

[0180] Where: k LS is the load shedding power tracking coefficient; ω d is the current speed.

[0181] When the receiving grid frequency increases, the DC voltage will increase in real time. dc / dt changes its speed reference value and quickly reduces the wind farm output power, as shown in formula (2.3.11). Since the speed adjustment amount Δω is usually not large, the quadratic term △ω is ignored here. 2 and the cubic term △ω 3 Combining equation (2.3.2) and equation (2.3.11), the fan reduces the active output ΔP wind The process can be compared to the process of a virtual capacitor providing inertia support, that is, the fan provides virtual inertia support for the system through the rotor kinetic energy, as shown in formula (2.3.12), and then the relationship between Δω and dUdc / dt can be obtained as shown in formula (2.3.13).

[0182]

[0183]

[0184] Where: ΔC vir k is the additional virtual capacitor on the DC side; d It is the coupling coefficient between the speed regulation and the DC voltage change rate.

[0185] Figure 5Schematic diagram of actively providing inertia support for wind turbines. The wind turbine operates steadily at load reduction point 2. At the beginning of the frequency rise of the receiving grid, the unbalanced power is the largest. The proposed virtual inertia control can make the wind turbine quickly and significantly reduce its output power. This process is shown in Figure 3 As shown by the red arrow from midpoint 2 to point 3, the frequency rise rate is effectively reduced. At this time, the mechanical power captured by the fan is greater than the electromagnetic power, the rotor speed increases and provides inertia support for the system by storing kinetic energy; as the speed increases, the mechanical power will gradually decrease, such as Figure 5 As shown in ΔPwind, the frequency rise can also be suppressed, showing a damping characteristic similar to that of a synchronous generator. After the synchronous generator of the receiving grid performs a frequency modulation, the system frequency deviation will gradually approach a constant value, and dUdc / dt will become 0. The wind turbine will no longer provide inertia support and will return to the steady-state operation point before the disturbance. This process is shown in Figure 3 The midpoint 4 to point 2 is indicated by the red arrow.

[0186] In summary, after taking into account the inertia of the wind farm active support system, equation (2.3.2) can be further modified to equation (2.3.14), that is, the wind farm can enhance its inertial effect by increasing the equivalent capacitance of the flexible DC system, so that its rotor kinetic energy and DC capacitor energy can jointly provide inertial support and increase its equivalent inertia time constant, as shown in equation (2.3.15), thereby suppressing frequency mutations and improving system frequency stability.

[0187]

[0188] Where: H S is the virtual inertia time constant of the system.

[0189] The closer the fan rotor speed is to the maximum speed (or optimal speed) during steady-state operation, the smaller the space for speed change and the less rotor kinetic energy that can be stored (or released); conversely, the more. Therefore, the speed at the start of the fan variable speed control needs to be adjusted according to the fan variable speed control. d0 Reasonable design d , in order to make full use of the rotor kinetic energy of each DFIG and realize the coordinated response of virtual inertia.

[0190] The fan rotor runs at the optimal speed ω opt At this point, the mechanical power captured by the fan reaches its maximum value and can play the greatest damping role. Therefore, when the disturbance causes the grid frequency to drop, in order to suppress the frequency drop more quickly, the minimum limit that the fan speed can drop is ω opt When the disturbance causes the grid frequency to rise, the wind turbine increases its speed to store kinetic energy, and the maximum speed increase limit is ω max Therefore, this paper designs the adjustable range of fan speed as [ω opt ,ω max]. Δω is proportional to dUdc / dt (i.e. df / dt). In order to prevent the frequency change rate df / dt from being too large and damaging the unit, the limit constraints of Δω and df / dt must be considered simultaneously when designing the kd parameters.

[0191] Based on dU dc The coupling relationship between / dt and df / dt is considered when the frequency rises, and the maximum value constraint of the DC voltage change rate (dU dc / dt) max With ω max The corresponding relationship between them is shown in formula (2.3.16); Similarly, when the frequency decreases (dU dc / dt) max The corresponding relationship between and ωopt. In summary, the coupling coefficient k d The value of is shown in formula (2.3.17).

[0192]

[0193] It can be seen from formula (2.3.17) that each DFIG can reasonably select its kd value according to ωd0, so as to realize the differentiated utilization of the kinetic energy of each wind turbine rotor under different wind speeds, and effectively improve the inertia support capacity of the wind farm.

[0194] Since the process of inertia support provided by DC capacitors and wind turbine rotors is short-lived and unsustainable, it can only suppress frequency mutations at the initial stage of disturbance. In order to maintain the receiving grid frequency within a safe range, the wind farm needs to have the ability to participate in system frequency regulation, thereby reducing system frequency deviation and ensuring its safe and stable operation. Based on the frequency and DC voltage coupling relationship established by formula (2.3.7), control strategies for wind power to participate in primary and secondary frequency regulation through flexible DC interconnection system are designed to improve frequency support capability.

[0195] 2.3.3 Primary frequency modulation control strategy based on variable speed control

[0196] In order to achieve a frequency regulation function similar to that of a synchronous machine, a wind farm primary frequency regulation coordination control strategy is designed based on local DC voltage measurement information, which takes into account the response difference between variable speed control and pitch angle control. By introducing the DC voltage deviation (ΔUdc) into the active control link of each DFIG in the wind farm, the active output of the wind turbine is adjusted, and the wind farm actively participates in the primary frequency regulation, thereby improving the frequency stability of the receiving AC power grid.

[0197] When the DC voltage reaches its dead zone period ΔU dcth When the wind turbine is in operation, it immediately changes the speed to adjust the mechanical power captured by the wind turbine, so that the wind farm actively participates in the primary frequency regulation. Figure 4The blue dotted line is shown. Combining formula (2.3.12), it can be seen that the frequency deviation (Δf) of the receiving grid is calculated by the DC voltage deviation (ΔU dc ) is transmitted to the wind farm in the form of , and satisfies △f≤△U dc ≤△ω≤△P wind , as shown in equations (2.3.18) and (2.3.19). If the speed of the wind farm is within the allowable adjustment range after the first frequency regulation, the wind turbine will operate stably in a new equilibrium state.

[0198]

[0199]

[0200] Where: η is the voltage regulation coefficient; kp is the coupling coefficient between the speed regulation amount and the DC voltage deviation amount.

[0201] In order to achieve a reasonable distribution of the primary frequency regulation power of the wind farm, combining equations (2.3.16) and (2.3.17), we can get ΔU dc The coupling coefficient k between Δω and p The value of

[0202]

[0203] 2.3.4 Primary frequency modulation based on pitch angle control

[0204] When the fan rotor speed reaches ω max (or ω opt ), that is, the primary frequency modulation capability based on variable speed control reaches the upper limit, by controlling its rotor speed to remain unchanged, maintaining the frequency modulation capability of the fan variable speed control, and starting Figure 4 The primary frequency modulation mode based on voltage-type additional pitch angle control within the red dashed line is shown. By superimposing the additional pitch angle Δβ generated by the DC voltage deviation onto the initial standby pitch angle β0, the active output of the wind farm is further changed, thereby achieving the coordination of wind turbine variable speed control and additional pitch angle control in primary frequency modulation. Coupling coefficient k in voltage-type additional pitch angle control β Can be defined as

[0205]

[0206] 2.3.5 Secondary frequency modulation control strategy

[0207] The secondary frequency regulation potential of large-scale wind power through flexible DC interconnection system is further explored. According to the local DC voltage measurement information, a secondary frequency regulation strategy based on additional pitch angle control is designed. Figure 3 As shown in the red dotted box in the middle, it presents the characteristics of a dominant power source as a whole, improving the frequency stability of the receiving-end power grid.

[0208] Since the frequency deviation of the receiving grid is directly coupled with the DC voltage, the secondary frequency regulation control target in the voltage-type additional pitch angle control is set to the rated DC voltage U dcref (Ignore line losses), after receiving the dispatching instruction, it actively participates in the DC voltage regulation of the flexible DC system, eliminates its steady-state deviation, and then realizes the error-free regulation of the receiving-end grid frequency. After the wind farm participates in the frequency regulation once, the grid frequency deviation tends to a constant value. If the wind turbine rotor speed does not reach its limit value ω at this time max (or ω opt ), after receiving the dispatching instruction, the DC voltage deviation generates an additional pitch angle Δβ through the proportional integral link, and the active output of the wind farm is adjusted only by changing the pitch angle, eliminating the frequency deviation that still exists after the primary frequency regulation, realizing the secondary frequency regulation function similar to the synchronous machine, and at the same time restoring the rotor speed to the initial speed, restoring the inertia support capacity of the wind farm; if the wind turbine rotor speed has reached its limit value ω max (or ω opt ), it means that the unbalanced power is large and the speed needs to be maintained unchanged, that is, the frequency regulation capability of the variable speed control is maintained, and then the pitch angle is further adjusted through the secondary frequency regulation, so that the variable speed control and the pitch angle control can jointly bear the system unbalanced power and maintain the DC voltage stable, thus achieving the zero-difference regulation of the receiving-end power grid frequency while making full use of the power regulation capability of the wind farm. Figure 6 Flowchart for the implementation of active frequency support coordinated control strategy.

[0209] Step 3 Considering the subsynchronous oscillation phenomenon that occurs with the increase of wind turbines, a subsynchronous oscillation accessory control method based on model-free adaptation is proposed, namely, the model-free adaptive controller (MFAC). This method uses the equivalent dynamic linearization method to describe the nonlinear dynamic characteristics of the system. First, the structure of the model-free adaptive controller (MFAC) is designed, and the partial derivative dynamic estimation method is improved. Secondly, facing the demand characteristics of subsynchronous oscillation accessory control, the MFAC model-free control (NMC) algorithm is improved and designed. Finally, the model-free adaptive controller (MFAC) is combined with the NMC algorithm to suppress subsynchronous oscillations.

[0210] Step 3.1 Control structure of MFAC

[0211] Step 3.1.1 Dynamic linearization of the system

[0212] The wind power transmission system can be regarded as a SISO discrete-time nonlinear system, which is described by the following model:

[0213] y(k+1)=f(y(k),...,y(kN),u(k),...,u(kL))+d(k) (3.1.1)

[0214] Where: u(k), y(k) represent the input and output signals of the controlled system at time k (that is, the output and input signals of the model-free controller at time k), L and N are the dimensions of input and output, f is an unknown nonlinear function, and d(k) represents the finite unknown disturbance to the system at time k.

[0215] For system (3.1.1), model-free adaptive control theory proposes to dynamically linearize the relationship between system output and input by updating pseudo partial derivatives, thereby constructing a dynamic linearization model for the subsynchronous damping controller that can be equivalently described by only input and output (I / O) data.

[0216] The compact dynamic linearization (CFDL) method only uses the input data of the system at the current moment to establish an equivalent dynamic linearization model of the system output at the next moment, which is specifically expressed as follows:

[0217]

[0218] Where: φ c (k) represents the pseudo partial derivative of the system (3.1.1). By adaptively updating the pseudo partial derivative, the equivalent linear fitting process of the system is dynamically realized, and φ c (k)| is bounded; △y(k+1)=y(k+1)-y(k), △u(k)=u(k)-u(k-1); △d(k)=d(k)-d(k-1), and |△d(k)|<Ω, Ω is a positive constant representing the limit value of Δd(k).

[0219] Step 3.1.2 Control structure of MFAC

[0220] The control structure inside the MFAC can be Figure 7 Indicated by Figure 7 It can be seen that the control structure of MFAC is mainly divided into three parts: the first is the pseudo partial derivative estimator, which continuously updates the pseudo partial derivative estimate and the system output estimate at this time through the I / O data at the current moment, denoted as and The second is a one-step forward predictive controller, which obtains the unlimited control output u0(k) according to the one-step forward weighted predictive control algorithm suitable for oscillation suppression; the third is the output limiter and compensator, which constrains the amplitude of the controller output, obtains the limited control signal u(k) and outputs it to the controlled system, and the compensator generates the compensation signal ξ(k) to maintain the boundedness of the tracking error. Since u(k) is both the input of the pseudo partial derivative estimator and the output of the one-step forward predictive controller, the entire MFAC presents closed-loop control and has good stability.

[0221] Step 3.2 Design of NMC algorithm for MFAC

[0222] According to the MFAC control structure, MFAC is divided into three modules: pseudo partial derivative estimation, one-step forward predictive control, and control output limiting and compensation. The internal algorithms of each module are designed as follows:

[0223] Pseudo-partial derivative estimation algorithm

[0224] In order to obtain the iterative algorithm of the pseudo partial derivative estimate, the following system output estimation algorithm based on CFDL format is established:

[0225]

[0226] Where: is the estimated value of the system output signal y(k); is the estimated error of the output, and γ is the error gain located at (0,1).

[0227] Therefore, the adaptive iterative algorithm for estimating the pseudo partial derivative is:

[0228]

[0229] Where: Γ(k)=η / (△u(k) 2 +μ) represents the weight coefficient of the pseudo partial derivative step change. The larger Γ(k) is, The faster the convergence speed is, the faster the system will converge. If Γ(k) is too large, the system will easily The convergence speed is too fast and the algorithm becomes unstable. η is a positive constant representing the step size factor of the pseudo partial derivative estimation algorithm. μ is a positive constant representing the penalty coefficient of the change of the pseudo partial derivative to determine the constraint strength of the pseudo partial derivative; F = 1-γ, located in the interval (0,1).

[0230] In order to improve the ability to track time-varying parameters during the iteration process, it is necessary to build the following algorithm reset mechanism:

[0231]

[0232] Among them: ε is a sufficiently small positive number, and is proven to be bounded.

[0233] Controller One-Step-Ahead Prediction Algorithm

[0234] First, the most common traditional MFAC control criterion function is established as:

[0235]

[0236] Where: y *represents the expected output of the controlled system (i.e., the reference value of the input signal of the MFAC), λ is a positive number, representing the penalty coefficient of the control criterion function, so that |u(k)-u(k-1)| is expected to be 0. Since the input signal u(k) needs to be reduced to 0 when the subsynchronous oscillation is suppressed and the wind power system returns to steady-state operation, the following new control criterion function is established:

[0237]

[0238] Calculate the partial derivative of (3.2.4) with respect to u(k) and set it equal to 0 to obtain the one-step forward predictive control algorithm of the traditional MFAC:

[0239]

[0240] Among them, for the problem of suppressing subsynchronous oscillations, the expected output y * is 0; λ is a positive number, and u(k) is expected to be 0.

[0241] Substituting algorithm (3.2.1) into function (3.2.5), we get:

[0242]

[0243] For equation (3.2.6), find the partial derivative with respect to u(k), and set the partial derivative to 0, and we get:

[0244]

[0245] The introduction of step factors ρ0, ρ1 (0<ρ0, ρ1≤1) makes the algorithm more flexible:

[0246]

[0247] Control output limit constraint and compensation algorithm

[0248] Since the normal operation of a DFIG with an attached MFAC is threatened by disturbances and serious failures, the output of the MFAC should be limited within a certain range of values ​​to prevent excessive control force and eliminate adverse effects on the normal control function of the DFIG.

[0249] Add the constraint u(k)∈[u min ,u max ]:

[0250]

[0251] Where: u0(k) is the original output of the controller before the constraint, which is shown in equation (3.2.9). In order to ensure the boundedness of the system tracking error, a compensation signal ξ(k) is introduced to u0(k), and its iterative algorithm is designed as:

[0252]

[0253] Then u0(k) is improved to

[0254]

[0255] During the real-time operation of the system, the driving data required by MFAC are only its input and output data. Specifically, according to equations (3.2.2) and (3.2.12), MFAC only needs the following three data at time k: the output y(k) of the controlled system at time k, and the output u(k) and u(k-1) of the controller at time k and time k-1. In addition, the main control output algorithm of MFAC shown in equation (3.2.12) only contains the simplest algebraic operations: addition, subtraction, multiplication, and division. The operation time and cost required are relatively small, so it has high practicality.

[0256] Step 3.3 Parameter optimization design method of MFAC

[0257] In the improved model-free adaptive control algorithm proposed in step 3.2, the adjustable control parameters λ,ρ0,ρ1,μ,η,γ, It has a strong correlation with the control performance and response speed of the controller, and its value needs to be reasonably optimized.

[0258] Step 3.3.1 Setting method based on parameter design principle in stability analysis

[0259] First, since the pseudo partial derivative in MFAC has h adaptive capability, its initial value It can be selected more roughly within a wider range.

[0260] Secondly, determine the values ​​of penalty factor λ and step size factors ρ0, ρ1. The values ​​of parameters λ, ρ0, ρ1 must meet the following conditions to ensure the stability of the controller:

[0261] Choose a suitable λ so that when λ>λ min hour,

[0262]

[0263] in, express The upper bound of . Since the update direction of the pseudo partial derivative is generally getting closer to 0, and the pseudo partial derivative has the sign preservation property, It can generally be considered

[0264] Choose a suitable ρ1 so that

[0265]

[0266] Among them, according to the conclusion in the stability proof

[0267]

[0268] First, ensure that ρ1≤ρ0. Based on this premise, select a suitable initial value, then modify ρ1 according to the stable value converged to by the pseudo-partial derivative, and verify again whether the modified ρ1 satisfies equation (3.3.2). If not, repeat the previous step until it is satisfied.

[0269] This can roughly determine the value range of the penalty factor λ and the step size factors ρ0, ρ1, and then adjust the specific values ​​according to the following design principles:

[0270] i. The smaller λ is, the faster the system responds, but overshoot may occur; the larger λ is, the slower the system responds.

[0271] ii. The step size factors ρ0, ρ1 affect the learning speed of the controller

[90] . The larger their values, the greater the amplitude of the controller output change per step.

[0272] Finally, the values ​​of the remaining adjustable parameters μ, η, γ in the control algorithms (3.2.2) and (3.2.12) can be adjusted according to the following design principles.

[0273] iii.μ is the weight coefficient (positive value) of the pseudo partial derivative performance index function, which is used to determine the constraint strength of the pseudo partial derivative. If μ is too small, the possibility of system instability will increase; if μ is too large, the oscillation amplitude of the controlled system output will increase and the convergence speed will slow down.

[0274] iv.η is used to adjust the estimation speed of pseudo partial derivatives, η∈(0,2]. The larger η is, the faster the pseudo partial derivatives converge.

[0275] v.γ is the gain of the output estimation error e0(k), γ∈(0,1). It affects the convergence speed of the system. The larger γ is, the faster the convergence speed of the controlled system tracking error will be; however, if γ is too large, the system tracking error may not converge effectively. Compared with traditional model-based control theory, one of the advantages of model-free adaptive control is that the range of each control parameter of the algorithm is relatively wide, and there is no need for strict and detailed calculation and analysis based on the mechanism of the object model.

[0276] Step 3.3.2 Intelligent optimization method based on performance indicators

[0277] The tuning method mentioned in step 3.3.1 is relatively rough, but it is still practical due to the MFAC's own adaptive ability and the advantage of not relying on an accurate model. However, if we want to maximize the control capability of MFAC, we need to propose a more sophisticated parameter optimization method.

[0278] Using performance indicators as objective functions to optimize controller parameters is a common parameter design method. According to the characteristics of the MFAC control criterion, the performance indicator based on the integral of the time product of the squared error (ITSE) is selected to measure the effect of the MFAC error tracking performance changing over time.

[0279] The control parameters to be optimized for MFAC are λ,ρ0,ρ1,μ,η,γ, Since the pseudo partial derivative itself can be adaptively updated according to different operating conditions, The value of can be set roughly, which also reflects a major advantage of MFAC; in addition, Determining the value first can provide a base value and a rough range of values ​​for the subsequent optimization of other parameters. Therefore, the optimization object of the method proposed in this step can be selected as [λ, ρ0, ρ1, μ, η, γ].

[0280] The specific optimization steps are as follows:

[0281] S1 establishes the objective function. The function established based on the ITSE indicator is:

[0282]

[0283] Where e(t) represents the tracking error of MFAC e(k) = y * -y(k). Based on the software background of MATLAB / SIMULINK, the model shown in formula (4-35) is established in the simulation program of MFAC in the test system model.

[0284] S2 sets the initial values ​​of [λ, ρ0, ρ1, μ, η, γ]. The selection of these parameters has a certain influence on the optimization effect and the required time. The design principles are the same as those mentioned in step 3.3.1 i.~v.

[0285] Genetic algorithm (GA) is used to optimize the control parameters. The GA toolbox in MATLAB can be used, the population size is set to 50, and the maximum number of iterations is set to 100.

[0286] S3 calculates the objective function value. By running the simulation program related to the objective function in SIMULINK, the value of formula (3.3.4) is calculated.

[0287] S4 determines whether the optimized [λ, ρ0, ρ1] satisfies the constraints, as shown in equations (3.3.1) and (3.3.2). If the constraints are not satisfied, a penalty term is added to the objective function shown in equation (3.3.4).

[0288] S5 determines whether the number of iterations reaches the maximum number of iterations. If not, go to Step 3 for the next optimization iteration; if so, select a set of [λ, ρ0, ρ1, μ, η, γ] values ​​with the smallest objective function value as the final optimal solution.

[0289] The flowchart of the above optimization method is as follows Figure 8 As shown in Figure 2, MFAC has strong adaptability and can optimize parameters based on only one working point.

[0290] Step 4 takes into account that the control of the individual converters by the MFAC controller in step 3 will have a certain impact on the dynamic performance and performance of the entire system, and due to the time-varying nature of subsynchronous oscillations, it is difficult to ensure their suppression effect under variable operating conditions. Based on the mathematical model and mechanism of the flexible direct current sending-end converter and the MFAC model-free adaptive controller, the subsynchronous oscillation problem is converted into a Markov decision process, and the overall framework of the oscillation suppression method, the environmental state set, the set of feasible actions, and the reward function are designed. The Deep Deterministic Policy Gradient algorithm (DDPG) is used to solve its decision process, and the optimal action strategy is obtained through deep machine learning of multiple samples under different wind speed operating conditions.

[0291] Step 4.1 Principle of Deep Deterministic Policy Gradient Algorithm

[0292] Reinforcement learning is an autonomous learning method in which an intelligent agent obtains the optimal strategy through trial and error in the process of interacting with the environment. This solution adopts the DDPG algorithm, which uses the actor-critic algorithm architecture and introduces deep neural networks (DNN) to ensure the continuity of action space and state space, effectively avoiding the dimensionality curse problem.

[0293] like Fig. 9 As shown, its network structure includes four deep neural networks: Actor online policy network, Actor target policy network, Critic online Q value network and Critic target Q value network, where the online network and the target network have the same architecture.

[0294] During the training process, the update of the Actor online network and Critic online network parameters θ and ω is achieved by gradient descent of the loss function and minimization of the loss function respectively. The goal of the Actor online network is to maximize the value function, so the loss function J(θ) of the network parameter θ and the loss function gradient for:

[0295]

[0296] Where: Q(s i ,a i ,ω) is the current Q value; s i ,a i are the state and action quantities at time i respectively; m is the number of samples sampled from the experience pool each time.

[0297] The goal of the critic online network is to approximate the value function. The loss function can be obtained by dividing the current Q value by the target Q value Q(s i+1 ,a i+1 ,ω′) is expressed as:

[0298]

[0299] γ i = r + γQ (s i+1 ,a i+1 ,ω′)(4.1.8)

[0300] The parameters θ′, ω′ of the Actor objective function and the Critic objective function are updated by software (soft update) through online network parameters:

[0301]

[0302] Where: τ is the update coefficient, generally 0<τ<<1.

[0303] In order to increase the exploration ability of the DDPG algorithm in the process of interacting with the environment, the DDPG algorithm outputs the action value π θ (s) is superimposed with random noise N, and the real action value a of its output ture for:

[0304] a ture =π θ (s)+N (4.1.10)

[0305] This scheme uses Uhlenbeck Ornstein (UO) noise to generate superimposed noise in the random process to explore more optimized action strategies.

[0306] Step 4.2 Subsynchronous oscillation suppression method based on DDPG

[0307] In this step, according to the relevant principles of MFAC adaptive controller and deep deterministic policy gradient algorithm, the overall framework of subsynchronous oscillation suppression method of wind power grid-connected system based on DDPG is established, and the environmental state set S and the feasible action set A in the framework are selected as well as the reward function is designed.

[0308] Step 4.2.1 Overall framework definition

[0309] The controlled electrical quantities in the VSC-HVDC converter controller are measured from the wind power grid-connected system and input into the intelligent agent as state quantities. The intelligent agent determines the action quantity, i.e., the output of the additional damping control signal, based on the state quantity and the reward obtained, and finally obtains the optimal action strategy of the additional damping control signal, such as Fig.10 shown.

[0310] Fig.10 It is the overall framework of the subsynchronous oscillation suppression method of the wind power flexible direct current grid-connected system based on DDPG. dHC (t),△u qHC (t) is the voltage oscillation amplitude of the VSC-HVDC sending-end controller in the rotating synchronous coordinate system at time t; △i dHC (t),△i qHC (t) The current oscillation amplitude of the VSC-HVDC sending-end controller in the rotating synchronous coordinate system at time t; i dHC.add (t),△i qHC.add (t) is the additional control variable of the current of the VSC-HVDC sending end controller in the rotating synchronous coordinate system at time t; △u dcH (t) is the DC side voltage of the VSC-HVDC sending-end converter station in the rotating synchronous coordinate system at time t, and λ1, λ2, λ3, λ4, and λ5 are the coefficients that weigh each other in the reward function.

[0311] Step 4.2.2 Environment state set

[0312] Subsynchronous oscillation is more obvious in the electrical quantities of the VSC-HVDC sending-end controller, so the environmental state can be defined as the oscillation amplitude of the electrical quantities of the VSC-HVDC sending-end controller. The environmental state set is set as:

[0313] s t ={△u dHC (t),△u qHC (t),△i dHC (t),△i qHC (t)}(4.2.1)

[0314] Step 4.2.3 Action Set

[0315] In order to suppress the oscillation of the wind power flexible direct current grid-connected system, the VSC-HVDC sending end controller can be improved, additional damping control can be added, and additional damping control signals can be injected into the control system. dHC ,i qHC Additional control is performed, so the action set can be defined as the additional damping control signal injected into the control system. The available action set is set as:

[0316] a t ={i dHC.add (t),i qHC.add (t)}(4.2.2)

[0317] Step 4.2.4 Reward function design

[0318] The reward function is an important driving signal for the agent to complete the exploration of the optimal action strategy. The oscillation amplitude of each controlled electrical quantity in the VSC-HVDC controller is the key to oscillation suppression. Therefore, the agent reward function is designed as:

[0319] R t =-(λ1△u dHC (t)+λ2△u qHC (t)+λ3△i dHC (t)+λ4△i qHC (t)+λ5△u dcH (t))(4.2.3)

[0320] Among them, λ1, λ2, λ3, λ4, and λ5 are coefficients that weigh each other in the reward function, and their values ​​are obtained through continuous trials and modifications during the training process.

[0321] The embodiment of the present invention provides a subsynchronous oscillation suppression optimization system for a wind-storage grid-connected system based on a deep deterministic policy gradient algorithm, comprising:

[0322] Interconnection module: used to establish interconnection models of flexible direct current transmission system, direct-drive wind farm and wind farm-flexible direct current system by adopting flexible direct current transmission VSC-HVDC access grid topology;

[0323] Energy storage module: used to introduce energy storage devices to build a topological structure based on wind storage system;

[0324] MFAC module: used to establish the model-free adaptive controller MFAC and algorithm;

[0325] Best Action Module: It is used to solve the decision-making process using a deep deterministic policy gradient algorithm and obtain the best action strategy through deep machine learning of multiple samples under different wind speed conditions.

[0326] An embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, any one of the methods for optimizing the suppression of subsynchronous oscillations in a wind-storage grid-connected system is implemented.

[0327] An embodiment of the present invention provides a storage medium storing a computer program. When the computer program is executed by a processor, any one of the methods for optimizing the suppression of subsynchronous oscillations in a wind-storage grid-connected system is implemented.

Claims

1. A subsynchronous oscillation suppression optimization method for a wind-storage grid-connected system, characterized in that: The following steps are involved: (1) Using the VSC-HVDC grid access topology, a VSC-HVDC system, a direct-drive wind farm, and a wind farm-VSC system interconnection model are established; (2) Introducing energy storage devices to construct a topological structure based on wind storage system; (3) Establish model-free adaptive controller MFAC and algorithm; (4) The decision-making process is solved using a deep deterministic policy gradient algorithm, and the optimal action strategy is obtained through deep machine learning of multiple samples under different wind speed operating conditions.

2. The subsynchronous oscillation suppression optimization method of a wind-storage grid-connected system according to claim 1 is characterized in that: Step (1) The formula is as follows: s△X=A△X 3. The method for optimizing subsynchronous oscillation suppression in a wind-storage grid-connected system according to claim 1, characterized in that: Step (2) comprises the following steps: (21) connecting the energy storage device to the DC line between the wind farm and the main grid through a DC / DC converter; (22) Using DC / DC converter for inner loop control; (23) Establish a virtual inertia control strategy based on the DC capacitor electric field energy and the differentiated rotor kinetic energy of the wind turbine.

4. A subsynchronous oscillation suppression optimization method for a wind-storage grid-connected system according to claim 3, characterized in that: Step (23) comprises the following steps: (231) A virtual inertia control strategy based on capacitor energy storage is established, and the formula is as follows: The inertia response process of the synchronous machine is described as: Where: H is the inertia time constant of the synchronous machine; f is the actual frequency of the AC power grid; P M is the mechanical power; P E is the electromagnetic power. Ignoring the active power loss of the line, the change of DC voltage is used to reflect the power imbalance relationship between the rectifier side and the inverter side, as shown in the following formula: Where: C eq is the equivalent capacitance of MMC; U dc is the measured DC voltage; P wind is the output power of the wind farm; P grid is the active power delivered to the grid side; Combining equations (2.3.1) and (2.3.2), we can obtain: Where: H MMC is the virtual inertia time constant of MMC; Integrate both sides of the above equation and simplify to get: Where: U dc0 is the rated DC voltage in steady state; ΔU dc =U dc -U dc0 is the deviation of DC voltage; f is the actual frequency of AC power grid; f0 is the rated frequency of AC power grid; Limit the DC voltage deviation to ±5%, ignore the quadratic term of the voltage deviation, and combine formula (2.3.5) to obtain Where: △f=f-f0 is the frequency deviation of the AC power grid, k c is the coupling coefficient between the DC voltage deviation and the frequency deviation; By introducing the receiving-end grid frequency deviation into the GSMMC DC voltage control link, a coupling relationship between DC voltage and frequency is established, so that it can actively participate in the grid frequency regulation and transmit the frequency fluctuation information to the DC side, providing response conditions for the wind farm to actively support the inertia. The formula is as follows: IN dcref =U dc0 +k c △f(2.3.7) Where: U dcref Set the DC voltage reference value for GSMMC. From the above formula, we can see that ΔU dc The coupling coefficient k between Δf c The larger the U dcref The larger the value, the easier it is to reach the DC voltage limit, resulting in the DC voltage being unable to transmit complete receiving-end grid frequency information; according to the maximum allowable DC voltage deviation ΔU dcmax The upper limit of the allowable frequency fluctuation Δf max , design k c The value of Then the virtual inertia time constant of MMC can be obtained as (232) A virtual inertia control strategy based on the kinetic energy of the fan rotor is established; the details are as follows: (233) A primary frequency modulation control strategy based on variable speed control is established; the formula is as follows: Where: η is the voltage regulation coefficient; kp is the coupling coefficient between the speed regulation and the DC voltage deviation, and its value is (234) A primary frequency modulation strategy based on pitch angle control is established, and the formula is as follows: (235) After the wind farm participates in the primary frequency regulation, the grid frequency deviation tends to a constant value. If the wind turbine rotor speed does not reach its limit at this time, after receiving the dispatch instruction, the DC voltage deviation generates an additional pitch angle Δβ through the proportional integral link. The active output of the wind farm is adjusted only by changing the pitch angle, eliminating the frequency deviation that still exists after the primary frequency regulation.

5. The method for optimizing subsynchronous oscillation suppression in a wind-storage grid-connected system according to claim 1, characterized in that: Step (3) comprises the following steps: (31) The relationship between the system output and input is dynamically linearized by updating the partial derivatives, that is, the equivalent dynamic linearization model of the output at the next moment is established by using the compact dynamic linearization CFDL method; then the control structure of the MFAC is established, including: the pseudo partial derivative estimator, which continuously updates the pseudo partial derivative estimate and the system output estimate at this moment through the I / O data at the current moment, denoted as and A one-step forward predictive controller obtains the unlimited control output u0(k) according to a one-step forward weighted predictive control algorithm suitable for oscillation suppression; an output limiter and a compensator, the limiter constrains the amplitude of the controller output, obtains the limited control signal u(k) and outputs it to the controlled system, and the compensator generates a compensation signal ξ(k) to maintain the boundedness of the tracking error; (32) Establish the NMC algorithm design, which is as follows: Pseudo partial derivative estimation algorithm, establish the following system output estimation algorithm based on CFDL format: Where: is the estimated value of the system output signal y(k); is the estimated error of the output, and γ is the error gain located at (0,1); therefore, the adaptive iterative algorithm for the pseudo partial derivative estimate is: Where: Γ(k)=η / (△u(k) 2 +μ) represents the weight coefficient of the pseudo partial derivative step change. The larger Γ(k) is, The faster the convergence speed is, the higher the convergence speed is. If Γ(k) is too large, the system is prone to The convergence speed is too fast and it becomes unstable. η is a positive constant representing the step size factor of the pseudo partial derivative estimation algorithm. μ is a positive constant representing the penalty coefficient of the change of the pseudo partial derivative to determine the constraint strength of the pseudo partial derivative; F = 1-γ, located in the interval (0,1); In order to improve the ability to track time-varying parameters during the iteration process, the following algorithm reset mechanism is constructed: Among them: ε is a sufficiently small positive number, and is proven to be bounded. Controller one-step-forward prediction algorithm: First, the MFAC control criterion function is established as: Where: y * represents the expected output of the controlled system (that is, the reference value of the input signal of the MFAC), λ is a positive number, which represents the penalty coefficient of the control criterion function, so that |u(k)-u(k-1)| is expected to be 0; since the input signal u(k) needs to be reduced to 0 when the subsynchronous oscillation is suppressed and the wind power system returns to steady-state operation, the following new control criterion function is established: Calculate the partial derivative of (3.2.4) with respect to u(k) and set it equal to 0 to obtain the one-step forward predictive control algorithm of the traditional MFAC: Among them, for the problem of suppressing subsynchronous oscillations, the expected output y * is 0; λ is a positive number, and u(k) is expected to be 0; Substituting algorithm (3.2.1) into function (3.2.5), we get: For equation (3.2.6), find the partial derivative with respect to u(k), and set the partial derivative to 0, and we get: The introduction of step factors ρ0, ρ1 (0<ρ0, ρ1≤1) makes the algorithm more flexible: Control output limit constraint and compensation algorithm: Add constraint u(k)∈[u min ,u max ]: Where: u0(k) is the original output of the controller before the constraint. In order to ensure the boundedness of the system tracking error, a compensation signal ξ(k) is introduced into u0(k). The iterative algorithm is designed as follows: Then u0(k) is improved to (33) Based on stability analysis, the adjustable control parameters λ, ρ0, ρ1, μ, η, γ, 6. The method for optimizing subsynchronous oscillation suppression in a wind-storage grid-connected system according to claim 1, characterized in that: Step (4) is as follows: convert the subsynchronous oscillation suppression problem into a Markov decision process, design the overall framework of the oscillation suppression method, the environmental state set, the feasible action set and the reward function, use the deep deterministic policy gradient algorithm to solve its decision process, and obtain the optimal action strategy through deep machine learning of multiple samples under different wind speed operating conditions; The network structure includes four deep neural networks: Actor online policy network, Actor target policy network, Critic online Q value network and Critic target Q value network. The online network and the target network have the same architecture. During the training process, the update of the Actor online network and Critic online network parameters θ and ω is achieved by gradient descent of the loss function and minimization of the loss function, respectively. The formula is as follows: Where: Q(s i ,a i ,ω) is the current Q value; s i ,a i are the state and action quantities at time i respectively; m is the number of samples sampled from the experience pool each time.

7. A subsynchronous oscillation suppression optimization system for wind-storage grid-connected system based on deep deterministic policy gradient algorithm, characterized in that: include: Interconnection module: used to establish interconnection models of flexible direct current transmission system, direct-drive wind farm and wind farm-flexible direct current system by adopting flexible direct current transmission VSC-HVDC access grid topology; Energy storage module: used to introduce energy storage devices to build a topological structure based on wind storage system; MFAC module: used to establish the model-free adaptive controller MFAC and algorithm; Best Action Module: It is used to solve the decision-making process using a deep deterministic policy gradient algorithm and obtain the best action strategy through deep machine learning of multiple samples under different wind speed operating conditions.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, a subsynchronous oscillation suppression optimization method for a wind-storage grid-connected system according to any one of claims 1 to 6 is implemented.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, a method for optimizing subsynchronous oscillation suppression in a wind-storage grid-connected system according to any one of claims 1 to 6 is implemented.

Images