Multi-time scale reactive voltage coordination control method for offshore wind plant

Abstract

The invention discloses a multi-time-scale reactive voltage coordination control method for an offshore wind power plant. The method comprises the following steps: collecting actually measured data of the wind power plant; inputting measured data of the wind power plant into a reactive power-voltage coordination control model constructed based on a depth deterministic gradient strategy to obtain a model output action; determining reactive power output values of the wind turbine generator and the reactive power compensation device according to the model output action, and performing coordination control on the wind power plant according to the determined reactive power output values of the wind turbine generator and the reactive power compensation device. According to the method, an interaction mechanism of continuous trial and error between individuals and the environment in a deep reinforcement learning algorithm is adopted, the operation control instruction of the wind power plant is obtained from input data by using good generalization ability of the deep neural network, and reactive voltage regulation and control of the wind power plant in an unknown environment are realized. According to the established objective function, the reactive power regulation equipment with different response time can be ensured to be reasonably regulated and controlled, and more dynamic reactive power reserve can be reserved for potential disturbance.

Description

Technical Field

[0001] The present invention belongs to the technical field of wind power control, and in particular relates to a multi-time scale reactive power and voltage coordinated control method for an offshore wind farm. Background Art

[0002] There are many reasons that affect voltage fluctuations, the most common of which are wind turbine type, wind conditions, grid load and control strategy. The fundamental reason is still wind conditions. Random changes in wind speed will cause changes in power, which in turn affects voltage fluctuations. These external disturbances may cause grid voltage fluctuations or even cross the boundary, and voltage control must be performed. Therefore, wind farms must have a certain reactive power support capability and coordinate the reactive power output of each reactive compensation device through control algorithms to support the grid connection point voltage. There are many ways to regulate voltage in wind farms. Among them, the most concerned is to establish a multi-level voltage coordination control model through reactive compensation devices with single or multiple characteristics, and use sensitivity coefficients to distribute reactive power. This method is only based on the current state of the system and does not consider the coordination between devices at different time scales and the impact of future states, so control lag and other problems often occur. The reactive power source of a wind farm includes wind turbines and wind farm reactive compensation devices. However, most of the current wind farms use reactive compensation devices to regulate reactive power, and the reactive regulation capacity of wind turbines has not been fully utilized.

[0003] In the study of reactive power and voltage control of wind farms, the commonly used method is to allocate reactive power according to the sensitivity coefficient, that is, the difference between the current grid-connected point voltage and the voltage setting value is controlled by PI to obtain the total reactive power demand of the wind farm, and then the reactive power is allocated according to the sensitivity coefficient between the reactive power generated by each wind turbine and each reactive compensation device and the grid-connected point voltage. This method is simple and easy to implement. However, this method only considers the static relationship between reactive power compensation equipment and voltage, and simply equates wind turbines and other devices as different reactive sources, and rarely considers the differences in response characteristics between them. In addition, the control method is mainly based on the optimization of the current state, ignoring the impact of the reactive power regulation process of these devices on the future state of the system. For continuously controlled devices such as wind turbines, the response time of reactive power output is in the second level (1 to 10 seconds); static VAR compensator (SVC) / static VAR generator (SVG) acts quickly, with a time constant in the millisecond level (SVC: 50 to 200ms, SVG: 20 to 100ms). When these reactive devices with different time scales and different working adjustment conditions are placed in the same wind farm to work, it is inevitable to coordinate them. When the voltage of the power grid fluctuates, sufficient reactive power needs to be provided to support voltage stability. In addition to the wind turbine itself being able to provide reactive support, there are other reactive compensation devices. Therefore, how to coordinate reactive devices with different time constants and different adjustment characteristics is the core difficulty of voltage control in the wind farm. Static VAR compensators, static synchronous compensators and wind turbines are continuous adjustment devices, among which SVC and SVG are relatively high in cost and small in capacity. Although the wind turbine has a large capacity, it is not suitable to output too much reactive power in order to improve the fault ride-through capability. Therefore, SVC, SVG and wind turbines can be used to compensate for small fluctuations, transients and dynamic processes between wind power and the power grid. SVC / SVG and wind turbines have different control targets and control time constants, which will have mutual influences during the control process and even cause unreasonable reactive flow. Therefore, it is urgent to study their coordinated control strategies from different scales such as targets, time and space. The present invention is mainly aimed at the reactive voltage coordination of different time scales between SVC / SVG and wind turbines.

[0004] The Model Predictive Control (MPC) method, which has been widely used in the field of industrial control, can effectively avoid the problem that reactive power optimization of wind farms is highly dependent on predictions and is affected by environmental factors. Using the MPC method to optimize the scheduling of buildings can effectively solve the problem of large deviations between the optimization scheduling scheme and the actual operation scenario caused by data prediction errors, and has strong robustness in the environment of prediction uncertainty. Dynamic modeling of wind farms for MPC is a challenge, and it is difficult to obtain all parameters. Accurate dynamic models are the prerequisite for designing efficient control strategies. In the MPC control optimization method, accurate modeling and identification of objects are often the most difficult part. Due to the limitations of the MPC method that relies on the state space model, machine learning methods circumvent the shortcomings of the MPC method by utilizing historical data of wind farms. The classic method of deep reinforcement learning in machine learning, DQN, combines deep learning with value-based reinforcement learning algorithms, but can only output discrete actions and cannot be applied in continuous action spaces. Since the state space of wind farm wind turbines and SVC regulation is continuous, if the operation control commands are discretely operated, the strategy training may be insufficient due to the time step being too long, or the curse of dimensionality may be caused by the step being too short. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a multi-time scale reactive power and voltage coordinated control method for an offshore wind farm.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] On the one hand, the present invention provides a multi-time scale reactive power and voltage coordinated control method for an offshore wind farm, comprising the following steps:

[0008] Step S1: Collecting measured data of the wind farm;

[0009] Step S2: inputting the measured data of the wind farm into the reactive power-voltage coordinated control model constructed based on the deep deterministic gradient strategy to obtain the model output action;

[0010] Step S3: Determine the reactive output values ​​of the wind turbine generator set and the reactive compensation device according to the model output action, and coordinately control the wind farm according to the reactive output values ​​of the wind turbine generator set and the reactive compensation device.

[0011] Furthermore, the wind farm measured data includes: the reactive power Q output by the wind turbine w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC .

[0012] Furthermore, the process of constructing the reactive power-voltage coordinated control model based on the deep deterministic gradient strategy includes the following steps:

[0013] Step A1: Determine model constraints;

[0014] Step A2: Define the model’s state space, action space, and reward function;

[0015] Step A3: Construct Actor-Critic network;

[0016] Step A4: training the Actor-Critic network.

[0017] Furthermore, the model constraints include: the reactive power constraints output by the wind turbine and the reactive power compensation device, the grid connection point voltage V PCC Constraints and reactive power regulation step constraints of wind turbines and reactive power compensation devices;

[0018] The reactive power constraints of the wind turbine and the reactive power compensation device are:

[0019]

[0020] Among them, Q max Indicates the upper limit of reactive power output, Q min represents the lower limit of reactive power output, subscript w represents wind turbine, and subscript s represents reactive power compensation equipment;

[0021] The wind farm grid connection point voltage V PCC The constraints are:

[0022]

[0023] in, represents the upper limit of node voltage, Indicates the lower limit of node voltage;

[0024] The reactive power regulation step length constraint condition of the wind turbine and the reactive power compensation device is:

[0025]

[0026] Among them, ΔQ max Indicates the maximum reactive power regulation step of the device.

[0027] Furthermore, the state space of the model includes: the reactive power Q output by the wind turbine w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage VPCC ;

[0028] The action space of the model includes: the reactive power Q output by the wind turbine w The reactive power Q output by the reactive power compensation device s ;

[0029] The reward function of the model is:

[0030] reward=-λ·|V ref -V real |-Q reserve

[0031] Among them, reward is the reward value, λ is the weight ratio, V ref is the reference voltage of the wind farm grid connection point, V real is the actual voltage at the wind farm grid connection point, Q reserve It is the remaining reactive power reserve of the reactive power compensation device.

[0032] Furthermore, in the Actor-Critic network, the Actor network includes: an input layer, a fully connected layer, an ELU function activation layer, and a tanh function output layer;

[0033] The Critic network includes: an input layer, a fully connected layer, a ReLU function activation layer, and an output layer.

[0034] Furthermore, the model training process in step A4 specifically includes:

[0035] Initialize the Actor network μ(s|θμ) and the Critic network Q(s, a|θ Q ) parameters θμ and θ Q , where s is the state space and a is the action space;

[0036] Initialize the parameters μ′ and Q′ of the target Actor network and the target Critic network;

[0037] Initialize the experience replay buffer R;

[0038] Obtain the current wind farm measured data, including the reactive power Q output by the wind turbines w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC ;

[0039] The measured data of the wind farm is used as the environmental state s t Enter the current Actor network and select action a t , action selection formula:

[0040]

[0041] in, is random noise;

[0042] The selected action a t The corresponding reactive power Q output by the wind turbine w The reactive power Q output by the reactive power compensation device s Act on the wind farm, obtain the measured data of the wind farm at the next moment, and use the measured data of the wind farm at the next moment as the next state s t+1 ;

[0043] Calculate the reward r at time t according to the reward function t ;

[0044] Transfer the state (s t , a t , r t ,s t+1 ) is stored in the experience replay buffer R;

[0045] Continue the above steps until the number of state transitions stored in the experience replay buffer R reaches a first preset threshold;

[0046] Randomly extract state transition data from the experience replay buffer R, and calculate the target value yt through the extracted state transition data. The calculation formula is:

[0047] y t =r t +γQ′(s t+1 ,μ′(s t+1 ,θ μ′ )|θ Q′ )

[0048] Among them, γ is the discount factor, θ μ′ is the parameter of the target Actor network, θ Q′ are the parameters of the target Critic network;

[0049] According to the calculated target value y i Calculate the loss value of the Actor network and the Critic network through the loss function;

[0050] Update the parameters of the Actor network and the Critic network through back propagation according to the loss value;

[0051] The parameters of the Actor network and the Critic network are repeatedly trained and updated until the calculated loss values ​​of the Actor network and the Critic network tend to be stable and are less than the second preset threshold.

[0052] Furthermore, the network structures of the target Actor network and the target Critic network are the same as those of the Actor network and the Critic network, and the initial parameters of the target Actor network and the target Critic network are the same as those of the Actor network and the Critic network. The parameter update formula of the target Actor network and the target Critic network is:

[0053] θ μ′ ←τθ μ +(1-τ)θ μ′

[0054] θ Q′ ←τθ Q +(1-τ)θ Q′

[0055] Here, τ is a constant less than 1.

[0056] Furthermore, the loss function of the Critic network is:

[0057]

[0058] Among them, L Q is the loss value of the Critic network, and N is the number of extracted state transition data;

[0059] The loss function of the Actor network is:

[0060]

[0061] Among them, L μ is the loss value of the Actor network.

[0062] Furthermore, the action a selected by the Actor network t Satisfy model constraints.

[0063] Compared with the prior art, the present invention has the following advantages:

[0064] The present invention provides a multi-time scale voltage coordination control strategy based on deep reinforcement learning for the voltage support and reactive power compensation problems of offshore wind farms. This method can coordinate reactive power compensation devices with different time constants in offshore wind farms. The deep deterministic policy gradient method (DDPG) is based on the actor-critic framework and can handle regulation strategy problems in continuous and high-dimensional action spaces. In DDPG reactive voltage control, decisions can no longer be made based only on the current time section, but can be controlled based on the overall optimal performance in the future time window, and according to the established objective function, it can not only ensure that these reactive power regulation devices with different response times are reasonably regulated and controlled, but also reserve more dynamic reactive power reserves for potential disturbances. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The figure is a flow chart of the method of the present invention.

[0066] Figure 2 It is a flowchart of the multi-time scale voltage coordinated control method of the present invention; DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0068] Embodiment 1:

[0069] This embodiment adopts a multi-time scale reactive power and voltage coordinated control method for an offshore wind farm, such as Figure 1 As shown, the following steps are included:

[0070] Step S1: Collect wind farm measured data, which includes: reactive power Q output by wind turbines w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC .

[0071] Step S2: inputting the measured data of the wind farm into the reactive power-voltage coordinated control model constructed based on the deep deterministic gradient strategy to obtain the model output action;

[0072] Step S3: Determine reactive output values ​​of the wind turbine generator set and the reactive compensation device according to the model output action, and coordinately control the wind farm according to the determined reactive output values ​​of the wind turbine generator set and the reactive compensation device.

[0073] Among them, the construction process of the reactive power-voltage coordinated control model based on the deep deterministic gradient strategy includes the following steps:

[0074] Step A1: Determine model constraints:

[0075] Determine the equality and inequality constraints: Compared with wind turbines, SVG has an extremely fast response speed, which is in the millisecond level. Therefore, SVG outputs reactive power and quickly improves the reactive support capacity of the system. When the output capacity of SVG reaches the set reference value (its reactive output does not exceed its capacity), it will no longer output reactive power, and the wind turbine will perform reactive compensation and make slight adjustments to the voltage. The reactive output of wind turbines has dynamic boundary conditions, that is, the reactive power limit of the unit output is related to the active output, and the reactive limits under different active output states are not the same. The reactive allocation strategy needs to fully consider the characteristics and constraints of each reactive source in the field, including reactive boundaries, reactive response characteristics, voltage constraints, etc. The reactive output of wind turbines has dynamic boundary conditions, that is, the reactive power limit of the unit output is related to the active output, and the reactive limits under different active output states are not the same.

[0076]

[0077] In the formula, Q max Indicates the upper limit of reactive power output, Q min It represents the lower limit of reactive power output, the subscript w represents the wind turbine, and the subscript s represents the reactive power compensation equipment SVC / SVG.

[0078] The Technical Specifications for Grid Access to Wind Farms stipulate that the PCC voltage at the wind farm grid connection point must be controlled between -3% and +7% of the rated value. Therefore, the PCC voltage of the wind farm should be controlled within this range.

[0079]

[0080] Where V max Represents the upper limit of node voltage, V min Indicates the lower limit of the node voltage.

[0081] In each round of control, the reactive power adjustment step of the reactive power compensation equipment should be smaller than its maximum adjustment step:

[0082]

[0083] Where ΔQ max Indicates the maximum reactive power regulation step of the device.

[0084] Step A2: Define the model’s state space, action space, and reward function:

[0085] The desired physical state and the set target are summarized as state si , and the reactive power output of the wind turbine and SVC / SVG equipment is action, denoted as a i The reward function for reactive power control is defined as r i (s i , a i ), the strategy is π(s i |a i )=p(A t =a i |S t =s i ,), the Markov decision process MDP of this problem can be summarized as a tuple<S,A,p,r,γ> (γ is a discount factor that affects the speed of strategy updating, A is the action of the wind farm, r is the reward of the wind farm, and S is the overall environmental state).

[0086] Define the state space: The reactive power output of the wind turbine and SVG is a continuous state variable. The state variable of the system is the reactive power Q output by the wind turbine and SVG. w and Q s 、SVC / SVG actual output voltage V s , grid connection point voltage V PCC The continuous state space can be processed directly using the DDPG algorithm without discretization. The DDPG algorithm uses a neural network to approximate the Q value function, which can estimate the continuous state space and thus handle the continuous state space in practical problems. The state space of the model includes: the reactive power Q output by the wind turbine w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC .

[0087] Define the action space: Use continuous action variables to control the reactive power Q output by the wind turbine and SVG w and Q s , set the maximum and minimum values. Allocate the overall reactive power demand to the wind turbines and reactive power compensation devices. The action space of the model includes: the reactive power Q output by the wind turbines w The reactive power Q output by the reactive power compensation device s .

[0088] Define the reward function: In order to make the agent as close as possible to the desired control target, it is necessary to define a reward function to guide the learning process. The learning goal is to make the reward as large as possible. Considering the constraints of the model, if the constraints are violated, a penalty mechanism is set to deduct the reward. The reward function setting comprehensively considers the voltage deviation and reactive power reserve of the grid connection point. In addition, the different time scales of the wind turbine and SVG will lead to delayed rewards. This is the main problem of training convergence and hyperparameter adjustment. The reward function of the model is:

[0089] reward=-λ·|V ref -V real |-Q reserve

[0090] Among them, reward is the reward value, λ is the weight ratio, V ref is the reference voltage of the wind farm grid connection point, V real is the actual voltage at the wind farm grid connection point, Q reserve It is the remaining reactive power reserve of the reactive power compensation device.

[0091] Step A3: Construct Actor-Critic network:

[0092] For the Actor-Critic network, just add a hidden layer between the input layer and the output layer. For the Actor network, the first two layers are composed of linear layers and activated by the ELU function. The third layer needs to standardize the output action between [-1,1], so the tanh function is used. For the Critic network, only ReLU is needed for activation in the input layer and hidden layer, and the output layer can directly use the linear layer. The expressions of the ReLU function, ELU function and tanh function are:

[0093]

[0094] In the Actor-Critic network, the Actor network includes: an input layer, a fully connected layer, an ELU function activation layer, and a tanh function output layer;

[0095] The Critic network includes: an input layer, a fully connected layer, a ReLU function activation layer, and an output layer.

[0096] Step A4: Training the Actor-Critic network. The training process specifically includes the following steps:

[0097] Initialize the Actor network μ(s|θμ) and the Critic network Q(s, a|θ Q ) parameters θμ and θ Q , where s is the state space and a is the action space;

[0098] Initialize the parameters μ′ and Q′ of the target Actor network and the target Critic network;

[0099] Initialize the experience replay buffer R;

[0100] Obtain the current wind farm measured data, including the reactive power Q output by the wind turbines w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC ;

[0101] The measured data of the wind farm is used as the environmental state s t Enter the current Actor network and select action a t , action selection formula:

[0102]

[0103] in, is random noise;

[0104] The selected action a t The corresponding reactive power Q output by the wind turbine w The reactive power Q output by the reactive power compensation device s Act on the wind farm, obtain the measured data of the wind farm at the next moment, and use the measured data of the wind farm at the next moment as the next state s t+1 ;

[0105] Calculate the reward r at time t according to the reward function t ;

[0106] The state, action, reward and next state (s) obtained from each interaction t , a t , r t ,s t+1 ) is stored as a one-time experience in the experience replay buffer. After a certain amount of experience is stored in the buffer, a batch of experience is randomly extracted from the buffer and the neural network is updated in a random order. Since the replay buffer contains all the historical experience data of the agent's interaction with the environment, it can avoid short-term correlation and improve the performance of the agent's learning. Moreover, since this method uses all the experience data instead of only the most recently acquired experience data, the network is updated and the experience used is relatively independent, which reduces the coupling of the parameters in the network and prevents problems such as overfitting and instability.

[0107] Continue the above steps until the number of state transitions stored in the experience replay buffer R reaches a first preset threshold;

[0108] Randomly extract state transition data from the experience replay buffer R, and calculate the target value yt through the extracted state transition data. The calculation formula is:

[0109] y t =r t +γQ′(s t+1 ,μ′(s t+1 ,θ μ′ )|θ Q′ )

[0110] Where γ is the discount factor, θ μ′ is the parameter of the target Actor network, θ Q′ are the parameters of the target Critic network;

[0111] According to the calculated target value y i Calculate the loss value of the Actor network and the Critic network through the loss function;

[0112] Update the parameters of the Actor network and the Critic network through back propagation according to the loss value;

[0113] The parameters of the Actor network and the Critic network are repeatedly trained and updated until the calculated loss values ​​of the Actor network and the Critic network tend to be stable and are less than the second preset threshold.

[0114] Among them, the network structure of the target Actor network and the target Critic network is the same as that of the Actor network and the Critic network, and the initial parameters of the target Actor network and the target Critic network are the same as those of the Actor network and the Critic network. The parameter update formula of the target Actor network and the target Critic network is:

[0115] θ μ′ ←τθ μ +(1-τ)θ μ′

[0116] θ Q′ ←τθ Q +(1-τ)θ Q′

[0117] Here, τ is a constant less than 1.

[0118] The loss function of the Critic network is:

[0119]

[0120] Among them, L Q is the loss value of the Critic network, and N is the number of extracted state transition data;

[0121] The loss function of the Actor network is:

[0122]

[0123] Among them, L u is the loss value of the Actor network.

[0124] The measured data of the wind farm is used as the environmental state s t Enter the current Actor network and select action a t , the selected action a t Satisfy model constraints.

[0125] Embodiment 2:

[0126] The parts not mentioned in this embodiment are the same as those in Embodiment 1.

[0127] like Figure 2 As shown, the reactive power coordinated control of wind farms and SVC / SVG is a fifth-order state equation with two control inputs. Therefore, the reactive power voltage coordinated control of wind turbines and SVC / SVG is a single-input and two-output controller. A multi-time scale voltage coordinated control method based on deep reinforcement learning includes the following steps:

[0128] (1) Reactive power compensation is performed on the grid-connected system of offshore wind farms to clarify the reactive power that is lacking or redundant in the system. The acquisition or absorption of the reactive power shortage or redundancy of the system can start from the voltage deviation of the grid-connected point. The node deviation is converted into the reactive power shortage through the proportional integral link. The voltage at the wind power grid-connected point is adjusted to a qualified range. The measured voltage value at the grid-connected point is subtracted from the reference value. The steady-state static error is eliminated through PI control, and a dead zone is set to prevent the equipment from operating too frequently, thereby obtaining the total reactive power shortage of the system. When the voltage at the grid-connected point fluctuates greatly due to the change of wind speed, the actual voltage value V can be calculated. real The reference value V set by the simulation system ref The difference between the two is used by the PI controller to obtain the reactive power compensation capacity required in real time during the operation of the entire wind farm.

[0129] (2) Construct a mathematical model for allocating overall reactive power demand to wind turbines and reactive power compensation devices. Compared with wind turbines, SVG has an extremely fast response speed, which is at the millisecond level. Therefore, SVG outputs reactive power and rapidly improves the reactive power support capacity of the system. When the SVG output capacity reaches the set value, it will no longer generate reactive power. The wind turbine will perform reactive power compensation and make slight adjustments to the voltage. The reactive power output of the wind turbine should ensure that its power factor meets the corresponding constraints. Reasonable use of the reactive power capacity of SVG and wind turbines can achieve fine control of the voltage at the wind farm grid connection point.

[0130] (3) Use deep reinforcement learning methods to allocate the overall reactive power demand to wind turbines and reactive power compensation devices. The Deep Deterministic Policy Gradient (DDPG) method is based on the actor-critic framework and can handle control strategy problems in continuous and high-dimensional action spaces. The main difference between DDPG and DQN is that DQN uses the target Q function to obtain the argmax function of greedy actions, while DDPG uses a target-determined policy function that is trained to approximate greedy actions. Compared with DQN, DDPG uses a policy network to fit the policy function in terms of action output, and directly outputs specific actions instead of the probability of actions. It can be solved in a high-dimensional environment of continuous actions, solving the problem of outputting continuous quantities.

[0131] Specifically, in step (1).

[0132] In this embodiment, a wind farm consisting of 6 1.5MW wind turbines is connected to a 25kV power system and outputs electricity to a 120kV grid through a 25km 25kV feeder. A 9MW wind farm is simulated by 3 pairs of 1.5MW wind turbines. The wind turbine uses a squirrel cage induction generator (IG). The stator winding is directly connected to the 60Hz grid and the rotor is driven by a variable pitch wind turbine. The pitch angle is controlled to limit the generator output power to the rated value when the wind speed exceeds the rated speed (9m / s). In order to generate electricity, the IG speed must be slightly higher than the synchronous speed. The speed is about 1pu at no load and 1.005pu at full load. Each wind turbine has a protection system for monitoring voltage, current and machine speed. The reactive power absorbed by the wind turbine is partially compensated by a capacitor bank connected to the low-voltage bus of each wind turbine (400kvar for each pair of 1.5MW wind turbines). To keep the 25kV voltage of bus B25 close to 1pu, the rest of the reactive power is provided by a 3Mvar SVG with a droop setting of 3%.

[0133] Table 1 shows the hyperparameters selected in the Actor-Critic network. The keras library is used to implement this fully connected neural network.

[0134] Table 1 Actor-Critic network hyperparameter settings

[0135] Hyperparameters set up Activation Function Relu Learning Rate 0.001 Hidden Layer 2 Number of hidden layer neurons 24、24 Optimizer Adam Batch size 32 Discount Factor 0.1

[0136] Open the Wind Farm module and look at Wind Turbine 1. Open the Wind Turbines menu and look at the two sets of parameters specified for the wind turbines and generators. Each wind turbine module represents two 1.5 MW turbines. Open the Turbines menu, select Turbine Data and check Display Wind Turbine Power Characteristics. The mechanical power of the turbines as a function of wind speed is displayed for wind speeds between 4 m / s and 10 m / s. The rated wind speed for producing the rated mechanical power (1 pu = 3 MW) is 9 m / s. Both the wind turbine model and the SVG are phase models, which allow for transient stability type studies with longer simulation times. In this example, the observation time of the system is 20 seconds. The wind speed of each wind turbine is controlled by the Wind 1 to Wind 3 modules. Initially, the wind speed is set to 11 m / s and then reduced to 3 m / s over 3 seconds, starting at t = 2 seconds for Wind Turbine 1. The same gust of wind is applied to "Turbine 2" and "Turbine 3" with a delay of 2 and 4 seconds respectively.

[0137] Simulate the response of the turbines to wind speed changes: Start the simulation and observe the signals on the "Wind Turbines" monitor, monitoring the active and reactive power of each wind turbine, the generator speed, the wind speed and the pitch angle. The active power of each pair of wind turbines starts to increase smoothly (together with the wind speed) and reaches the rated value of 3MW after about 2 seconds. Initially, the output power exceeds 3MW and the pitch angle is increased from 0 to 10 degrees to restore the output power to the rated value, and then the pitch angle of the turbine blades is zero degrees. As the generated active power decreases, the absorbed reactive power also increases. At rated power, each pair of wind turbines produces 1.47Mvar. At a wind speed of 3m / s, the total output power measured on the B25 bus is 6.35MW, and the SVG maintains the voltage at 1.008pu by absorbing 0.8Mvar of reactive power).

[0138] Under the condition of constant power factor control of the unit, the increase of active output of the wind farm will cause the voltage change of the wind farm grid connection point. The voltage of the wind farm grid connection point will increase accordingly with the increase of reactive output of the wind farm. When the wind farm absorbs reactive power, and the reactive power absorbed by the wind farm is greater than the charging reactive power generated by the high-voltage transmission line, the voltage of the grid connection point will decrease accordingly. When there is no wind or the wind is relatively small, the output of the entire wind farm is low, the line is lightly loaded, and the reactive power required by the system is less than the charging power provided by the transmission line. The transmission system injects reactive power into the power grid, which is capacitive and requires a certain inductive capacity to absorb the excess reactive power. When the wind farm output is large, the transmission line is overloaded, and its charging power is smaller than the reactive power consumed by the wind farm. In order to maintain the system voltage within a safe range, the reactive power demand of the transformer and the transmission line must be met by a certain capacitive reactive power compensation capacity.

[0139] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0140] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-time scale reactive power and voltage coordinated control method for an offshore wind farm, characterized in that: The following steps are involved: Step S1: Collecting measured data of the wind farm; Step S2: inputting the measured data of the wind farm into the reactive power-voltage coordinated control model constructed based on the deep deterministic gradient strategy to obtain the model output action; Step S3: Determine the reactive output values ​​of the wind turbine generator set and the reactive compensation device according to the model output action, and coordinately control the wind farm according to the reactive output values ​​of the wind turbine generator set and the reactive compensation device.

2. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 1, characterized in that: The wind farm measured data includes: reactive power Q output by wind turbines w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC .

3. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 1, characterized in that: The reactive power-voltage coordinated control model construction process based on the deep deterministic gradient strategy includes the following steps: Step A1: Determine model constraints; Step A2: Define the model’s state space, action space, and reward function; Step A3: Construct Actor-Critic network; Step A4: training the Actor-Critic network.

4. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 3, characterized in that: The model constraints include: the reactive power constraints output by the wind turbine and the reactive compensation device, the grid connection point voltage V PCC Constraints and reactive power regulation step constraints of wind turbines and reactive power compensation devices; The reactive power constraints of the wind turbine and the reactive power compensation device are: Among them, Q max Indicates the upper limit of reactive power output, Q min represents the lower limit of reactive power output, subscript w represents wind turbine, and subscript s represents reactive power compensation equipment; The wind farm grid connection point voltage V PCC The constraints are: in, represents the upper limit of node voltage, Indicates the lower limit of node voltage; The reactive power regulation step length constraint condition of the wind turbine and the reactive power compensation device is: Among them, ΔQ max Indicates the maximum reactive power regulation step of the device.

5. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 3, characterized in that: The state space of the model includes: the reactive power Q output by the wind turbine w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC ; The action space of the model includes: the reactive power Q output by the wind turbine w The reactive power Q output by the reactive power compensation device s ; The reward function of the model is: reward=-λ·|V ref -V real |-Q reserve Among them, reward is the reward value, λ is the weight ratio, V ref is the reference voltage of the wind farm grid connection point, V real is the actual voltage at the wind farm grid connection point, Q reserve It is the remaining reactive power reserve of the reactive power compensation device.

6. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 3, characterized in that: In the Actor-Critic network, the Actor network includes: an input layer, a fully connected layer, an ELU function activation layer, and a tanh function output layer; The Critic network includes: an input layer, a fully connected layer, a ReLU function activation layer, and an output layer.

7. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 3, characterized in that: The model training process in step A4 specifically includes: Initialize the Actor network μ(s|θ μ ) and Critic network Q(s, a|θ Q ) parameter θ μ and θ Q , where s is the state space and a is the action space; Initialize the parameters μ′ and Q′ of the target Actor network and the target Critic network; Initialize the experience replay buffer R; Obtain the current wind farm measured data, including the reactive power Q output by the wind turbines w , reactive power Q output by reactive power compensation device s , the actual output voltage of the reactive power compensation device V s , Wind farm grid connection point voltage V PCC ; The measured data of the wind farm is used as the environmental state s t Enter the current Actor network and select action a t , action selection formula: in, is random noise; The selected action a t The corresponding reactive power Q output by the wind turbine w The reactive power Q output by the reactive power compensation device s Act on the wind farm, obtain the measured data of the wind farm at the next moment, and use the measured data of the wind farm at the next moment as the next state s t+1 ; Calculate the reward r at time t according to the reward function t ; Transfer the state (s t , a t , r t ,s t+1 ) is stored in the experience replay buffer R; Continue the above steps until the number of state transitions stored in the experience replay buffer R reaches a first preset threshold; Randomly extract state transition data from the experience replay buffer R, and calculate the target value y through the extracted state transition data t , the calculation formula is: y t =r t +γQ′(s t+1 ,μ′(s t+1 ,the μ′ )|θ Q′ ) Where γ is the discount factor, θ μ′ is the parameter of the target Actor network, θ Q′ are the parameters of the target Critic network; According to the calculated target value y i Calculate the loss value of the Actor network and the Critic network through the loss function; Update the parameters of the Actor network and the Critic network through back propagation according to the loss value; The parameters of the Actor network and the Critic network are repeatedly trained and updated until the calculated loss values ​​of the Actor network and the Critic network tend to be stable and are less than the second preset threshold.

8. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 7, characterized in that: The target actor network and the target critic network have the same network structure as the actor network and the critic network. The initial parameters of the target actor network and the target critic network are the same as the initial parameters of the actor network and the critic network. The parameter update formula of the target actor network and the target critic network is: i μ′ ←tth μ +(1-τ)θ μ′ i Q′ ←tth Q +(1-τ)θ Q′ Here, τ is a constant less than 1.

9. A multi-time scale reactive power and voltage coordinated control method for an offshore wind farm according to claim 7, characterized in that: The loss function of the Critic network is: Among them, L Q is the loss value of the Critic network, and N is the number of extracted state transition data; The loss function of the Actor network is: Among them, L μ is the loss value of the Actor network.

10. The method for multi-time scale reactive power and voltage coordinated control of an offshore wind farm according to claim 7, characterized in that: The Actor network selects an action a t Satisfy model constraints.

Images