Offshore wind turbine short-time domain economic model predictive control method

Abstract

The application discloses a kind of offshore wind turbine short-time domain economic model predictive control method, belong to wind power generation control strategy field.The method is by establishing the state space model of variable speed wind turbine and economic stage cost function, constructs short-time domain economic model predictive control optimization problem.The core is, using reinforcement learning off-line training a parameterized terminal value function, to approximate infinite time domain economic cost, and embed it in short-time domain optimization problem.In online control, only need limited wind speed preview information, by rolling solving this optimization problem realizes closed-loop control.The application reduces the online calculation burden, meets the real-time control requirements, effectively compensates the long-time domain economic information missing under limited prediction time domain, thereby in improving wind energy capture efficiency, reduce structural fatigue load, and excellent comprehensive performance is achieved.

Description

Technical Field

[0001] This invention belongs to the field of wind power generation control strategies. Background Technology

[0002] Wind turbine generators are the core equipment in wind energy conversion systems, and their control performance directly affects wind energy capture efficiency, output power quality, structural fatigue load, and turbine lifespan. With the increasing size of wind turbine generators and the increasing complexity of operating environments, the control objectives are no longer limited to simple speed or power tracking. Instead, they need to comprehensively consider multiple requirements under different wind conditions, including wind energy capture, power regulation, drivetrain load mitigation, pitch control constraints, and real-time calculation feasibility. Therefore, achieving economical and efficient operation of wind turbine generators under conditions of random wind speed disturbances and limited predictive information is a crucial issue in the field of wind power control.

[0003] Traditional model predictive control (MMDC) methods can utilize the dynamic model and constraints of the wind turbine to calculate control inputs such as generator torque and pitch angle within a rolling optimization framework, thereby regulating the turbine's state and output power. However, traditional tracking-type MMDC typically relies on a pre-defined reference trajectory or setpoint, and its objective function primarily penalizes deviations of state and control variables relative to reference values. This makes it difficult to directly reflect the comprehensive trade-off between wind energy capture efficiency, structural fatigue loads, and operational economy. Therefore, in wind turbine control problems where economic performance is the core objective, traditional tracking-type MMDC has certain limitations.

[0004] Economic model predictive control (EMC) can directly incorporate economic performance indicators into the online optimization objective function. For example, it can include aerodynamic capture power, power generation, drivetrain fatigue load, or control execution cost as part of the optimization objective. This avoids excessive reliance on fixed setpoints and is more suitable for the economical operation control of wind turbine generators under complex wind conditions. Compared with traditional model predictive control, EMC can directly optimize the closed-loop economic performance of the unit while satisfying state and input constraints, making it an effective method for achieving efficient operation of wind turbine generators.

[0005] However, existing economic model predictive control methods still face the following problems in practical wind turbine applications. Firstly, economic model predictive control typically requires a long prediction time domain to accurately reflect the long-term economic performance of the turbine. The longer the prediction time domain, the more state variables, control variables, and constraints are involved in the optimization problem, increasing the burden of online solution and making it difficult to meet the requirements of rapid real-time control of wind turbines. Secondly, wind speed exhibits significant randomness and fluctuation. In practical engineering, only relatively reliable wind speed preview information within a short time range is usually available, making it difficult to accurately predict wind speed changes over a longer future time domain. When economic model predictive control uses a long prediction time domain, wind speed in subsequent time domains often has to be handled using average wind speed, maintaining the current wind speed, or other approximations. This introduces prediction errors and model mismatch, thereby weakening the practical effect of long-term economic optimization.

[0006] In summary, existing technologies still lack a wind turbine control method that can simultaneously address the following requirements: first, maintaining good long-term economic performance under finite wind speed preview conditions; second, avoiding the high online computational burden caused by long-prediction-time-domain economic model predictive control; third, retaining the explicit processing capability of model predictive control for unit state constraints and input constraints; and fourth, utilizing historical operating data and reinforcement learning methods to extract long-term economic information under wind speed stochasticity. Therefore, it is necessary to propose a reinforcement learning-enhanced short-term-domain economic model predictive control method for variable-speed wind turbines to improve wind energy utilization efficiency, reduce structural fatigue loads, and improve the closed-loop economic performance and real-time control feasibility of wind turbines under random wind speed conditions. Summary of the Invention

[0007] The purpose of this invention is to provide a reinforcement learning-based predictive control method for variable-speed wind turbine generators that considers finite wind speed previews. This method addresses the problems of traditional economic model predictive control relying on long prediction time domains, high online computational burden, and short-time-domain economic model predictive control lacking long-time-domain economic information. This method learns the long-time-domain economic value function of the wind turbine generator under random wind speed conditions offline through reinforcement learning, and embeds the learned value function as the terminal cost into the short-time-domain economic model predictive control optimization problem. This allows short-time-domain online optimization to achieve approximate long-time-domain economic performance while maintaining low online computational load and explicit constraint handling capabilities.

[0008] The present invention provides a short-time economic model predictive control method for offshore wind turbine generators, comprising the following steps:

[0009] S1. Based on the aerodynamic power model, transmission chain dynamics model and power generation model, establish the state space model of the variable speed wind turbine generator set. The state variables include the wind turbine speed, generator speed and transmission shaft torsion angle, and the control variables include the generator torque and blade pitch angle.

[0010] S2. Establish a stage cost function that includes the wind energy capture economic term and the transmission chain fatigue load penalty term;

[0011] S3. Based on the state-space model and stage cost function, construct an economic model predictive control optimization problem with the sum of economic stage cost and terminal cost as the objective function within a finite prediction time domain.

[0012] S4. Establish a parameterized terminal value function represented by a deep neural network to approximate the economic cost in the infinite time domain;

[0013] S5. Train the parameters of the terminal value function offline using reinforcement learning;

[0014] S6. Embed the trained terminal value function as the terminal cost into the short-time-domain economic model predictive control optimization problem, and solve the optimal control sequence online.

[0015] S7. Apply the first control input in the optimal control sequence to the wind turbine generator set, and repeatedly solve the problem to achieve closed-loop control.

[0016] Preferably, the constraints of the economic model predictive control optimization problem in step S3 include wind turbine speed constraints, generator torque constraints, pitch angle constraints, and pitch rate constraints.

[0017] Preferably, the stage cost function in step S2 is constructed by weighting and combining the wind energy capture economic term and the transmission chain fatigue load penalty term, and the optimization objective between wind energy capture efficiency and fatigue load suppression is balanced by the weighting coefficient.

[0018] Preferably, in step S5, a reinforcement learning method based on the idea of ​​value iteration is used to train the terminal value function. The first-order economic model predictive control optimizer is used as the action generator to calculate the control action in the continuous control input space. The target value is constructed using the immediate economic cost and the next state value to update the parameterized terminal value function.

[0019] In a further preferred embodiment, during the reinforcement learning training process, batch data is randomly sampled from the experience replay dataset to construct instantaneous rewards, mean squared error is used as the value function to train the loss, gradient descent is used to update the parameters of the main network, and soft updates are performed on the parameters of the target network.

[0020] Preferably, the deep neural network in step S4 is a fully connected feedforward neural network with three hidden layers, containing 128, 128, and 64 neurons respectively, and the activation function is... x represents the input state variable, and the state variable is subjected to min-max normalization at the network input.

[0021] Preferably, in step S6, the continuous-time optimization problem is discretized using the collocation method during online solving, and the IPOPT (Interior Point OPTimizer) solver is used to achieve online solving through the CasADi (Computer Algebra (System) with Automatic Differentiation) framework.

[0022] Preferably, in step S5, before reinforcement learning training, the deep neural network is pre-trained using the cost value obtained offline by predicting the control using a long-term nominal economic model, and then fine-tuned using reinforcement learning.

[0023] Preferably, in step S7, the wind turbine generator state and the currently available wind speed preview information are re-acquired at each sampling time, and the economic model predictive control optimization problem is reconstructed and solved using the current state as the initial value to achieve rolling time-domain closed-loop control.

[0024] The beneficial effects of this invention are:

[0025] (1) This invention uses the long-term value function obtained from reinforcement learning as the terminal cost of economic model predictive control, enabling short-time-domain economic model predictive control to compensate for the missing long-time-domain economic information under the limited prediction time domain. Thus, the closed-loop economic performance of wind turbine generators can be improved without significantly increasing the online prediction time domain.

[0026] (2) This invention retains the ability of economic model predictive control to explicitly process unit state constraints and control input constraints. Compared with reinforcement learning control strategies alone, this invention can directly consider engineering constraints such as rotor speed, generator torque, pitch angle and pitch rate during the optimization process, making it more suitable for the safe and reliable operation of wind turbine generator sets.

[0027] (3) This invention trains the terminal value function through offline reinforcement learning, transferring the impact of wind speed randomness on long-term economic performance to the offline learning stage, thus reducing the reliance on accurate long-term wind speed prediction in the online stage. In actual operation, the controller only needs a limited wind speed preview to complete short-term optimization, improving the applicability of the method in engineering scenarios.

[0028] (4) The present invention adopts short prediction time domain online optimization, which significantly reduces the scale of online optimization problem and computational burden compared with long time domain economic model predictive control, which is conducive to meeting the requirements of rapid sampling and real-time control of wind turbine generator sets.

[0029] (5) This invention considers both wind energy capture and structural fatigue load through economic performance indicators, enabling the controller to coordinate and optimize between improving power generation revenue and reducing mechanical damage to the unit, thereby improving the overall operating performance of the wind turbine generator set.

[0030] (6) This invention is applicable to the economic optimization control of variable speed wind turbine generator sets under random turbulent wind conditions, limited wind speed preview and real-time computing resource constraints. It can be used for the design of advanced control systems for onshore or offshore wind turbine generator sets, and can also be extended to wind farm-level coordinated control and other economic forecast control scenarios of renewable energy systems. Attached Figure Description

[0031] Figure 1 A framework diagram for predictive control of economic models based on reinforcement learning;

[0032] Figure 2 Wind turbine control decision logic structure diagram;

[0033] Figure 3 Graph showing turbulent wind speed variation;

[0034] Figure 4 Simulation results of state variables;

[0035] Figure 5 Simulation results of controlled variables;

[0036] Figure 6 Simulation results of aerodynamic power and output power. Detailed Implementation

[0037] The technical solution of the present invention will be further explained and described below with reference to specific embodiments.

[0038] like Figure 1 As shown, the method proposed in this embodiment includes two parts: an offline training module and an online control module. The technical solution of this invention will be further explained and described below with reference to specific embodiments.

[0039] This embodiment uses a 5 MW offshore wind turbine at NREL (National Renewable Energy Laboratory) as the controlled object and proposes a reinforcement learning-enhanced short-time domain economic model predictive control method. The method includes: establishing a nonlinear dynamic model of the wind turbine, constructing economic performance indicators, designing a short-time domain economic model predictive controller embedded with a terminal value function, and offline training of the terminal value function through reinforcement learning. Finally, it achieves economical and efficient operation of the wind turbine under stochastic wind speed conditions within an online rolling optimization framework. The steps include:

[0040] Step (1): Establish a nonlinear dynamic model of the offshore wind turbine generator set. This model is used to describe the relationship between wind speed input, generator torque, pitch angle and the dynamic state of the generator set.

[0041] The aerodynamic power captured by the wind turbine rotor of an offshore wind turbine can be represented by the following model:

[0042] (1)

[0043] In equation (1), air density, Where is the radius of the wind turbine. For wind speed, The wind energy utilization coefficient, The pitch angle is the propeller angle. The tip velocity ratio; where, the tip velocity ratio The definition of is:

[0044] (2)

[0045] In equation (2), ω is the angular velocity of the wind turbine.

[0046] The pitch angle described in this model Windmill angular velocity The following constraints must be satisfied:

[0047] (3)

[0048] (4)

[0049] (5)

[0050] In equations (3) to (5), This represents the rate of change of the pitch angle. These represent the minimum and maximum values ​​of the pitch angle, respectively. These represent the minimum and maximum values ​​of the rate of change of the pitch angle, respectively. These are the minimum and rated values ​​of angular velocity.

[0051] The dynamic model of the transmission chain is expressed as:

[0052] (6)

[0053] In equation (6), The moment of inertia of the wind turbine. and These are low-speed shafting and damping coefficient, respectively. For the rotation angle of the drive shaft, The pneumatic torque; the pneumatic torque The expression is:

[0054] (7).

[0055] The generator output power model consists of the generator-side dynamic equations and the generator output power equations; the generator-side dynamic equations are expressed as follows:

[0056] (8)

[0057] (9)

[0058] In equation (8), The moment of inertia of the generator. For the generator angular acceleration, The rate of change of the drive shaft angle. This refers to the gearbox transmission ratio. This represents the generator torque.

[0059] The generator output power equation is expressed as:

[0060] (10)

[0061] in, This refers to the generator efficiency.

[0062] Generator torque in this model The constraints are:

[0063] (11)

[0064] (12)

[0065] This is the rated value of the generator torque. The rate of change of the generator torque. These represent the minimum and maximum values ​​of the generator torque change rate.

[0066] Define the system state variables and control variables as follows:

[0067] (13)

[0068] The continuous-time nonlinear state equation of the wind turbine generator can be expressed as:

[0069] (14)

[0070] The above model is used to construct subsequent economic model prediction and control problems and reinforcement learning training problems.

[0071] Step (2): Constructing economic performance indicators

[0072] To simultaneously consider wind energy harvesting capacity and drivetrain fatigue load, the following economic stage cost function is constructed:

[0073] (15)

[0074] in:

[0075] (16)

[0076] (17)

[0077] To characterize the capture performance of wind turbine aerodynamic power, this invention uses a minimization objective function, thus representing the aerodynamic power... Take the negative sign; Used to characterize fatigue loads in the transmission chain. and The weighting coefficient is used to balance the optimization objectives between wind energy capture and fatigue load. Therefore, the control objective of this embodiment is to improve wind energy capture efficiency, reduce drivetrain fatigue load, and ensure real-time online calculation while meeting constraints such as rotor speed, pitch angle, generator torque and its rate of change.

[0078] Step (3): Constructing a reinforcement learning-enhanced short-term economic model for predictive control problems

[0079] At each sampling time Construct the following short-time economic model for predictive control optimization:

[0080] (18)

[0081] The constraints are:

[0082] (19)

[0083] (20)

[0084] And satisfy the input and state constraints corresponding to equations (3)-(5), (11) and (12). To predict the step size, The sampling period is As a discount factor, To parameterize the terminal value function, This represents the unit status at the current sampling time; This represents the set of piecewise continuous control functions defined over the sampling period. This represents the expected value.

[0085] The optimal control sequence is obtained at each sampling time. Then, the control effect within the first sampling interval is applied only to the wind turbine generator, that is:

[0086] (twenty one)

[0087] The system state is re-acquired at the next sampling time and the solution is repeated, thus forming a rolling optimization control mechanism.

[0088] Step (4): Offline training of reinforcement learning for the terminal value function

[0089] Since short-term economic model predictive control cannot fully reflect long-term economic information, this embodiment utilizes reinforcement learning to train the terminal value function offline. It is used to approximate the optimal economic performance index in the infinite time domain.

[0090] 1. Construct the optimal value function and action value function

[0091] The control process of a wind turbine generator can be abstracted as a Markov decision process, and its optimal value function is... and optimal action value function satisfy:

[0092] (twenty two)

[0093] (twenty three)

[0094] in, Indicates the current state Execution control The next state that is reached later.

[0095] 2. Generating Reinforcement Learning Actions Based on a First-Order EMPC Action Generator

[0096] To adapt to the continuous action space, a one-step economic model predictive controller is used as the action generator in reinforcement learning, and its optimization form is as follows:

[0097] (twenty four)

[0098] in:

[0099] (25)

[0100] In equation (24), Main network parameters, Defined as the closed-loop control law generated by the one-step economic model predictive controller, where U is the set of feasible control u; The action value function corresponding to the main network.

[0101] 3. Target value and network update

[0102] Randomly sample batches of data from the experience playback dataset. Constructing instant returns:

[0103] (26)

[0104] The target value is defined as:

[0105] (27)

[0106] in, Here are the target network parameters. Mean squared error is used as the value function for training loss.

[0107] (28)

[0108] The main network parameters are updated using gradient descent:

[0109] (29)

[0110] in, The learning rate is used. The target network parameters are soft-updated according to the following rules:

[0111] (30)

[0112] in, This is for updating coefficients.

[0113] 4. Neural Network Structure

[0114] Terminal value function A fully connected feedforward neural network is used, with three hidden layers and 128, 128, and 64 neurons respectively. To improve numerical stability, min-max normalization can be applied to the state variables at the network input. The activation function for the hidden layers can be selected as follows:

[0115] (31)

[0116] Where x is the input state variable;

[0117] To accelerate the training process, the network can be pre-trained using the cost obtained offline by the long-term nominal EMPC, and then fine-tuned using reinforcement learning.

[0118] Step (5): Implementation process of online rolling optimization control

[0119] like Figure 2 As shown, the online control phase is executed according to the following process:

[0120] Step 5.1: Status Acquisition

[0121] At sampling time Collect the status of wind turbine generators It includes wind turbine speed, generator speed and drive shaft torsion angle, and obtains measurable wind speed or limited wind speed preview information.

[0122] Step 5.2: Construct the optimization problem

[0123] In the current state As initial values, call the terminal value function obtained from offline training. The reinforcement learning-enhanced short-time domain economic model prediction and control optimization problem is constructed as shown in equation (18).

[0124] Step 5.3: Solve for the optimal control sequence

[0125] The optimization problem is solved using a nonlinear optimization algorithm. The continuous-time optimization problem is discretized using the collocation method, and the IPOPT solver is used to achieve online solution via the CasADi framework.

[0126] Step 5.4: Apply control input

[0127] The first step of the obtained optimal control sequence is applied to the wind turbine generator:

[0128] (32)

[0129] Step 5.5: Optimization of Repeated Scrolling

[0130] At the next sampling time Repeat steps 5.1 to 5.4 to achieve rolling optimization closed-loop control.

[0131] This invention has been verified through simulation under typical turbulent wind conditions. A schematic diagram of turbulent wind speed variation is shown below. Figure 3 As shown, the average wind speed is 11 m / s. Turbulence intensity represents the ratio of the standard deviation of wind speed to the average wind speed. A turbulence intensity of 0.15 is used, and a wind speed model based on the von Karman spectrum is employed. To illustrate the superiority of the method of this invention, the following three control methods are compared:

[0132] 1) The method of this invention (RL-EMPC): the prediction time domain is 1s, and the terminal value function obtained by reinforcement learning training is embedded;

[0133] 2) Long-term economic model predictive control (LH-EMPC): The prediction time domain is 15s, of which only the wind speed preview in the first 1s is known, and the rest of the time domain is approximated by constant values;

[0134] 3) Short-term economic model predictive control (SH-EMPC): The prediction time is 1 second, but the learned terminal value function is not used.

[0135] Figure 4 These are the simulation results for the state variables. Figure 5 These are the simulation results for the controlled variables. Figure 6 It is the output power of the wind turbine. aerodynamic power captured from the wind by the aerodynamic unit The simulation results.

[0136] To further quantify and compare the performance of the three methods, the following performance metrics are defined:

[0137] Average aerodynamic capture power:

[0138] (33)

[0139] Average power generation:

[0140] (34)

[0141] Mean fatigue load index:

[0142] (35)

[0143] in, For simulation steps, , where is the scaling factor. The economic performance indicators of the simulation results under turbulent wind speeds are shown in Table 1.

[0144] Table 1

[0145] RL-EMPC LH-EMPC SH-EMPC 4.7058e5 4.70033e5 4.70026e5 4.44238e5 4.43737e5 4.43707e5 0.005444 0.006207 0.006913 Average computation time (s) 0.039 0.26 0.019

[0146] The simulation results lead to the following conclusions:

[0147] (1) Under the same 1s wind speed preview conditions, by simply extending the prediction time domain, the economic performance improvement of LH-EMPC compared with SH-EMPC is relatively limited, but the online computing burden increases significantly;

[0148] (2) The method of the present invention introduces the learned terminal value function into the short-time domain EMPC, which improves the average aerodynamic capture power and average power generation while maintaining a low online computational load, and reduces the fatigue load of the transmission chain;

[0149] (3) Compared with SH-EMPC, the method of this invention has... The significant reduction in indicators indicates that the present invention has a better effect in suppressing structural fatigue;

[0150] (4) Compared with LH-EMPC, the online solution time of the method of the present invention is significantly reduced, making it more suitable for real-time control applications of actual wind turbine generator sets.

Claims

1. A short-time domain economic model predictive control method for offshore wind turbine generators, characterized in that, Includes the following steps: S1. Based on the aerodynamic power model, transmission chain dynamics model and power generation model, establish the state space model of the variable speed wind turbine generator set. The state variables include the wind turbine speed, generator speed and transmission shaft torsion angle, and the control variables include the generator torque and blade pitch angle. S2. Establish a stage cost function that includes the wind energy capture economic term and the transmission chain fatigue load penalty term; S3. Based on the state-space model and stage cost function, construct an economic model predictive control optimization problem with the sum of economic stage cost and terminal cost as the objective function within a finite prediction time domain. S4. Establish a parameterized terminal value function represented by a deep neural network to approximate the economic cost in the infinite time domain; S5. Train the parameters of the terminal value function offline using reinforcement learning; S6. Embed the trained terminal value function as the terminal cost into the short-time-domain economic model predictive control optimization problem, and solve the optimal control sequence online. S7. Apply the first control input in the optimal control sequence to the wind turbine generator set, and repeatedly solve the problem to achieve closed-loop control.

2. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, The constraints of the economic model predictive control optimization problem in step S3 include wind turbine speed constraints, generator torque constraints, pitch angle constraints, and pitch rate constraints.

3. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, The stage cost function described in step S2 is constructed by weighting the wind energy capture economic term and the transmission chain fatigue load penalty term, and the optimization objective between wind energy capture efficiency and fatigue load suppression is balanced by the weighting coefficients.

4. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, In step S5, a reinforcement learning method based on the value iteration idea is used to train the terminal value function. The first-order economic model predictive control optimizer is used as the action generator to calculate the control action in the continuous control input space. The target value is constructed using the immediate economic cost and the next state value to update the parameterized terminal value function.

5. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 4, characterized in that, During the reinforcement learning training process, batch data is randomly sampled from the experience replay dataset to construct instantaneous rewards. Mean squared error is used as the value function to train the loss. Gradient descent is used to update the parameters of the main network and soft updates are performed on the parameters of the target network.

6. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, The deep neural network described in step S4 is a fully connected feedforward neural network with three hidden layers, containing 128, 128, and 64 neurons respectively, and the activation function is... x represents the input state variable, and the state variable is subjected to min-max normalization at the network input.

7. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, In step S6, the collocation method is used to discretize the continuous-time optimization problem during online solving, and the interior-point method is used to optimize the solver and realize online solving through the CasADi framework.

8. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, In step S5, before reinforcement learning training, the deep neural network is pre-trained using the cost value obtained offline by predicting the control using a long-term nominal economic model, and then fine-tuned using reinforcement learning.

9. The short-time domain economic model predictive control method for offshore wind turbine generators according to claim 1, characterized in that, In step S7, the wind turbine generator status and the currently available wind speed preview information are re-acquired at each sampling time. The economic model predictive control optimization problem is reconstructed and solved using the current status as the initial value, thereby realizing rolling time-domain closed-loop control.

Images