A fuzzy predictive control method and system for floating wind turbines based on reinforcement learning

Abstract

The application discloses a kind of fuzzy prediction control method and system of floating wind turbine based on reinforcement learning, comprising: constructing multiple local linear models, discretization is carried out and is obtained by augmenting transformation prediction model;Future wind speed is predicted based on LSTM, and the prediction result is used as disturbance term;Respectively design multiple model predictive controller for independent variable pitch control of floating wind turbine, and obtain multiple optimal variable pitch control instructions by solving through rolling optimization;Based on fuzzy logic, according to current operating condition, the optimal variable pitch control instruction output is weighted and fused, and after saturation constraint and rate constraint are applied, the final independent variable pitch control instruction is obtained;The objective function weight of each model predictive controller is optimized online using reinforcement learning algorithm, and multi-objective optimization control is realized in the control stage.The application can realize the multi-objective optimization control of efficiency, vibration suppression and load reduction of floating wind turbine in the operating area above rated wind speed.

Description

Technical Field

[0001] This invention belongs to the field of offshore wind power generation technology, and in particular relates to a fuzzy predictive control method and system for floating wind turbines based on reinforcement learning. Background Technology

[0002] As floating wind turbines continue to grow in size, the environmental loads they bear will become more severe. To ensure the long-term safe and stable operation of floating wind turbines, while maximizing energy capture, minimizing the structural loads on key components, and mitigating the platform's oscillations under marine environmental excitation, a reasonable multi-objective control strategy for floating wind turbines needs to be proposed.

[0003] Pitch control, as a servo control technology for maintaining rated power and mitigating fatigue loads, provides an algorithm-based control scheme. Currently, pitch control can be broadly classified into two categories: collective pitch control (CPC) and independent pitch control (IPC). In collective pitch control, all blades receive the same pitch command to maintain the output power at the target rated value. Independent pitch control, on the other hand, applies different pitch commands to each blade to reduce potential structural loads.

[0004] Traditional pitch control primarily employs PID controllers, which are simple in structure and easy to implement, but have limitations in load mitigation and robustness. To overcome the performance limitations of PI methods, academia has proposed various advanced control methods such as adaptive fuzzy control, sliding mode control, H2, H∞, and linear quadratic regulation (LQR) control. These methods have superior performance potential compared to traditional PI controllers, but they are more difficult to design and struggle with strict state and input constraints. In recent years, Model Predictive Control (MPC) has attracted considerable attention due to its ability to solve multi-objective optimal control problems under various constraints and has been successfully applied in wind turbine control. However, in practical applications, current MPC-based wind turbine pitch control still faces the following difficulties and challenges: The accuracy of the predictive model is highly dependent on the accuracy of the system modeling. Because floating wind turbines operate in complex and variable marine environments, their dynamic characteristics exhibit significant nonlinearity and uncertainty. Large modeling errors can weaken the effectiveness of control strategies and even lead to system instability.

[0005] Predictive control performance largely depends on the appropriate configuration of target weight coefficients. For highly coupled and strongly disturbed systems such as floating wind turbines, the importance of different control targets changes dynamically under different operating conditions. Traditional fixed-weight methods (such as manual tuning and genetic algorithm optimization) have limitations in adaptability and cannot meet the dynamic optimization requirements of systems in complex and changing environments.

[0006] In the operating range above the rated wind speed, due to the large range of wind speed fluctuations, high turbulence intensity, and significant coupling effect between aerodynamic loads and structural response, the control strategy based on a linearized model at a single wind speed point is difficult to effectively adapt to the dynamic characteristics of the system across the entire wind speed range, which may lead to a decrease in control performance and fluctuations in power output.

[0007] Therefore, there is an urgent need to design a new predictive control method to overcome the problem of insufficient control performance of floating wind turbines in strong winds and complex marine environments in the existing technology. Summary of the Invention

[0008] This invention provides a fuzzy predictive control method and system for floating wind turbines based on reinforcement learning. By combining deep learning, reinforcement learning and fuzzy logic control technology, it can achieve predictive compensation for wind speed disturbances, adaptive adjustment of control target weights and multi-model joint control, thereby realizing multi-objective optimization control of floating wind turbines in operating areas above rated wind speed, including efficiency improvement, vibration suppression and load reduction.

[0009] A fuzzy predictive control method for floating wind turbines based on reinforcement learning includes the following steps: (1) Under different operating conditions in the area above the rated wind speed, multiple local linear models describing the floating wind turbine are constructed. The local linear models are discretized and augmented to obtain the prediction model. (2) Predict future wind speed based on long short-term memory network, and use the prediction results as a disturbance term to compensate for each prediction model; (3) Design multiple model predictive controllers to perform independent pitch control on the floating wind turbine. Using the predictive model constructed in step (1) and the disturbance term obtained in step (2), establish the system state prediction equation and obtain multiple optimal pitch control commands through rolling optimization. (4) Based on fuzzy logic, the optimal pitch control command output by multiple model predictors is weighted and fused according to the current operating conditions, and the final independent pitch control command is obtained after applying saturation constraints and rate constraints. (5) The weights of the objective function of each model predictive controller are optimized online by using reinforcement learning algorithm. Multiple weight adjustment agents are obtained through training and used as weight adjusters of the model predictive controller in the control stage to achieve multi-objective optimization control.

[0010] Furthermore, in step (1), the locally linear model is expressed as follows: ; in, It represents the rate at which the system state increment changes over time; Indicates the system state increment; Indicates the system output increment; This represents the pitch angle control input for the three blades; This indicates wind speed disturbance input; These represent the system state matrix, the effect matrix of the pitch angle control input on the state, the influence matrix of wind speed disturbance on the state, the system output matrix, the transfer matrix of the pitch angle control input to the output, and the transfer matrix of the wind speed disturbance to the output, respectively. These parameters were obtained using the high-fidelity simulation software OpenFAST in the linearization module.

[0011] Discretize the local linear model and perform an augmented transformation to obtain the prediction model: ; In the formula, , , , Ts is the sampling period.

[0012] Define augmented state variables: and order , , , Then the formula for the prediction model can be simplified to: ; Based on the above prediction model, the system state and output at N future time points can be predicted.

[0013] Furthermore, in step (2), the Long Short-Term Memory network is composed of an input layer, an LSTM layer, a Dropout layer, a fully connected layer, and an output layer connected in sequence to predict the future wind speed sequence. .

[0014] A long short-term memory network was constructed and trained using historical wind speed data to capture the nonlinear dynamic characteristics of wind speed time series.

[0015] In step (3), in order to achieve the tracking of the ideal state and at the same time minimize the wear of the blade actuator, that is, to bring the maximum control benefit with the minimum control cost, the objective function of the predictive controller MPC of each model is constructed as follows: ; In the formula, , representing the sequence of six output state quantities of the system: generator speed error, tower forward and backward deflection, platform pitch angle, and blade root flapping torque of the three blades; ; represents a reference value indicating the output state; This represents the incremental sequence of pitch control; To output the weight matrix, To control the incremental weight matrix.

[0016] Transform the objective function in the above equation into a standard quadratic form: Where E and F are matrices derived from the prediction model, To control the incremental sequence.

[0017] The optimal control sequence is obtained by solving the above optimization problem using a quadratic programming (QP) solver: ; The solution process needs to satisfy the following constraints: 1. That is, the total control input increment must meet the constraint range determined by the upper and lower limits of the pitch control quantity; 2. That is, the increment of the control input at each sampling moment must meet the constraint range determined by the upper and lower limits of the pitch control quantity rate.

[0018] The first control variable is selected as the current control increment input for each model predictive controller, i.e.: .

[0019] Furthermore, in step (4), the fuzzy logic specifically refers to: Multiple control sub-intervals are constructed based on the wind speed range above the rated wind speed, and each control sub-interval corresponds to a model predictive controller. Corresponding membership functions are set for each model predictive controller, and the participation degree and output weight of each model predictive controller under the current operating condition are determined based on the average wind speed at the current time or within the sliding time window. In this way, the optimal pitch control commands output by multiple model predictive controllers are weighted and fused.

[0020] Furthermore, a membership function with interval overlap characteristics is adopted so that the model predictive controllers corresponding to adjacent control sub-intervals can jointly participate in control within the overlapping region; When a floating wind turbine operates within a certain control sub-section, it is controlled by the corresponding single model predictive controller; when the floating wind turbine operates in the transition region of an adjacent control sub-section, it is controlled by two adjacent model predictive controllers, thereby avoiding control abrupt changes caused by controller switching.

[0021] Furthermore, in step (4), the saturation constraint is used to limit the pitch angle of each blade within the allowable physical range, and the rate constraint is used to limit the rate of change of the pitch angle within a unit sampling period.

[0022] Specifically, the operating range is divided into There are 3 control sub-intervals, each corresponding to a model predictive controller. To describe the suitability of each controller under the current operating wind speed conditions, a fuzzy membership function is constructed. , .

[0023] in, Indicates the first The membership function value of each MPC represents the effectiveness of the controller under the current operating conditions; Indicates the MPC number; This indicates the total number of MPCs. Indicates the first The average wind speed at each sampling time is used as the input variable for fuzzy discrimination.

[0024] The range of values ​​for the membership function is: ,when When, it indicates the first i Each controller is perfectly suited to the current operating conditions; when When, it means that the controller does not participate in control under the current operating condition.

[0025] To reduce the impact of instantaneous wind speed fluctuations on controller switching, the average wind speed within a sliding time window is used as the fuzzy inference input, and its calculation formula is as follows: ; In the formula, Indicates the first The average wind speed at that moment; Indicates the first The instantaneous wind speed value at each sampling moment; Indicates that you are currently away from taking a walk; It is a time index variable; For a sliding time window, satisfying ; The average time window length, This is the system sampling period.

[0026] The original weights of each controller are calculated based on the membership function: ; In the formula, Indicates the first Each controller at time The original weights are then normalized to obtain the final fused weights. ; In the formula, Indicates the first Normalized weights for each MPC; This is the sum of the original weights of all MPCs. The normalized weights of each MPC satisfy: , .

[0027] Each MPC outputs control increments. The final pitch increment is obtained through weighted fusion: ; The control increment update of the pitch command at time k is as follows: ; To ensure the safe operation of the actuator, saturation limits are imposed on control commands: ; in, For the three blade pitch commands after saturation constraint; These are the upper and lower limits of the propeller pitch angle, respectively.

[0028] Rate limiting of pitch commands after saturation: ; in, The blade pitch rate after saturation constraint; These are the upper and lower limits of the pitch rate, respectively.

[0029] Finally, the pitch control command issued to the pitch actuator is as follows: ; In the above process, to reduce online computational complexity, when a certain controller satisfies: When this condition is met, it is determined that it does not participate in the current control calculation, and only applies to cases that satisfy: The controller performs a rolling optimization solution.

[0030] Furthermore, in step (5), a TD3 reinforcement learning agent is used. After training, the agent outputs the corresponding weight adjustment amount in real time according to the current state input of the floating wind turbine, and corrects the objective function weight matrix of each model predictive controller online. The status inputs include generator speed error, tower forward and backward deflection, platform pitch motion status, and blade root load information for the three blades.

[0031] Multiple TD3 policy agents, trained using reinforcement learning algorithms, are capable of adjusting the control weight matrices Q and R of the model predictive controller in real time based on the state parameters of the floating wind turbine system. The design process is as follows: Constructing the TD3 state space: , This indicates the error state of the generator speed relative to the rated speed; Indicates the forward and backward deflection state of the tower; This indicates the platform's pitching motion. These represent the flapping torque at the root of each of the three blades.

[0032] Constructing the TD3 action space: The weight matrices Q and R are modified as follows: , , , , . , , , , These are the scaling factors of the order of magnitude for the output state variables and the pitch increments, respectively.

[0033] The Actor network structure for each TD3 agent consists of a state feature input layer, hidden layers, and an action output layer. The hidden layers are composed of fully connected layers with 120, 60, and 30 neurons respectively, each followed by a ReLU activation function. The action output layer is a fully connected layer with 5 neurons, followed by a Tanh activation function.

[0034] The Critic network structure is designed as follows: Each TD3 agent's Critic network includes a state path, an action path, and a common path. The state path consists of a state feature input layer and two fully connected layers with 120 and 60 neurons respectively, followed by a ReLU activation function. The action path consists of an action feature input layer and a fully connected layer with 60 neurons. The common path consists of a concatenation layer, a fully connected layer, and an output layer. The state path and action path features are fused and then passed through a ReLU activation function, a fully connected layer with 30 neurons, another ReLU activation function, and a fully connected layer with 1 neuron, outputting the Q-value. Preferably, each TD3 agent has two Critic networks with identical structures and independent parameters, used to estimate the Q-value separately and take the smaller value during target computation to suppress overestimation of the value function and improve training stability.

[0035] Design Reward Functions: Five different reward functions were designed based on the different effects of rotational speed error, tower forward and backward deflection, platform pitch motion, blade root load, and pitch control amplitude on the floating wind turbine system. Rotational speed tracking reward function: ; Tower deflection reward function: ; Platform exercise reward function: ; Leaf root load reward function: ; Pitch cost reward function: ; Total reward function: ; In the formula, , , , , These are the weighting coefficients for each reward function.

[0036] The present invention also provides a fuzzy predictive control system for floating wind turbines based on reinforcement learning, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned fuzzy predictive control method for floating wind turbines.

[0037] This invention also provides a fuzzy predictive control system for floating wind turbines based on reinforcement learning, including a data acquisition and state acquisition module, a local model construction module, a wind speed prediction and compensation module, a multi-model predictive control module, a fuzzy logic fusion module, a reinforcement learning weight adjustment module, a constraint processing module, and an actuator driving module; wherein, the modules are connected to each other through a signal transmission interface or a control bus to form a closed-loop control system.

[0038] The data acquisition and status monitoring module is used to collect unit status information, structural response information and environmental input information during the operation of the floating wind turbine, and to preprocess, filter, synchronize and standardize the collected data to form status input quantities for subsequent control.

[0039] Preferably, the information collected by the data acquisition and status monitoring module includes one or more of the following: generator speed, output power, tower forward and backward deflection, platform pitch motion state, blade root load of the three blades, pitch angle, pitch angular rate, and wind speed information; wherein, the wind speed information includes instantaneous wind speed, average wind speed, wind speed disturbance estimate and its changing trend information.

[0040] The local model building module is used to pre-build multiple local linear models of floating wind turbines based on different operating conditions above the rated wind speed, and to store, call and switch between each local linear model. Each local linear model corresponds to an independent model prediction controller, which is used to characterize the local dynamic characteristics of the floating wind turbine within the corresponding wind speed range.

[0041] The wind speed prediction and compensation module is used to build and run a long short-term memory network model based on historical wind speed sequences and current wind speed measurement information. It predicts wind speed disturbances in the future prediction time domain and sends the prediction results to the multi-model prediction control module for disturbance compensation of each local prediction model, thereby improving the accuracy of state prediction.

[0042] The multi-model predictive control module, connected to the local model construction module and the wind speed prediction compensation module, is used to call the local predictive models related to the current operating conditions, combine the wind speed prediction results, the current system state and the reference target, construct rolling optimization problems for each model predictive controller, and solve them online to obtain multiple candidate pitch control increments or candidate pitch control commands.

[0043] Each model predictive controller is equipped with independent objective function weight parameters, and can comprehensively optimize objectives such as maintaining rated power, suppressing platform motion, suppressing tower vibration, reducing blade root load, and smoothing pitch control according to the control requirements of different operating conditions of floating wind turbines.

[0044] The fuzzy logic fusion module is used to perform weighted fusion of the output results of multiple model predictive controllers based on the current wind speed conditions, so as to achieve smooth transition and joint control between different local controllers.

[0045] The reinforcement learning weight adjustment module is used to adaptively adjust the objective function weights of each model predictive controller according to the real-time operating status of the floating wind turbine, so as to improve the environmental adaptability and multi-objective coordinated optimization capability of the control system under complex sea conditions and variable wind speed conditions.

[0046] Preferably, the reinforcement learning weight adjustment module is composed of a TD3 reinforcement learning agent. After training, the agent outputs the corresponding weight adjustment amount in real time according to the current state input of the floating wind turbine, and corrects the objective function weight matrix of each model prediction controller in the multi-model prediction control module online.

[0047] The state inputs in the reinforcement learning weight adjustment module preferably include generator speed error, tower forward and backward deflection, platform pitch motion state, and blade root load information of the three blades, which are used to comprehensively reflect the current power deviation, structural vibration level, and load characteristics of the floating wind turbine.

[0048] The weight adjustment results output by the reinforcement learning weight adjustment module are used to dynamically change the degree of attention the model predictive controller pays to different control objectives, so that the control system can adaptively balance power stability, platform vibration suppression, load reduction and actuator wear reduction according to different operating conditions.

[0049] The constraint processing module is used to perform physical constraint correction on the final pitch control command output by the fuzzy logic fusion module to ensure that the control command meets the amplitude and rate limits of the floating wind turbine pitch actuator.

[0050] Preferably, the constraint processing module includes a pitch angle saturation processing unit and a pitch rate limiting unit; the pitch angle saturation processing unit is used to limit the pitch angle of each blade within the allowable physical range, and the pitch rate limiting unit is used to limit the pitch angle change rate within a unit sampling period to avoid overload or frequent operation of the actuator.

[0051] The actuator drive module, connected to the constraint processing module, is used to receive the final independent pitch control command after constraint processing and send it to the pitch actuators corresponding to the three blades of the floating wind turbine to achieve independent adjustment of the pitch angle of the three blades.

[0052] Furthermore, the actuator drive module may include a drive interface unit and an execution feedback unit; wherein, the drive interface unit is used to send control commands to the pitch servo system, and the execution feedback unit is used to collect the pitch angle and execution status information after actual execution, and feed them back to the data acquisition and status monitoring module to form a closed-loop control loop.

[0053] To reduce the online computational burden on the system, the control system also includes a controller activation management unit, which is used to determine whether each model predictive controller participates in the rolling optimization calculation at the current sampling time based on the controller weight information output by the fuzzy logic fusion module. When the weight of a certain model predictive controller is zero, the controller does not participate in the solution at the current sampling time, and only performs online optimization for model predictive controllers with non-zero weights.

[0054] The control system can be deployed in the main control system of the floating wind turbine, edge controller, industrial computer or embedded control platform; among them, the local model management module, wind speed prediction module, multi-model prediction control module, fuzzy logic fusion module and reinforcement learning weight adjustment module can be implemented by software program, and the data acquisition and status monitoring module and actuator control module can achieve hardware and software coordination through sensor interface and actuator interface.

[0055] In a preferred embodiment, the wind speed prediction module and the reinforcement learning weight adjustment module are operated in an offline training and online deployment manner; wherein, the model training is completed using the floating wind turbine simulation environment or historical operating data in the offline stage, and only forward inference and control quantity calculation are performed in the online stage, thereby reducing the online computational complexity and improving real-time control performance.

[0056] In another preferred embodiment, the multi-model predictive control module uses a quadratic programming solver for online optimization calculations. The solver can be deployed on an industrial controller or a high-performance embedded computing platform to meet the real-time requirements of the floating wind turbine control system.

[0057] The control system described in this invention integrates local model predictive control, wind speed disturbance prediction, fuzzy logic fusion, and reinforcement learning weight adaptive adjustment, enabling floating wind turbines to achieve smooth switching of multiple models, adaptive trade-off of control objectives, and independent pitch coordination under complex operating conditions above rated wind speed. This effectively improves power control stability, reduces platform oscillation response, reduces tower and blade structural loads, and improves the smoothness of actuator operation and system robustness.

[0058] Compared with the prior art, the present invention has the following beneficial effects: 1. It can adapt to the nonlinear dynamic changes of floating wind turbines within a wide wind speed range above the rated wind speed. By constructing multiple local linear models under different operating conditions and combining them with fuzzy logic to achieve multi-model collaborative control, the controller can automatically match the optimal model according to the current wind speed, thereby effectively characterizing the nonlinear characteristics of the system, reducing the error caused by the mismatch of a single model, and improving the stability and robustness of the system under complex sea conditions and strong winds.

[0059] 2. Wind speed prediction compensation can reduce the impact of model mismatch and improve model prediction accuracy. An LSTM-based wind speed prediction mechanism is introduced to estimate future disturbances in advance and use them for model compensation. This gives the controller a certain feedforward capability, thereby reducing prediction deviations caused by wind speed fluctuations, improving the accuracy of system state prediction, enhancing the foresight of control optimization, and ultimately improving power stability and structural response control performance.

[0060] 3. It can dynamically adjust control weights through reinforcement learning to improve multi-objective control performance. Reinforcement learning is used to adaptively adjust the MPC weight parameters, enabling the controller to dynamically allocate weights among multiple objectives such as power control, platform vibration suppression, and load reduction based on real-time operating conditions. This avoids the insufficient adaptability caused by fixed weights, thereby continuously achieving better overall control performance under different operating conditions.

[0061] 4. It can achieve smooth switching between multiple controllers through fuzzy fusion mechanism, avoiding performance degradation of single-model controllers under complex operating conditions. By using fuzzy membership functions to weight and fuse the outputs of different controllers, the controllers can achieve continuous transition between different wind speed ranges, avoiding the control abruptness problem caused by traditional hard switching. At the same time, it fully leverages the advantages of each local model within its applicable range, improving the overall control smoothness and stability.

[0062] 5. Reduce online computational burden and improve real-time application capabilities through controller activation management mechanism. By judging controller weights, only controllers that make effective contributions under the current operating conditions are activated to participate in online optimization calculations, reducing redundant QP solution processes, thereby reducing system computational complexity and improving the real-time performance of the control algorithm while ensuring control performance, meeting the embedded deployment requirements in practical engineering. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating the framework of a fuzzy predictive control method for floating wind turbines based on reinforcement learning, as proposed in this invention.

[0065] Figure 2 This is a flowchart of the LSTM training process in this invention.

[0066] Figure 3 This is a schematic diagram of the membership function of the fuzzy system in this invention.

[0067] Figure 4 This is a training block diagram of the reinforcement learning algorithm in this invention.

[0068] Figure 5 This is a schematic diagram of a fuzzy predictive control system for a floating wind turbine based on reinforcement learning, as described in this invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0071] like Figure 1As shown, a fuzzy predictive control method for floating wind turbines based on reinforcement learning is used for independent pitch control of floating wind turbines in the operating range above rated wind speed. This method comprehensively utilizes local linear modeling, multi-model predictive control, wind speed prediction compensation, fuzzy logic fusion, and reinforcement learning weight adjustment mechanism to achieve multi-objective coordinated control of floating wind turbines in complex marine environments, including maintaining rated power, suppressing platform motion, and mitigating structural loads.

[0072] In this embodiment, the controlled object is the OC4-Deepcwind semi-submersible floating wind turbine model developed by the University of Maine. The controller design was completed in the OpenFAST simulation platform and SimuLink.

[0073] Since the aerodynamic characteristics, structural response, and platform coupling dynamics of a floating wind turbine change significantly with wind speed when it operates above the rated wind speed, this embodiment uses a multi-operating-point linearization modeling method to describe the system in order to improve the controller's adaptability in a wide wind speed range.

[0074] Specifically, within the wind speed range of 12 m / s to 25 m / s, multiple steady-state operating points are set at intervals of 1 m / s, and the OC4 floating wind turbine model is linearized using the OpenFAST linearization module at each steady-state operating point to obtain a local linear model under the corresponding operating conditions.

[0075] At each operating point, the steady-state operating state of the floating wind turbine at the corresponding wind speed is first obtained. Then, the system is linearized with small disturbances around the steady-state point to obtain the system state matrix, control input matrix, disturbance input matrix and output matrix, which are used to characterize the local linear dynamic relationship between pitch input, wind speed disturbance and system state and output.

[0076] In this embodiment, the control input is the independent pitch angle increment of the three blades, the disturbance input is the wind speed disturbance, and the outputs include generator speed error, tower forward and backward deflection, platform pitch angle, and the first-order blade root flapping moment of the three blades. This allows for the construction of a locally linear model suitable for different wind speed conditions. ; To facilitate the construction of a model predictive controller, the local linear model is further discretized, and an augmented approach is used to construct a discrete predictive model for predictive control. ; This approach allows both state and output variables to be incorporated into the prediction framework, facilitating the subsequent construction of rolling optimization problems.

[0077] Although multiple sets of local linear models can be obtained at 1 m / s intervals within the wind speed range of 12 m / s to 25 m / s, designing controllers for each model and having them all participate in the optimization solution during the control phase would significantly increase the online computational load of the system, which is not conducive to real-time control. Therefore, this implementation further filters each local model.

[0078] This implementation method employs a model gap metric to perform similarity analysis on the local linear models corresponding to different wind speed operating points. By calculating the gap metric value between adjacent wind speed operating point models, the degree of dynamic difference between different models is assessed. The gap metric calculation formula is as follows: ; In the formula, The transfer function representing adjacent linear models; This represents the conjugate of the transfer function; This represents the maximum gap distance between adjacent models, with a value ranging from [0, 1]. When the gap metric between adjacent models is small, it indicates that they are similar in dynamic characteristics and can be covered by the same local controller; when the gap metric is large, it indicates that the system dynamics change significantly with wind speed, and the corresponding model should be retained and configured with an independent controller.

[0079] Through the above model gap analysis and screening process, the number of local models can be reduced while ensuring the accuracy of model representation, thereby reducing the online solution burden under the multi-model predictive control framework.

[0080] In a preferred embodiment of this invention, five representative local linear models are selected as the basis for the prediction models of the multi-model predictive controller, with corresponding center wind speeds of 12 m / s, 15 m / s, 18 m / s, 21 m / s and 24 m / s, respectively, to cover the main operating range above the rated wind speed.

[0081] To reduce the impact of random wind speed fluctuations on the accuracy of predictive control, this implementation introduces an LSTM wind speed prediction model to predict future wind speed disturbances in advance, and uses the prediction results as disturbance feedforward information input to the predictive control module.

[0082] like Figure 2 As shown, specifically, historical wind speed time series data during the operation of the floating wind turbine are first collected, and the raw wind speed data is preprocessed. The preprocessing includes outlier removal, missing value completion, filtering, and normalization to improve the stability and prediction accuracy of subsequent model training.

[0083] Then, a supervised learning sample is constructed using a sliding time window approach, with historical wind speed sequences at several consecutive moments as input samples and wind speed values ​​at several future moments as prediction labels, to establish a wind speed prediction sample set.

[0084] In this embodiment, the LSTM wind speed prediction model consists of an input layer, an LSTM layer, a Dropout layer, a fully connected layer, and an output layer connected sequentially. The input layer is used to input historical wind speed sequences, the LSTM layer is used to extract temporal dependency features from the wind speed sequences, the Dropout layer is used to reduce the risk of model overfitting, the fully connected layer is used to complete feature mapping, and the output layer is used to output the wind speed prediction results for the future prediction time domain.

[0085] The trained LSTM model is deployed in the control system. During the control phase, it receives current and historical wind speed data in real time and outputs a wind speed prediction sequence for the next few steps. This prediction sequence serves as the wind speed disturbance input to the prediction model, compensating for state prediction errors caused by wind speed fluctuations, thereby improving the system's prediction accuracy and control foresight.

[0086] Based on the selected local linear models, this implementation designes corresponding model predictive controllers (MPCs). Each MPC uses its corresponding local linear model within the wind speed range as the prediction model and employs a rolling time-domain optimization strategy to solve for the optimal pitch control sequence at the current moment.

[0087] In this embodiment, the MPC control objectives include: maintaining the generator speed stable near the rated value, reducing the tower's forward and backward vibration, suppressing the platform's pitching motion, reducing the blade root flapping torque of the three blades, and limiting the pitch action amplitude and rate of change, so as to reduce actuator wear and ensure control smoothness.

[0088] To achieve the above control objectives, each MPC constructs an objective function: , Among them, the output error term Used to describe objectives such as power control, platform vibration suppression, and load mitigation, controlling incremental terms. This is used to penalize excessive changes in pitch maneuvering. Further, by combining pitch angle amplitude constraints and pitch angle rate constraints, a quadratic performance index function is constructed. This transforms the control problem into a standard quadratic programming problem, which can then be solved online using the QP solver.

[0089] At each sampling time, each local MPC constructs a corresponding rolling optimization problem based on the current system state, the reference target, and the future wind speed disturbance sequence predicted by LSTM, obtains its own optimal pitch control sequence, and takes the first term of the optimal sequence as the candidate control output for the current time: ; To ensure that each local MPC has good basic control performance within its corresponding wind speed range, this implementation performs offline optimization tuning of the weight matrices Q and R of each controller. Specifically, the fixed values ​​of the weight matrices Q and R are obtained through Bayesian optimization.

[0090] Bayesian optimization uses the comprehensive performance index of closed-loop control as the objective function, and comprehensively considers multiple performance indicators such as generator speed error, tower forward and backward deflection, platform pitch motion, blade root flapping torque and pitch action amplitude, and searches for the optimal combination of Q and R parameters within the preset parameter range.

[0091] During the optimization process, the OpenFAST simulation platform was used as the performance evaluation environment. In each iteration, the Bayesian optimization algorithm generates candidate weight parameters, calls the MPC under the corresponding working condition for closed-loop simulation, calculates the performance evaluation index based on the simulation results, and then gradually approaches the optimal solution through surrogate model updates and iterative acquisition function iterations.

[0092] After the offline optimization described above, each local MPC obtains its corresponding baseline weight matrix Q and R. This baseline weight matrix represents the controller's basic control preference within its respective wind speed range and serves as the initial basis for subsequent online weight adjustment in reinforcement learning.

[0093] Because wind speed varies continuously between multiple applicable ranges of local models during actual operation, using a single controller for hard switching can easily lead to discontinuous control output or even abrupt changes. Therefore, such as... Figure 3 As shown, this embodiment uses a fuzzy logic fusion method to perform weighted fusion of multiple local MPC outputs.

[0094] Specifically, multiple control sub-intervals are constructed based on the wind speed range of the operating area above the rated wind speed. Each sub-interval corresponds to a local MPC, and a fuzzy membership function related to the average wind speed is established for each controller to characterize the applicability of each controller under the current wind speed conditions.

[0095] To mitigate the impact of instantaneous wind speed fluctuations on controller switching, this implementation uses the average wind speed within a sliding time window as the fuzzy inference input. Subsequently, the original weights are calculated based on the membership degrees of each controller, and then normalized to obtain the final fusion weights for each local MPC. .

[0096] In a preferred embodiment of this invention, the controllers at both ends employ shoulder-shaped membership functions, while the controller in the middle employs triangular membership functions. This ensures that at any given time, typically only one or two adjacent controllers have non-zero weights, thereby guaranteeing a smooth transition between adjacent wind speed ranges and helping to reduce the number of controllers simultaneously participating in online calculations.

[0097] Finally, the candidate pitch control increments output by each local MPC are weighted and summed according to their corresponding fusion weights to obtain the final pitch control increment at the current moment. And based on this, update the independent pitch control commands for the three blades.

[0098] Based on the fact that the local MPC baseline weight matrices Q and R have been determined offline through Bayesian optimization, this embodiment further introduces the TD3 reinforcement learning algorithm to perform online adaptive correction of the baseline weights, so as to improve the multi-objective coordination capability of the control system under complex working conditions.

[0099] Specifically, corresponding TD3 agents are trained for each local MPC. Each TD3 agent outputs a weight matrix Q and a correction amount A for R based on the current operating state of the floating wind turbine, thereby dynamically adjusting the trade-off between control objectives such as generator speed stability, platform pitch motion suppression, tower vibration reduction, blade load reduction, and pitch smoothness in the MPC.

[0100] like Figure 4 As shown, in this embodiment, the state input of the TD3 agent... The preferred parameters include generator speed error, tower forward and backward deflection, platform pitch motion, and blade root flapping torque of the three blades, to comprehensively reflect the current system's power deviation, structural vibration level, and load status. (Action output) The baseline weight matrices Q and R are used to modify the MPC. Preferably, the action space can be represented as: And it is used to correct the benchmark weight parameters: , , , , .in, , , , , These are scaling factors of the order of magnitude for the output state variables and the pitch increments, respectively. , , , , These are the weight parameters adjusted by TD3.

[0101] In the TD3 algorithm, the Actor network employs a multi-layer fully connected structure to adjust actions based on the input state and output weights. Its policy function is expressed as: .in, Represents an Actor network. This represents the parameters of the Actor network. The Critic network uses a dual Q-structure to estimate the state-action value function: , .in, and These represent the parameters of the two Critic networks. By using a dual-Q network and taking the smaller value when calculating the target value, overestimation of the value function can be effectively suppressed and training stability can be improved.

[0102] The reward function comprehensively considers generator speed tracking error, tower forward and backward deflection, platform pitch amplitude, blade root flapping torque, and pitch amplitude to guide the agent to achieve a reasonable trade-off among multi-objective control tasks. Preferably, the total reward function can be expressed as: ; ; ; ; ; ; in, This refers to the weighting coefficient for the reward items; This refers to the generator speed error state; This refers to the forward and backward deflection state of the tower. The platform is in a state of pitching motion; The flapping torque state at the leaf root of the three blades; , , This represents the change in pitch action of the three blades.

[0103] During training, the parameters of the Actor network, two Critic networks, and their corresponding target networks are first initialized, denoted as follows: , , ,and , , And build an experience replay pool. Then, in each training step, based on the current state... Actions are generated by an Actor network, and exploration noise is superimposed: .in To explore noise.

[0104] Actions Applying to the floating wind turbine environment to obtain the state at the next moment. and instant rewards Forming state transition samples The samples are then stored in the experience replay pool. Subsequently, a batch of samples is randomly sampled from the experience replay pool for network training to break the temporal correlation between samples and improve training stability.

[0105] For each sample obtained from sampling, the target action for the next time step is first generated through the target Actor network, and then truncated noise is superimposed on it to obtain: ; in, To smooth out noise, For noise cutoff boundary, This represents the cutoff function. Further, the target Q-values ​​are calculated using two target Critic networks respectively, and the smaller value is used to construct the TD target value: .

[0106] in, This is the discount factor.

[0107] Using target value Construct loss functions for two Critic networks and update their parameters using gradient descent. Preferably, the first... The loss function of a Critic network is expressed as: , .

[0108] in, Let be the batch size. By minimizing the above loss function, the Critic network's estimate of the state-action value function gradually approaches the target value.

[0109] During the Actor network update process, a delayed update strategy is adopted, meaning the Actor network is not updated at every step, but rather after several Critic network updates. The optimization objective of the Actor network is to maximize the state-action values ​​output by the Critic network, and its objective function can be expressed as: ; Its policy gradient can be expressed as: ; This update method enables the Actor network to gradually output better weight adjustment actions.

[0110] After each update of the main network parameters, a soft update is performed on the target network to allow the target network to slowly track changes in the main network. The specific update formula is as follows: ; ; in, It is a soft update coefficient, and .

[0111] Repeat the above training steps until the Actor network and Critic network converge, obtaining the optimal strategy that can adaptively adjust the MPC weight matrix according to the floating wind turbine's operating state. The trained TD3 agent is then deployed offline in the control system, performing only forward inference computation during the real-time control phase. This action is used to correct the MPC weight matrix online, eliminating the need for online learning and thus meeting the real-time requirements of the floating wind turbine control system.

[0112] After obtaining the final pitch control increment through fuzzy logic fusion, this embodiment further includes a constraint processing module to ensure that the control commands meet the physical limitations of the actuator. The constraint processing module includes a pitch angle saturation processing unit and a pitch rate limiting unit.

[0113] The pitch angle saturation processing unit is used to limit the pitch control commands of the three blades to within the allowable amplitude range; the pitch rate limiting unit is used to limit the pitch angle change rate within a unit sampling period to prevent the pitch actuator from being overloaded or from experiencing increased mechanical wear due to frequent operation.

[0114] After the above constraint processing, the final pitch control command is sent to the pitch servo actuators corresponding to the three blades to realize independent adjustment of the pitch angle of the three blades.

[0115] To reduce the online computational complexity of the multi-model predictive control framework, this implementation further includes a controller activation management mechanism. This mechanism determines which controllers need to participate in online optimization at the current sampling time based on the controller weight information output by the fuzzy logic fusion module.

[0116] When the fusion weight of a certain local MPC is zero, it is determined that the controller does not participate in the control calculation under the current operating condition, and its corresponding rolling optimization problem is not solved at the current sampling time; only controllers with fusion weights greater than zero are subjected to state prediction, objective function construction and QP online optimization solution.

[0117] In the preferred case, at any given sampling time, only one or two local MPCs that match the current wind speed conditions are active, thereby reducing the scale of parallel optimization and improving the real-time performance of the control system while ensuring control smoothness.

[0118] like Figure 5 As shown, in this embodiment, the operation process of the control system is as follows: First, the data acquisition and status monitoring module acquires real-time information such as current wind speed, generator speed, platform pitch motion, tower forward and backward deflection, blade root load, and blade pitch angle; then, the wind speed prediction and compensation module predicts future wind speed disturbances; subsequently, the multi-objective prediction control module calls the currently activated local model predictive controller, and calculates candidate pitch control increments by combining the current system state, reference target, and future wind speed prediction results; next, the reinforcement learning weight adjustment module corrects the baseline weights of each local model predictive controller online; then, the fuzzy logic fusion module performs weighted fusion of multiple candidate control quantities based on the average wind speed; finally, the constraint processing and execution mechanism drive module generates independent pitch control commands that meet physical constraints and applies them to the floating wind turbine.

[0119] After control is executed, the new operating status of the floating wind turbine is fed back to the data acquisition and status monitoring module and the reinforcement learning weight adjustment module, forming a complete closed-loop control loop. Through continuous rolling optimization and closed-loop feedback, real-time independent pitch control of the floating wind turbine in complex marine environments above rated wind speed is achieved.

[0120] Compared with the traditional single-model fixed-weight predictive control method, this implementation method is based on OpenFAST, selects the OC4 floating wind turbine model to construct multiple local linear models, and combines model gap measurement to achieve representative model selection. This can more accurately characterize the dynamic characteristics of the system under different operating conditions above the rated wind speed and improve the matching degree between the predictive model and the actual operating state.

[0121] By introducing a wind speed prediction and compensation module, this implementation can estimate future wind speed disturbances in advance and implement disturbance compensation, thereby improving the accuracy of system state prediction and the foresight of control. By using Bayesian optimization to tune the baseline weight matrices Q and R of each local MPC offline, the basic control performance of the controller within the wind speed range can be improved. By further introducing TD3 reinforcement learning to perform online adaptive correction of the weight matrix, the control system can achieve dynamic coordination between power stability, platform vibration suppression, tower vibration reduction and blade load reduction according to the real-time operating status.

[0122] Meanwhile, the fuzzy logic fusion mechanism can ensure a smooth transition between multiple local controllers in different wind speed ranges, avoiding control abrupt changes caused by hard switching; the controller activation management mechanism can further reduce the online optimization calculation burden, making the present invention more suitable for engineering applications of real-time control systems for floating wind turbines.

[0123] In summary, the fuzzy predictive control method and system for floating wind turbines based on reinforcement learning proposed in this embodiment can achieve better power regulation performance, lower platform motion response, smaller critical structural loads, and better control smoothness and real-time performance in complex marine environments above rated wind speed.

[0124] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fuzzy predictive control method for floating wind turbines based on reinforcement learning, characterized in that, Includes the following steps: (1) Under different operating conditions in the area above the rated wind speed, multiple local linear models describing the floating wind turbine are constructed. The local linear models are discretized and augmented to obtain the prediction model. (2) Predict future wind speed based on long short-term memory network, and use the prediction results as a disturbance term to compensate for each prediction model; (3) Design multiple model predictive controllers to perform independent pitch control on the floating wind turbine. Using the predictive model constructed in step (1) and the disturbance term obtained in step (2), establish the system state prediction equation and obtain multiple optimal pitch control commands through rolling optimization. (4) Based on fuzzy logic, the optimal pitch control command output by multiple model predictors is weighted and fused according to the current operating conditions, and the final independent pitch control command is obtained after applying saturation constraints and rate constraints. (5) The weights of the objective function of each model predictive controller are optimized online by using reinforcement learning algorithm. Multiple weight adjustment agents are obtained through training and used as weight adjusters of the model predictive controller in the control stage to achieve multi-objective optimization control.

2. The fuzzy predictive control method for floating wind turbines based on reinforcement learning according to claim 1, characterized in that, In step (1), the locally linear model is expressed as follows: ； in, It represents the rate at which the system state increment changes over time; Indicates the system state increment; Indicates the system output increment; This represents the pitch angle control input for the three blades; This indicates wind speed disturbance input; These represent the system state matrix, the effect matrix of pitch angle control input on the state, the influence matrix of wind speed disturbance on the state, the system output matrix, the transfer matrix of pitch angle control input to output, and the transfer matrix of wind speed disturbance to output, respectively.

3. The fuzzy predictive control method for floating wind turbines based on reinforcement learning according to claim 1, characterized in that, In step (2), the Long Short-Term Memory network is composed of an input layer, an LSTM layer, a Dropout layer, a fully connected layer, and an output layer connected in sequence to predict the future wind speed sequence. .

4. The fuzzy predictive control method for floating wind turbines based on reinforcement learning according to claim 1, characterized in that, In step (4), the fuzzy logic specifically refers to: Multiple control sub-intervals are constructed based on the wind speed range above the rated wind speed, and each control sub-interval corresponds to a model predictive controller. Corresponding membership functions are set for each model predictive controller, and the participation degree and output weight of each model predictive controller under the current operating condition are determined based on the average wind speed at the current time or within the sliding time window. In this way, the optimal pitch control commands output by multiple model predictive controllers are weighted and fused.

5. The fuzzy predictive control method for floating wind turbines based on reinforcement learning according to claim 4, characterized in that, By employing a membership function with interval overlap characteristics, the model predictive controllers corresponding to adjacent control sub-intervals can jointly participate in control within the overlapping region. When a floating wind turbine operates within a certain control sub-section, it is controlled by the corresponding single model predictive controller; when the floating wind turbine operates in the transition region of an adjacent control sub-section, it is controlled by two adjacent model predictive controllers, thereby avoiding control abrupt changes caused by controller switching.

6. The fuzzy predictive control method for floating wind turbines based on reinforcement learning according to claim 1, characterized in that, In step (4), the saturation constraint is used to limit the pitch angle of each blade within the allowable physical range, and the rate constraint is used to limit the rate of change of the pitch angle within a unit sampling period.

7. The fuzzy predictive control method for floating wind turbines based on reinforcement learning according to claim 1, characterized in that, In step (5), a TD3 reinforcement learning agent is used. After training, the agent outputs the corresponding weight adjustment amount in real time according to the current state input of the floating wind turbine, and corrects the objective function weight matrix of each model predictive controller online. The status inputs include generator speed error, tower forward and backward deflection, platform pitch motion status, and blade root load information for the three blades.

8. A fuzzy predictive control system for floating wind turbines based on reinforcement learning, characterized in that, The system includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the fuzzy predictive control method for floating wind turbines as described in any one of claims 1-7.

Images