A wind turbine blade load reduction adaptive control method based on reinforcement learning and bayesian optimization
Patent Information
- Application Number
- CN202611303625.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-26
- Publication Date
- 2026-11-03
AI Technical Summary
Existing blade load reduction control methods lack safety constraints and online adaptive capabilities in wind turbine units, leading to simulation deployment failures and an inability to maintain stable control performance under different wind farm environments and unit aging conditions.
In a simulation environment, the independent blade pitch control strategy is trained through reinforcement learning, with load over-limit as a constraint. After deployment on a real wind turbine, Bayesian optimization is used for online parameter adjustment. Combined with Lagrange multiplier constraints, safety boundaries are ensured, and the system gradually adapts to wind field characteristics and turbine aging characteristics.
Stable load reduction control was achieved in different wind farms and throughout the life cycle of the unit, avoiding control actions that exceeded the equipment's tolerance limits. The controller's adaptability and safety were ensured through calibration strategies based on measured data.
Smart Images
Figure CN123028121A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent control technology for wind power generation, specifically a wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization. Background Technology
[0002] When large wind turbines operate under complex wind conditions such as turbulence, wind shear, and gusts, the alternating aerodynamic loads borne by the blade roots are the main factors leading to blade fatigue, bolt breakage, and damage to the transmission system. To reduce these loads, independent pitch control technology is widely used, which reduces unbalanced loads by independently adjusting the pitch angle of each blade.
[0003] Existing blade load reduction control methods are not adaptable to highly nonlinear and time-varying systems. They also lack safety constraint mechanisms and effective means of transferring simulations to reality, making it difficult to maintain stable control performance under different wind farm environments and unit aging conditions.
[0004] Currently, the mainstream load reduction control methods in the industry are proportional-integral control (PIC) or linear quadratic regulators based on linearized models. These methods linearize the wind turbine at a specific operating point, design controllers with fixed parameters, and achieve feedback regulation of the blade root load. Model predictive control and nonlinear control methods, through a more accurate mathematical description of the aerodynamic-structural coupling model of the wind turbine, predict future responses based on the current system state and optimize the control sequence, theoretically offering better handling of nonlinear characteristics. Reinforcement learning methods, through autonomous learning of control strategies via interaction with a simulation environment, have made breakthroughs in fields such as robot control and autonomous driving. Their ability to learn complex nonlinear mappings without requiring an accurate model has attracted attention in the wind power sector.
[0005] The existing solutions described above have inherent limitations. Proportional-integral control (PIC) or linear quadratic regulators based on linearized models can only be designed for a single operating point. However, the aerodynamic characteristics of wind turbines vary significantly across different wind speed ranges, and the control performance deteriorates noticeably when the wind speed deviates from the operating point. Model predictive control and nonlinear control methods rely on accurate system models and high online computing power, making large-scale deployment in actual turbine controllers difficult. Constraints in computing resources and modeling accuracy limit their engineering practicality. Applying reinforcement learning directly to wind turbine load control faces three core challenges: first, the training process may generate control actions exceeding safety limits, causing damage to the equipment; second, the difference between the simulation environment and the real wind turbine leads to a decline in the effectiveness of the trained strategy after deployment, i.e., the simulation-to-real-world transfer problem; and third, the climate characteristics and turbine aging characteristics differ across wind farms, making it difficult for fixed strategies to adaptively adjust. In summary, existing methods cannot achieve reliable transfer of control strategies from simulation to reality and adaptation under different wind farm conditions while ensuring safety. Summary of the Invention
[0006] This invention provides an adaptive control method for wind turbine blade load reduction based on reinforcement learning and Bayesian optimization, which solves the problem that existing blade load reduction control methods lack safety constraints and online adaptive capabilities, leading to simulation deployment failures.
[0007] To achieve the above objectives, the present invention provides the following technical solution: An adaptive control method for load reduction of wind turbine blades based on reinforcement learning and Bayesian optimization includes the following steps: A Markov decision process framework is established in a simulation environment, and an independent blade pitch control strategy is trained by reinforcement learning algorithm. During the training process, the degree of blade load exceeding the limit is added as a constraint to the optimization objective to obtain an offline training strategy. The offline training strategy was deployed to a real wind turbine controller, and measured data of the wind turbine were collected during operation. The Bayesian optimization method is used to adaptively adjust the controller parameters online, using the measured data as observation samples. Each time Bayesian optimization is updated, the new parameter configuration is loaded into the strategy network, and the controller gradually adapts to the wind farm characteristics and unit aging characteristics as operating data accumulates.
[0008] Preferably, the reinforcement learning algorithm is a proximal policy optimization algorithm, which ensures the stability of the training process by limiting the magnitude of each policy update.
[0009] Preferably, incorporating the degree of blade load exceeding the limit as a constraint into the optimization objective includes: Set load safety constraints as follows ,in This indicates the expected extent of load exceeding limits under the current strategy. The preset load safety threshold is used; the strategy update direction is dynamically adjusted by Lagrange multipliers so that the strategy generated during training satisfies the load safety constraint.
[0010] Preferably, the Lagrange multipliers are non-negative. When the policy violates the safety constraints, the Lagrange multipliers are increased to apply a penalty, and when the policy satisfies the safety constraints, the penalty term is zero.
[0011] Preferably, the load safety constraints and Lagrange multiplier constraints are optimized in the following form:
[0012] in, For Lagrange multipliers, [This refers to the expected reward.] To minimize the value of the Lagrange multipliers under the condition that they are nonnegative, To obtain the maximum value in the policy parameter space.
[0013] Preferably, the objective function of the near-end strategy optimization algorithm is:
[0014] in, Indicates the pitch angle. [] represents the mathematical expectation calculation. To compare the old and new strategies and measure the magnitude of the strategy update, it is the ratio of the probability of the current strategy's output action to the probability of the action output by the strategy before the update. Let be the advantage function, representing how much higher the performance of an action is compared to the average level in a given state. The representative will Limited to [ Within the interval, This is the clipping threshold.
[0015] Preferably, the state space of the Markov decision process framework includes wind speed, rotor speed, measured bending moment at the blade root, pitch angle, and blade azimuth angle, while the action space consists of the independent pitch command increments for the three blades.
[0016] Preferably, the reward function of the reinforcement learning algorithm is a weighted sum of the load term and the power loss term:
[0017] in, and These are the weighting coefficients for the load term and the power loss term, respectively. Let t be the leaf root bending moment. This represents the design maximum value of the blade root bending moment. Let be the power loss at time t, which is the difference between the theoretical maximum power and the actual power. This is the maximum reference value for power loss, which can be set as the rated power.
[0018] Preferably, the online adaptive adjustment of controller parameters using a Bayesian optimization method includes: The weight coefficients in the reward function and the hyperparameters of the policy network are used as variables to be optimized. The objective function is fitted using a Gaussian process surrogate model, and the optimal parameter configuration is updated iteratively by maximizing the acquisition function. The function to maximize the acquisition is:
[0019] in, Indicates the expected improvement value. Let be the parameter vector to be optimized. , The learning rate represents the policy network. This represents the optimal parameter configuration that has been observed so far.
[0020] Preferably, the iterative process of online adaptive adjustment is as follows: The existing observation data is fitted using a Gaussian process surrogate model. The point that maximizes the acquisition function is searched in the parameter space as the next set of parameters to be tested. After the new parameter configuration is deployed on the wind turbine and run, the load reduction effect is measured. The new observation points are added to the dataset to update the Gaussian process model.
[0021] Compared with existing technologies, the present invention has the following advantages: The present invention provides an adaptive control method for wind turbine blade load reduction based on reinforcement learning and Bayesian optimization. A Markov decision process framework is established in a simulation environment, and the independent blade pitch control strategy is trained by reinforcement learning. This allows the control strategy to autonomously learn the optimal pitch action sequence under nonlinear aerodynamic characteristics without a precise system model, avoiding the dependence of traditional linear controllers on a single operating point. During the training process, the degree of blade load exceeding the limit is added as a constraint to the optimization objective, so that the strategy update direction is always constrained by the safety boundary, avoiding the generation of control actions that exceed the equipment tolerance limit during the reinforcement learning exploration phase. The offline training strategy is deployed to the real wind turbine controller, and after collecting measured data during operation, the controller parameters are adjusted online adaptively using the Bayesian optimization method. This allows the strategy trained in the simulation environment to be calibrated successively through real operating data, solving the problem of dynamic differences between the simulation environment and the real wind turbine. After each Bayesian optimization update, the new parameter configuration is loaded into the strategy network, so that the controller gradually adapts to the climate characteristics of the wind farm and the aging characteristics of the unit as the operating data accumulates, maintaining the control effect under different wind farm environments and stages of the unit's life cycle. Attached Figure Description
[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a flowchart of an adaptive control method for wind turbine blade load reduction based on reinforcement learning and Bayesian optimization, according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating the two-stage process of offline training and online adaptive training in an embodiment of the present invention. Figure 3 This is a schematic diagram of the interaction framework between the reinforcement learning agent and the environment in an embodiment of the present invention. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0027] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.
[0028] See Figure 1 In one embodiment of the present invention, a wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization is provided, which solves the technical problem that existing blade load reduction control methods lack safety constraints and online adaptive capabilities, leading to simulation deployment failures. This method trains an independent blade pitch control strategy in a simulation environment using reinforcement learning and uses load exceeding limits as a safety constraint. After deploying the offline strategy to a real wind turbine, Bayesian optimization is used to adjust the controller parameters online, enabling the controller to adapt to wind farm characteristics and turbine aging as operational data accumulates.
[0029] See Figure 1 and Figure 2 The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization provided in one embodiment of the present invention is divided into two stages: offline training and online adaptation, and includes the following steps.
[0030] S1: Establish a Markov decision process framework in a simulation environment, train the blade independent pitch control strategy through reinforcement learning algorithm, and add the degree of blade load exceeding the limit as a constraint to the optimization objective during the training process to obtain the offline training strategy.
[0031] Specifically, a Markov decision process framework is established in a high-fidelity wind turbine simulator. The state space is defined as easily measurable or estimable state variables such as wind speed, rotor speed, measured bending moment at the blade root, pitch angle, and blade azimuth angle; the action space is defined as the independent pitch command increments of the three blades; and the reward function is defined as a negative weighted sum, i.e., minimizing the combined objective of blade fatigue load and power loss.
[0032] reward function Defined as a negative weighted sum, penalizing both load and power loss:
[0033] in, and These are the weighting coefficients for the load term and the power loss term, respectively, with values ranging from 0 to 1, and satisfying the following conditions: + =1, Let t be the leaf root bending moment. This represents the design maximum value of the blade root bending moment. Let be the power loss at time t, which is the difference between the theoretical maximum power and the actual power. This is the maximum reference value for power loss, which can be set as the rated power.
[0034] Interpretively, the design of the reward function directly determines the optimization objective of the reinforcement learning agent. Since load and power loss have different dimensions and cannot be directly added, a normalization process is used—the leaf root moment is divided by the design maximum value. Rate loss divided by rated power The terms are transformed into dimensionless relative values, thus enabling weighted summation. and The relative value of determines the bias of the control strategy between reducing load and maintaining power generation.
[0035] The reinforcement learning algorithm employs a proximal policy optimization algorithm, which ensures the stability of the training process by limiting the magnitude of each policy update. The objective function of this algorithm is:
[0036] in, Indicates the pitch angle. [] represents the mathematical expectation calculation. To compare the old and new strategies and measure the magnitude of the strategy update, it is the ratio of the probability of the current strategy's output action to the probability of the action output by the strategy before the update. Let be the advantage function, representing how much higher the performance of an action is compared to the average level in a given state. The representative will Limited to [ Within the interval, The recommended value for the cropping threshold is 0.2.
[0037] The intelligent agent interacts with the simulation environment millions of times, accumulates experience data and updates the policy network parameters, and gradually learns the optimal load reduction control strategy under various wind conditions.
[0038] A Lagrangian constraint mechanism is introduced into the reinforcement learning training framework. Safety thresholds are set for the instantaneous peak value of the bending moment at the blade root and the fatigue equivalent load, and load exceeding these thresholds is added as a constraint condition to the optimization objective. Specifically, the load safety constraints are set as follows:
[0039] in, This indicates the expected extent of load exceeding limits under the current strategy. This indicates the preset load safety threshold.
[0040] The above constraint problem can be transformed into the following form:
[0041] in, The Lagrange multiplier is a non-negative number, dynamically adjusted to balance load reduction benefits and safety constraints. ] represents the expected reward, which is the overall evaluation of the control effect. To minimize the value of the Lagrange multipliers under the condition that they are non-negative (inner optimization). To maximize the value in the policy parameter space (outer optimization).
[0042] The physical meaning of this formula is: when the strategy violates security constraints, i.e. hour, If it is a positive number, multiply by Subtracting this from the reward is equivalent to imposing a penalty; the penalty term is zero when the policy satisfies safety constraints. The system automatically adjusts its strategy, increasing penalties for frequent violations of limits. The agent interacts with the simulation environment for millions of steps, updating its policy according to the above rules at each step, gradually converging to the optimal policy that satisfies the safety boundaries.
[0043] S2: Deploy the offline training strategy to the real wind turbine controller and collect the actual measurement data of the wind turbine during operation.
[0044] Specifically, the policy network trained in step S1 is deployed to the actual wind turbine controller. During operation, measured data from the wind turbine are collected, including wind speed sequences, load response (blade root bending moment), and pitch control actions.
[0045] Interpretive offline training is conducted in a high-fidelity simulator. While the simulator can accurately simulate the aerodynamic-structural characteristics of wind turbines, differences still exist between it and real wind turbines (such as nonlinear effects not covered by the model, sensor noise characteristics, and differences in actuator dynamic response). Therefore, the strategies trained in the simulation need to be deployed on real wind turbines first, and their actual performance needs to be evaluated using measured data, providing a data foundation for subsequent online adaptive adjustments.
[0046] S3: Using the Bayesian optimization method, the measured data is used as the observation sample to perform online adaptive adjustment of the controller parameters.
[0047] Specifically, the weighting coefficients in the reward function (the relative weights of the load term and the power loss term) , and key hyperparameters of the policy network (such as the learning rate) ) as the variable to be optimized The objective function is fitted using a Gaussian process surrogate model, and real-world operating data is collected as observation samples. The optimal parameter configuration is then iteratively updated by maximizing the collected function.
[0048] The goal of Bayesian optimization is to maximize the actual load reduction effect function. The core of Bayesian optimization is to maximize the expected improvement of the acquisition function.
[0049] in, Indicates the expected improvement value, and represents the parameter. How much improvement compared to the current best? Let be the parameter vector to be optimized. , The learning rate represents the policy network. This represents the optimal parameter configuration that has been observed so far.
[0050] The execution flow of Bayesian optimization is as follows: First, fit the existing observation data using a Gaussian process surrogate model to give the objective function. The second step is to estimate the mean and variance; the third step is to search for the maximum value in the parameter space. The points are used as the next set of test parameters; the third step is to deploy the new parameter configuration on the wind turbine and run it for a period of time to obtain the load reduction effect through actual measurement. The fourth step is to add the new observation points to the dataset and update the Gaussian process model.
[0051] Interpretive, Gaussian process surrogate models can provide mean and variance estimates of the objective function under small sample conditions—the mean represents the expected load reduction effect under those parameters, and the variance represents the uncertainty in the prediction. Acquisition function The parameters are set higher in positions with high mean and large variance. Therefore, it can perform a fine search near the current optimal solution while also actively exploring parameter regions that have not yet been tried, achieving a balance between utilization and exploration, and finding the optimal parameter combination with as few real wind turbine tests as possible.
[0052] S4: After each Bayesian optimization update, the new parameter configuration is loaded into the strategy network, and the controller gradually adapts to the wind farm characteristics and unit aging characteristics as the operating data accumulates.
[0053] Specifically, after each Bayesian optimization update, the new parameter configuration is loaded into the policy network, and the controller adjusts accordingly. As operational data accumulates, the controller gradually adapts to the climate characteristics of the wind farm and the aging characteristics of the turbine.
[0054] Repeat the Bayesian optimization iterative process described above, and the parameter configuration will be optimized after each iteration.
[0055] Explanatoryly, different wind farms exhibit significant differences in climatic characteristics such as turbulence intensity, average wind speed, and wind direction variation patterns, making it difficult for fixed-parameter controllers to achieve optimal performance across all wind farms. Furthermore, during long-term operation, wind turbine blade stiffness decreases due to fatigue, and pitch system response delay increases due to wear; fixed strategies cannot automatically compensate for these changes. Bayesian optimization, by continuously collecting real-world operating data under the current wind farm and turbine conditions, repeatedly calibrates controller parameters, ensuring the strategy network always matches the current environmental conditions, thus maintaining good load reduction control performance throughout the entire lifecycle. Repeating this iterative process allows parameter configuration to be continuously optimized with data accumulation, enabling adaptive adjustments without manual intervention.
[0056] Validity verification In one possible implementation, the technical effects and feasibility of the present invention are illustrated by taking the independent pitch control of a large wind turbine blade as an example.
[0057] A Markov decision process framework was built in a high-fidelity wind turbine simulator, and a proximal policy optimization algorithm was used to train the blade-independent pitch control strategy. The training process covered the entire wind speed range from cut-in wind speed to cut-out wind speed, including various wind conditions such as different turbulence intensities and extreme gusts. During training, a Lagrangian constraint mechanism was used to ensure that the strategy met the blade root bending moment safety boundary.
[0058] The reward function weight coefficients are: w1=0.7 (load term) and w2=0.3 (power loss term), which can be adjusted online according to Bayesian optimization.
[0059] Near-end strategy optimization parameters: pruning threshold The policy network is a multilayer perceptron structure, and the learning rate α is a hyperparameter to be optimized.
[0060] Lagrange constraints: load safety threshold The Lagrange multiplier is determined based on the design limit of the bending moment at the blade root. The initial value is 0, and it is automatically adjusted during training.
[0061] Bayesian optimization: Parameters to be optimized The acquisition function is expected to be improved.
[0062] During the offline training phase, the agent interacts with the simulation environment for millions of steps, collecting experience data and updating the policy network parameters to gradually learn the load reduction control strategy. The Lagrange multipliers are dynamically adjusted during training to ensure the policy converges to the optimal solution that satisfies the safety boundary.
[0063] The trained policy network is deployed to a real wind turbine controller to collect measured data such as wind speed sequence, load response, and pitch action.
[0064] In the online adaptive phase, using measured data as observation samples, Bayesian optimization is employed to optimize the data. , and learning rate Iterative optimization is performed. In each iteration, the new parameter configuration is loaded into the policy network, and the load reduction effect is measured on the wind turbine. After updating the Gaussian process model, the next iteration begins.
[0065] The policy network trained in this method in a simulation environment, after deployment on real wind turbines, can adapt to the climatic characteristics of different wind farms and the aging characteristics of the turbines through continuous calibration via Bayesian optimization. The Lagrange constraint mechanism effectively prevents exploration actions from exceeding limits in the early stages of training, ensuring that safety boundary constraints remain effective throughout. The two-stage architecture of offline training and online Bayesian adaptive optimization allows the controller to continuously optimize as operational data accumulates, enabling parameter adjustments without manual intervention. This maintains stable load reduction performance in different wind farm environments and at different stages of the turbine's lifecycle, outperforming fixed-parameter controllers.
[0066] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
[0067] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.
Claims
1. A wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization, characterized in that, Includes the following steps: A Markov decision process framework is established in a simulation environment, and an independent blade pitch control strategy is trained by reinforcement learning algorithm. During the training process, the degree of blade load exceeding the limit is added as a constraint to the optimization objective to obtain an offline training strategy. The offline training strategy was deployed to a real wind turbine controller, and measured data of the wind turbine were collected during operation. The Bayesian optimization method is used to adaptively adjust the controller parameters online, using the measured data as observation samples. Each time Bayesian optimization is updated, the new parameter configuration is loaded into the strategy network, and the controller gradually adapts to the wind farm characteristics and unit aging characteristics as operating data accumulates.
2. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 1, characterized in that, Reinforcement learning algorithms are proximal policy optimization algorithms that ensure the stability of the training process by limiting the magnitude of each policy update.
3. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 2, characterized in that, Incorporating the degree of blade load exceeding limits as a constraint into the optimization objective includes: Set load safety constraints as follows ,in This indicates the expected extent of load exceeding limits under the current strategy. The preset load safety threshold is used; the strategy update direction is dynamically adjusted by Lagrange multipliers so that the strategy generated during training satisfies the load safety constraint.
4. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 3, characterized in that, The Lagrange multiplier is a non-negative number. When the policy violates the safety constraints, the Lagrange multiplier increases to impose a penalty. When the policy satisfies the safety constraints, the penalty term is zero.
5. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 3, characterized in that, The optimized form of the load safety constraints and Lagrange multiplier constraints is as follows: in, For Lagrange multipliers, [This refers to the expected reward.] To minimize the value of the Lagrange multipliers under the condition that they are nonnegative, To obtain the maximum value in the policy parameter space.
6. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 2, characterized in that, The objective function of the near-end policy optimization algorithm is: in, Indicates the pitch angle. [] represents the mathematical expectation calculation. To compare the old and new strategies and measure the magnitude of the strategy update, it is the ratio of the probability of the current strategy's output action to the probability of the action output by the strategy before the update. Let be the advantage function, representing how much higher the performance of an action is compared to the average level in a given state. The representative will Limited to [ Within the interval, This is the clipping threshold.
7. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 1, characterized in that, The state space of the Markov decision process framework includes wind speed, rotor speed, measured bending moment at the blade root, pitch angle, and blade azimuth angle, while the action space consists of independent pitch command increments for the three blades.
8. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 1, characterized in that, The reward function of a reinforcement learning algorithm is a weighted sum of the payload term and the power loss term: in, and These are the weighting coefficients for the load term and the power loss term, respectively. Let be the leaf root bending moment at time t. This represents the design maximum value of the leaf root bending moment. Let be the power loss at time t, which is the difference between the theoretical maximum power and the actual power. This is the maximum reference value for power loss, which can be set as the rated power.
9. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 1, characterized in that, Online adaptive adjustment of controller parameters using Bayesian optimization methods includes: The weight coefficients in the reward function and the hyperparameters of the policy network are used as variables to be optimized. The objective function is fitted using a Gaussian process surrogate model, and the optimal parameter configuration is updated iteratively by maximizing the acquisition function. The function to maximize the acquisition is: in, Indicates the expected improvement value. Let be the parameter vector to be optimized. , The learning rate represents the policy network. This represents the optimal parameter configuration that has been observed so far.
10. The wind turbine blade load reduction adaptive control method based on reinforcement learning and Bayesian optimization according to claim 1, characterized in that, The iterative process of online adaptive adjustment is as follows: The existing observation data is fitted using a Gaussian process surrogate model. The point that maximizes the acquisition function is searched in the parameter space as the next set of parameters to be tested. After the new parameter configuration is deployed on the wind turbine and run, the load reduction effect is measured. The new observation points are added to the dataset to update the Gaussian process model.