Wind turbine generator PID controller adaptive setting method and system based on reinforcement learning

Abstract

The invention discloses a wind turbine generator PID controller adaptive setting method and system based on reinforcement learning, and the method comprises the steps: constructing a state space containing the wind speed, the rotating speed, the power, the pitch angle, the torque and the mechanical load, training an intelligent body through a DDPG algorithm, and outputting PID parameter increments (delta Kp, delta Ki and delta Kd); and the power tracking precision, the rotating speed stability, the mechanical load suppression, the action smoothness and the coupling interference between the variable pitch and the torque control are cooperatively optimized through a multi-target comprehensive reward function. An event-driven mechanism is adopted in the setting process, online fine adjustment is triggered when the wind speed abrupt change is larger than or equal to 2 m / s or the power deviation continuously exceeds the limit (larger than or equal to 5% of rated power reaches 10 seconds), and a dynamic learning rate attenuation and parameter rollback safety mechanism is introduced. According to the method, global joint optimization without an accurate model is realized, and the self-adaptability, the robustness and the operation life of the wind turbine generator under all working conditions are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of wind turbine PID controller, and particularly relates to a wind turbine PID controller adaptive tuning method and system based on reinforcement learning. BACKGROUND

[0002] The existing wind turbine (especially variable pitch, variable speed wind turbine) control usually adopts the classic PID (proportional-integral-derivative) controller, which is the most mainstream method at present. The existing technology usually decomposes the complex wind turbine joint control problem into two independent subtasks: one is to adjust the pitch angle β through the variable pitch controller to track the optimal generator speed; the other is to adjust the generator torque Tg through the torque controller to track the target power. Each sub-controller is usually a PID, and the parameters are tuned offline, such as manual tuning or theoretical calculation based on a simplified model. The main disadvantages are as follows: (1) Ignoring the strong coupling effect between subsystems: the variable pitch and torque control systems of the wind turbine are highly coupled in physics. Independent design and tuning will ignore this dynamic interaction, resulting in the inability to achieve global optimal control of the entire system, and even may cause oscillation or overshoot due to coupling interference.

[0003] (2) Dependence on simplified model, poor robustness: the controller design relies on a simplified linear model of the wind turbine dynamics, while the actual wind turbine is a strong nonlinear, time-varying, uncertain complex system. The simplified model cannot accurately reflect the real working conditions (such as turbulent wind, mechanical fatigue, etc.), resulting in a sharp decline in the performance of the fixed parameter PID controller under model mismatching conditions, and insufficient robustness.

[0004] (3) Unable to adapt online: the offline tuned parameters are fixed and cannot be adjusted online according to real-time changing wind conditions (such as wind speed, wind direction, turbulence intensity) and turbine operating conditions (such as aging, component wear), making it difficult to maintain optimal performance in the full operating range.

[0005] In view of the above core problems in the prior art that the subsystem coupling is ignored, the controller parameters cannot be adapted online, and the dependence on the simplified model leads to poor robustness, the present application aims to solve the following technical problems: (1) How to realize the joint and global optimization of the variable pitch and torque control subsystems of the wind turbine, avoid the coupling effect ignored by independent design, and thus improve the power generation efficiency and operating stability of the entire wind turbine.

[0006] (2) How to construct an adaptive tuning mechanism that does not depend on an accurate physical model, so that the parameters of the PID controller can be adjusted online and autonomously according to real-time wind conditions and turbine conditions, to cope with the nonlinear, time-varying and uncertain nature of the wind turbine, and ensure robustness and optimal performance in the full operating range.

[0007] (3) How to design a computationally efficient reinforcement learning architecture suitable for engineering deployment, which can both utilize the powerful self-learning and optimization capabilities of reinforcement learning and meet the stringent real-time requirements of wind turbine control. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide an adaptive tuning method and system for PID controllers of wind turbines based on reinforcement learning. The present invention achieves global joint optimization without the need for an accurate model, which significantly improves the adaptability, robustness and service life of wind turbines under all operating conditions.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An adaptive tuning method for a PID controller of a wind turbine based on reinforcement learning includes the following steps: S1. Construct a reinforcement learning environment, defining the state space S, action space A, and multi-objective comprehensive reward function R; S2. Train the reinforcement learning agent based on the Deep Deterministic Policy Gradient (DDPG) algorithm, so that it outputs the incremental adjustment of the PID controller parameters. S3. During the operation of the wind turbine, online fine-tuning is triggered by an event-driven approach. When the preset triggering conditions are met, the reinforcement learning agent is used to perform online adaptive tuning of the PID controller parameters corresponding to the pitch control and / or torque control.

[0010] Preferably, the state space S includes state variables that are collected in real time and normalized by wind turbine sensors and SCADA system. The state variables include at least: wind speed, generator speed, actual output power, pitch angle, transmission chain torque, and mechanical load of key components.

[0011] Preferably, the action space A is a three-dimensional continuous action vector. A t ={Δ K p Δ K i Δ K d}, respectively, correspond to the proportionality coefficient. K p Integral coefficient K i and differential coefficients K d The adjustment increments are all limited within a preset safety boundary: Δ K p ∈[-0.1,0.1], Δ K i∈ [-0.05, 0.05], Δ K d ∈ [-0.02, 0.02]; meanwhile, the absolute value of the integral coefficient K1 does not exceed 50% of its initial setting value.

[0012] Preferably, the multi-objective comprehensive reward function R is composed of the following weighted sub-items: (1) power tracking accuracy reward R power for minimizing the deviation of actual power from target power; (2) generator speed stability penalty R stability for suppressing speed fluctuation; (3) total mechanical load penalty R load for reducing the fatigue load of key components; (4) actuator action smoothness penalty R smoothness for reducing the sharp changes in pitch angle and generator torque; (5) coupling interaction penalty R coupling for explicitly suppressing the dynamic coupling interference between the pitch control system and the torque control system.

[0013] Preferably, the weights of the sub-items in the multi-objective comprehensive reward function are dynamically adjusted according to the current wind speed interval: in the low wind speed zone, the weight of power tracking accuracy is emphasized, and in the high wind speed zone, the weights of mechanical load and coupling interaction penalty are emphasized.

[0014] Preferably, the event-driven online fine-tuning trigger condition includes any of the following situations: (a) wind speed change Δv ≥ 2 m / s; (b) output power deviation ΔP ≥ 5% of rated power and the deviation duration ≥ 10 seconds.

[0015] Preferably, a dynamic learning rate strategy is adopted in the online fine-tuning process: the initial learning rate is 0.001, and then it is exponentially decayed by a decay factor of 0.95 every 100 steps until it stabilizes at 0.0001.

[0016] Preferably, an engineering safety mechanism is also included: if parameter adjustment for 5 consecutive times leads to a decrease in the reward function value, immediately freeze parameter update and roll back PID parameters to the last parameter set with stable reward.

[0017] A wind turbine PID controller adaptive setting system, comprising: a data acquisition module for obtaining real-time operating state data of the wind turbine; a state preprocessing module for normalizing the state data; The reinforcement learning agent module based on the DDPG architecture comprises an actor network and a critic network, and is used for outputting PID parameter increments and evaluating strategy performance; The parameter updating module is used for updating the PID controller parameters according to the agent output and safety constraints. The trigger judgment module is used for monitoring whether the event trigger condition of online fine-tuning is met. The safety monitoring module is used for performing parameter freezing and rollback operations.

[0018] Preferably, the actor network is a three-layer fully connected neural network, and the output is scaled to the action boundary after being activated by tanh; the critic network is a two-layer fully connected network, the input is a splicing vector of a state and an action, and the output is a Q value; during the training process, experience replay, target network soft update and gradient clipping mechanisms are adopted to guarantee training stability.

[0019] The present application can achieve the following beneficial effects: 1. The present application synchronously sets the PID parameters of the variable pitch control and torque control through the reinforcement learning framework, and explicitly introduces a coupling interaction penalty term in the reward function, effectively suppressing the dynamic coupling interference between the two subsystems, overcoming the control performance decline or oscillation problem caused by ignoring the coupling effect in the traditional independent setting method, and improving the overall operation efficiency and stability of the wind turbine.

[0020] 2. The present application adopts a model-free deep reinforcement learning method (DDPG), which does not need to rely on the accurate physical model of the wind turbine, and can adjust the PID parameters online according to the real-time wind conditions (such as sudden change of wind speed, turbulence) and the unit state (such as component aging, wear), effectively dealing with the nonlinearity, time-varying nature and uncertainty of the system, and maintaining excellent control performance in the whole working condition range.

[0021] 3. The present application designs a multi-objective comprehensive reward function including power tracking accuracy, generator speed stability, total mechanical load, actuator action smoothness and coupling interaction penalty, and dynamically adjusts the weight of each item according to the current wind speed range - low wind speed area focuses on power tracking to maximize energy capture, high wind speed area focuses on mechanical load suppression and coupling penalty to ensure structural safety, and takes into account power generation efficiency and equipment life.

[0022] 4. The present application adopts an event-driven online fine-tuning mechanism, which only triggers parameter updating when the wind speed change is greater than or equal to 2 m / s or the power deviation is continuously out of limit (greater than or equal to 5% of rated power and lasts for more than 10 seconds), which significantly reduces the consumption of computing resources; at the same time, the safety mechanism of freezing and rolling back when the reward decreases for 5 consecutive times is set, which effectively prevents parameter oscillation or loss of control, and ensures the reliability and recoverability of the system in the real running environment. BRIEF DESCRIPTION OF DRAWINGS

[0023] The application will be further described below in conjunction with the accompanying drawings and examples: Fig. 1 The parameter optimization architecture diagram of the Actor-Critic RL-based controller of the application; Fig. 2 The online fine-tuning trigger logic and learning rate decay flowchart of the application. DETAILED DESCRIPTION

[0024] The preferred scheme is as shown in Figs. 1-2 A PID controller adaptive tuning method and system for wind turbine based on reinforcement learning, An innovative method of applying reinforcement learning (RL) technology to online adaptive tuning of key PID controllers (such as variable pitch control and torque control) for wind turbines. This method realizes dynamic adjustment of PID parameters by constructing a reinforcement learning environment, defining state space, action space and reward function, and using Deep Deterministic Policy Gradient (DDPG) algorithm for training and deployment. The specific scheme is as follows: S1. Construction of reinforcement learning environment and data flow specification: The reinforcement learning environment defines the interface between the agent (Agent) and the wind turbine system, which is the basis for implementing the adaptive tuning strategy. The core task of this stage is to standardize the input data (state S), define the output action (action A), and design a multi-objective reward function R to evaluate the pros and cons of the strategy.

[0025] S1.1 Definition of state space S and real-time data acquisition: The state space S constitutes the basis for the agent to make decisions. It is composed of real-time operating data of the wind turbine at each time step t to reflect the physical state and performance of the system.

[0026] S1.1.1 Composition of state variables: State data is collected in real time through the SCADA system and sensor network of the wind turbine, forming a high-dimensional state vector S t . These data provide the agent with complete system operating condition information, enabling it to perceive environmental changes and make appropriate parameter adjustments. The state vector S at least includes the following key operating states and mechanical load data: wind speed v (unit: m / s), generator speed ω g (rpm), actual output power P act (kW), pitch angle β (°), transmission chain torque Tg (Nm), and mechanical load or stress J at key positions (such as blade bottom, tower bottom, main shaft)total (N or Pa).

[0027] S1.1.2 The data input flow follows this order: Real-time sensor data → SCADA → Data preprocessing module → Normalized state vector S t → DDPG neural network input.

[0028] S1.1.3 Data Normalization Preprocessing: To ensure the stability and efficiency of deep neural network training, all state variables must undergo strict normalization preprocessing before being input into the DDPG network, i.e., normalization operation is performed on each variable x:

[0029] in x min and x max These represent the historical minimum and maximum values ​​of the variable, respectively. Normalization aims to eliminate differences between different physical units (e.g., the magnitude of wind speed is much smaller than that of mechanical load). If the magnitudes of the input data differ too much, the network weights may be dominated by the larger-scale inputs, thus affecting training convergence speed and performance. Through normalization, all features are mapped to similar numerical ranges, ensuring the stability of the training process.

[0030] S1.2 Definition and parameter adjustment constraints of motion space A: Action space A defines how the reinforcement learning agent intervenes in the PID controller of the wind turbine. The agent outputs incremental adjustments to the existing PID controller parameters, rather than absolute values, which ensures the gradualness and safety of the adjustments.

[0031] S1.2.1 Composition of action variables: The agent outputs a three-dimensional action vector. A t ={Δ K p Δ K i Δ K d}, respectively, correspond to the proportionality coefficient. K p Integral coefficient K i and differential coefficients K d The adjustment increment.

[0032] S1.2.2 Safety constraints of action space: Wind turbines belong to high-value and high-inertia systems, and drastic changes in PID parameters can lead to system oscillation or even loss of control. Therefore, the actions of the agent must be strictly bounded, which is the key engineering interface for combining reinforcement learning with classical control theory. The adjustment amplitude of each parameter is strictly constrained within a predefined range: Δ K p ∈[-0.1, 0.1],Δ K i ∈[-0.05, 0.05],Δ K d ∈[-0.02, 0.02]。

[0033] S1.2.3 Integral coefficient limitation: To prevent integral saturation or controller divergence, an absolute value constraint is imposed on the absolute value of the integral coefficient K i : |Δ K i | does not exceed 50% of its initial setting value. This safety limit is a core measure to ensure the robustness of the control system.

[0034] S1.3 Design of multi-objective comprehensive reward function R: The reward function R is the driving force guiding the agent to learn the optimal strategy. This method adopts a multi-objective comprehensive reward function, aiming to balance the power generation efficiency, mechanical load suppression and system stability of wind turbines, while explicitly addressing the coupling interference problem between subsystems.

[0035] S1.3.1 Mathematical expression and target weight of comprehensive reward function: The design form of the comprehensive reward function R weights each control target with weight ω i to achieve collaborative control: ; S1.3.2 Specific engineering definition of key sub-items R i : (1) Power tracking accuracy ( R power ): Measures the deviation of actual power from target power. Minimizing the deviation maximizes power generation efficiency.

[0036] ; (2) Generator speed standard deviation penalty ( R stability ): Suppresses the fluctuations of generator speed ω g to ensure operational stability.

[0037] ; (3) Total mechanical load penalty (Pmech) R load ): directly penalizes the fatigue load of key components such as blades and tower J total , which helps to extend the lifetime of the wind turbine.

[0038] ; (4) Actuator motion smoothness penalty (Psmooth) R smoothness ): penalizes the adjustment increment of pitch angle β and generator torque Tg , which avoids frequent or drastic actuator motion and reduces mechanical wear.

[0039] ; (5) Coupling interaction penalty (Pcouple) R coupling ): This term is the key innovation of the proposed method. Existing techniques usually ignore the strong dynamic coupling between pitch and torque control systems. By explicitly adding this penalty term, the agent is guided to find a joint optimization strategy that tends to be more smooth and coordinated, avoiding dynamic interference between the two subsystems, thus achieving the global joint optimization of the wind turbine.

[0040] ; S1.3.3 Dynamic adjustment logic of weight coefficients ω 1 ~ω 5: The weight coefficients ω i determine the priority of each objective under different operating conditions. Fixed weights ω i will lead to an optimal RL policy under a specific condition, but poor robustness across conditions.

[0041] (1) Initial determination: The initial values of weight coefficients ω 1 ~ω 5 are obtained by running a genetic algorithm optimization on historical wind farm data.

[0042] (2) Dynamic adjustment: In actual operation, the weights need to be dynamically adjusted according to the wind speed interval, which essentially introduces a condition switching mechanism, greatly enhancing the practicality of the RL model. For example: In the low wind speed region (partial load region), focus on ω 1 (power tracking) to achieve maximum power point tracking (MPPT); in the high wind speed region (full load region), focus on ω 3 (load) and ω 5 (coupling penalty) to prioritize mechanical integrity and system stability.

[0043] S2. Optimization of deep reinforcement learning strategy and parameter iteration setting: The policy optimization phase describes in detail how the agent (based on the DDPG algorithm) calculates and optimizes the output action A (i.e. the PID parameter increment) according to the state S and the reward R.

[0044] S2.1 Policy network design based on DDPG architecture: This method uses the deep deterministic policy (DDPG) algorithm. DDPG is a model-free, Actor-critic architecture algorithm that can directly output continuous action values, making it very suitable for fine-tuning PID parameters (continuous values) and avoiding control errors caused by traditional discretization.

[0045] S2.1.1 Architecture and output of Actor network (policy network): Actor network μ(S|θ μ ) Responsible for formulating strategies, i.e. outputting determined actions according to the current state A t {Δ K p , Δ K i , Δ K d}. The specific structure is a 3-layer fully connected neural network: input layer (state dimension, normalized state vector S t ), hidden layer [64, 32] neurons (using ReLU activation function), output layer (using tanh activation function to scale the value action boundary).

[0046] S2.1.2 Evaluation mechanism and Q value calculation of Critic network (value network): Critic network Q(S,A|θ Q ) Responsible for evaluating the long-term cumulative reward that the Actor network can obtain by taking action A in a given state S, i.e. the Q value. The specific structure is a 2-layer fully connected network: input layer (state and action concatenated vector), hidden layer [64, 64] neurons (using ReLU activation function), output layer (single value Q).

[0047] S2.2 Data flow and core algorithm of agent policy training S2.2.1 Experience replay and training stability enhancement: To break the time correlation of continuous interaction data (which is a common challenge in deep reinforcement learning), the agent will generate experience tuples (S, A, R, S') from its interaction with the environment and store them in a replay buffer. S t , A t ,R t , S t+1 The data is stored in the experience replay buffer pool D. The buffer pool capacity is set to 10. 6 During training, batch data is randomly sampled from the buffer pool to ensure the Independent and Identically Distributed (IID) nature of the training process, thereby significantly improving the stability and convergence speed of training.

[0048] S2.2.2 DDPG Network Update Process: DDPG ensures training stability through a target network and a soft update mechanism. 1) Value Network Update: Critic network parameters θ Q Updated by minimizing the mean squared error (MSE). 2) Policy network update: Actor network parameters θ μ Update along the gradient ascent direction, with the goal of maximizing the evaluation value of the Critic for the current policy. 3) Parameters of the target network θ' Through soft update coefficients τ Slowly track the main network parameters to avoid oscillations caused by rapid updates to the target value. The soft update formula is as follows: θ' ← τθ +(1- τ ) θ' Among them, the soft update coefficient τ Set to 0.005. 4) Gradient clipping: To prevent gradient explosion during training and subsequent divergence, gradient clipping is used, with the threshold set to 1.0.

[0049] S2.3 Iterative update process for controller parameter tuning S2.3.1 Determination of Setpoint and Parameter Update Formula: PID Parameters K p,t+1 , K i,t+1 , K d,t+1 In each tuning cycle t The update is based on the incremental adjustment amount of the agent's output. A t ={Δ K p Δ K i Δ K d During updates, the absolute values ​​of the parameters and the incremental boundary constraints defined in S1.2.2 must be applied (Clip). ; S2.3.2 Smoothness Requirements of the Strategy Exploration Mechanism: To ensure that the agent can effectively explore the PID parameter space and avoid getting trapped in local optima, the deterministic action A output by the Actor network must be smooth. t Superimposed exploration noise N t This forms the final execution action A. exe =A t +N t The exploration noise was generated using an Ornstein-Uhlenbeck (OU) stochastic process. The noise generated by the OU process is time-dependent and smooth, which is crucial for continuous control tasks in high-inertia physical systems like wind turbines. Using smooth noise avoids the impact of violent, discontinuous exploration movements on the wind turbine's mechanical structure, thus ensuring control stability.

[0050] S3, Online Adaptive Tuning: The online adaptive fine-tuning of PID parameters is not continuous, but rather adopts an event-driven mode, only starting when the system operating state changes significantly or the performance deviates from the target, in order to maximize the computational efficiency and operational stability of the control system.

[0051] S3.1 Online Fine-tuning Startup Logic and Triggering Conditions: The online fine-tuning feature aims to address the performance degradation of fixed-parameter PID controllers under highly nonlinear and time-varying conditions. This event-driven tuning mode restricts fine-tuning to critical states, enabling on-demand learning, optimizing the consumption of onboard computing resources, and ensuring that the system is controlled by a stable set of parameters most of the time.

[0052] S3.1.1 Wind Speed ​​Sudden Change Trigger Mechanism: When environmental input changes drastically, the system needs to adapt quickly. Sudden wind speed changes are one of the main reasons for the sharp decline in the performance of wind turbine control systems. The trigger condition is that the wind speed change Δv is greater than or equal to 2 m / s.

[0053] S3.1.2 Power Deviation Duration Judgment Mechanism: When the unit's operating performance deviates from the preset target for an extended period, it indicates that the current fixed PID parameters are no longer suitable for the current unit condition (such as aging or component wear), and adaptive learning needs to be initiated. The trigger condition is that the output power deviation ΔP is greater than or equal to 5% of the rated power, and this deviation must last for more than 10 seconds.

[0054] S3.2 Dynamic control strategy for adaptive learning rate: Dynamic management of the learning rate is key to ensuring the stability and convergence of online learning.

[0055] Initial learning rate setting and exponential decay strategy: 1) Initial learning rate: When fine-tuning is triggered, the learning rate is initially set to 0.001 to quickly respond to sudden events. 2) Exponential decay strategy: Subsequently, an exponential decay strategy is adopted, that is, the learning rate decays to 0.95 times the previous step every 100 steps. 3) Stabilized learning rate: After decay, the learning rate will stably decrease to 0.0001.

[0056] The purpose of this decay strategy is to achieve the control objective: a high initial learning rate ensures that the system quickly finds a new parameter region, while the subsequent decrease in the learning rate makes the parameter adjustment step size smaller, and the system enters a fine convergence phase. This effectively avoids the problem that the parameters may continue to oscillate around the optimal value due to a continuously high learning rate, and ensures the stable convergence of the control system.

[0057] S3.3 Engineering Safety Mechanism: To enhance the system's robustness in real-world operating environments and address inappropriate exploration or environmental noise interference during online learning, this method employs a robust parameter freezing and rollback mechanism. This mechanism provides a rapid and safe escape route, ensuring the recoverability of system operations.

[0058] Freeze condition: If the reinforcement learning agent performs 5 consecutive actions (i.e., PID parameter adjustments) that cause the reward R obtained by the system to decrease.

[0059] Operation execution: After the mechanism is activated, immediately perform the following operations: 1) Pause the online learning and updating of PID parameters; 2) Roll back the PID parameters to the previous set of parameters with stable rewards.

[0060] This safety mechanism is key to addressing the risk of exploration failures in online learning. It effectively prevents control system malfunctions or shutdowns caused by overexploration or parameter oscillations.

[0061] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. An adaptive tuning method for a PID controller of a wind turbine based on reinforcement learning, characterized in that, Includes the following steps: S1. Construct a reinforcement learning environment, defining the state space S, action space A, and multi-objective comprehensive reward function R; S2. Train the reinforcement learning agent based on the Deep Deterministic Policy Gradient (DDPG) algorithm, so that it outputs the incremental adjustment of the PID controller parameters. S3. During the operation of the wind turbine, online fine-tuning is triggered by an event-driven approach. When the preset triggering conditions are met, the reinforcement learning agent is used to perform online adaptive tuning of the PID controller parameters corresponding to the pitch control and / or torque control.

2. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 1, characterized in that, The state space S includes state variables that are collected in real time and normalized by wind turbine sensors and SCADA system. The state variables include at least: wind speed, generator speed, actual output power, pitch angle, transmission chain torque, and mechanical load of key components.

3. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 1, characterized in that, The action space A is a three-dimensional continuous action vector. A t ={Δ K p Δ K i Δ K d }, respectively, correspond to the proportionality coefficient. K p Integral coefficient K i and differential coefficients K d The adjustment increments are all limited within preset safety boundaries: Δ K p ∈[-0.1,0.1], Δ K i ∈[-0.05,0.05], Δ K d ∈[-0.02,0.02]; at the same time, the absolute value of the integral coefficient Ki does not exceed 50% of its initial set value.

4. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 1, characterized in that, The multi-objective comprehensive reward function R is composed of the following weighted sub-items: (1) Power tracking accuracy bonus R power This is used to minimize the deviation between the actual power and the target power; (2) Generator speed stability penalty R stability This is used to suppress speed fluctuations; (3) Total mechanical load penalty R load This is used to reduce fatigue loads on critical components; (4) Actuator motion smoothness penalty R smoothness This is used to reduce drastic changes in pitch angle and generator torque; (5) Coupling interaction penalty R coupling It is used to explicitly suppress dynamic coupling interference between the pitch control system and the torque control system.

5. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 4, characterized in that, The weights of each sub-item in the multi-objective comprehensive reward function are dynamically adjusted according to the current wind speed range: in the low wind speed range, the weight of power tracking accuracy is emphasized, and in the high wind speed range, the weight of mechanical load and coupling interaction penalty is emphasized.

6. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 1, characterized in that, The event-driven online fine-tuning triggering conditions include any of the following: (a) Wind speed change Δv ≥ 2 m / s; (b) Output power deviation ΔP ≥ 5% of rated power and the duration of the deviation ≥ 10 seconds.

7. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 1, characterized in that, A dynamic learning rate strategy is used during online fine-tuning: the initial learning rate is 0.001, and then it decays exponentially every 100 steps with a decay factor of 0.95 until it stabilizes at 0.0001.

8. The adaptive tuning method for a wind turbine PID controller based on reinforcement learning according to claim 1, characterized in that, It also includes an engineering safety mechanism: if five consecutive parameter adjustments cause the reward function value to decrease, the parameter update will be frozen immediately and the PID parameters will be rolled back to the previous set of parameters with stable rewards.

9. An adaptive tuning system for a PID controller of a wind turbine generator, characterized in that: The system, employing the method according to any one of claims 1–8, comprises: The data acquisition module is used to acquire real-time operating status data of the wind turbine. A state preprocessing module is used to normalize the state data; The reinforcement learning agent module, based on the DDPG architecture, includes an Actor network and a Critic network, used to output PID parameter increments and evaluate policy performance. The parameter update module is used to update the PID controller parameters based on the agent's output and safety constraints. The trigger judgment module is used to monitor whether the event triggering conditions for online fine-tuning are met; The security monitoring module is used to perform parameter freezing and rollback operations.

10. The adaptive tuning system for a wind turbine PID controller according to claim 9, characterized in that, The Actor network is a three-layer fully connected neural network, and its output is scaled to the action boundary after tanh activation; the Critic network is a two-layer fully connected network, with the input being a concatenated vector of state and action, and the output being the Q value; during training, experience replay, soft update of the target network, and gradient pruning mechanisms are used to ensure training stability.

Images