A reward adaptive wind power generator power control method

Abstract

This invention relates to a reward-based adaptive wind turbine power control method, comprising the following steps: S1, acquiring wind turbine data and effective wind speed prediction values; S2, inputting the wind turbine data and effective wind speed prediction values ​​into a trained effective wind speed estimation model to obtain effective wind speed estimates; S3, determining the operating status; S4, converting the input to a first network to obtain training experience, which is then stored in the experience pool of the first network; S5, obtaining the output and single-step reward of the first network based on the trained first network; S6, converting the control input signal of the wind power generation system to obtain the control input signal, and performing power control on the wind turbine based on the control input signal; S7, updating the parameters of the first network based on a second mini-batch training set, and executing S8 after the update is completed; S8, at the update time, returning to S1. Compared with the prior art, this invention has advantages such as stabilizing wind power generation, reducing wind turbine load and unit losses.

Description

Technical Field

[0001] This invention relates to wind turbine power control, and more particularly to a reward-adaptive wind turbine power control method. Background Technology

[0002] Currently, with the depletion of traditional fossil fuels and the increasingly serious environmental degradation they cause, new energy technologies have received high attention from the international community. Accelerating the development of renewable energy has become an essential path for countries worldwide to solve environmental and energy problems, and is also a top priority for future economic and technological development. Wind energy, as the most mature and largest-scale clean new energy source, is free, clean, and pollution-free. Wind power generation has a significant competitive advantage compared to most renewable energy generation technologies. Many regions in China have abundant wind energy resources. Developing wind power generation can provide important support for national economic development and play an increasingly important role in the national energy structure.

[0003] Improving wind power generation efficiency is crucial for wind energy development and economic benefits. However, due to the nonlinear characteristics of wind power systems, the maximum output power point of wind turbines changes with wind speed. To ensure maximum utilization of wind energy, appropriate control strategies are needed to guarantee maximum power output. Therefore, the MPPT (Maximum Power Point Tracking) control algorithm plays a decisive role in improving wind power generation efficiency. Furthermore, when the wind speed exceeds the rated wind speed, the wind turbine's power output becomes unstable and overload occurs. Therefore, when the wind speed exceeds the rated wind speed, the pitch angle of the wind turbine should be controlled to stabilize the wind power generation at maximum power and reduce equipment fatigue caused by overload.

[0004] Meanwhile, wind speed, as the driving force of wind turbines, plays a crucial role in the control design of wind power generation systems. Most existing maximum power point tracking (MPPT) control algorithms rely on accurate effective wind speed information. To achieve precise control of wind power generation systems, accurate estimation and prediction of effective wind speed are necessary to improve system performance and increase the economic benefits of wind power generation.

[0005] The essence of MPPT (Maximum Power Point Tracking) is to change the wind turbine speed through control signals to maximize the wind energy utilization coefficient, thereby achieving maximum power output. Traditional MPPT methods cannot respond promptly to changes in wind speed, have large power point tracking errors, and impose numerous restrictions on the operating conditions of wind power generation systems. However, with the rapid development of artificial intelligence technology, intelligent algorithms are increasingly being applied to many control fields. Therefore, researchers have attempted to combine traditional MPPT methods with intelligent algorithms, proposing many intelligent MPPT control methods to improve the power control effect of wind power generation systems and increase the economic benefits of wind power generation. However, most of these works primarily consider MPPT control of wind turbines below rated wind speed, without simultaneously considering constant power control at and above rated wind speeds. This leads to unstable wind power generation, high turbine load, high unit losses, and slow convergence speed of the neural network learning in intelligent MPPT control methods. Summary of the Invention

[0006] The purpose of this invention is to provide a reward-adaptive wind turbine power control method to overcome the defects of the prior art.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A reward-based adaptive wind turbine power control method includes the following steps:

[0009] S1. Obtain the wind turbine data at time t and the effective wind speed prediction value at time t. The wind turbine data includes wind turbine blade deflection, wind turbine blade torsion, wind turbine blade pitch angle, wind turbine aerodynamic power, wind turbine angular velocity, and wind turbine power generation.

[0010] S2. Input the wind turbine data at time t and the predicted effective wind speed at time t into the trained LSTM-based effective wind speed estimation model to obtain the estimated effective wind speed at time t.

[0011] S3. Based on the effective wind speed estimate and wind turbine power generation at time t, determine the operating status at time t and obtain the operating status marker value;

[0012] S4. The wind turbine aerodynamic power, wind turbine angular velocity, effective wind speed estimate and operating status mark value at time t are transformed to obtain the input of the first network. The first network is constructed based on the deep deterministic policy gradient algorithm. The input of the first network is combined with the training experience at time t-1 to obtain the training experience at time t. The training experience at time t is stored in the experience pool of the first network.

[0013] S5. Train the first network to obtain the trained first network. Based on the trained first network, obtain the output of the first network at time t and the single-step reward at time t. Store the single-step reward as a reward signal into the experience pool of the first network.

[0014] S6. Convert the output of the first network at time t into the control input signal of the wind power generation system, and perform power control on the wind turbine based on the control input signal.

[0015] S7. Determine the experience pool in the power control process, obtain the second mini-batch training set from the experience pool in the power control process, update the parameters of the first network based on the second mini-batch training set, and execute S8 after the update is completed.

[0016] S8. Update time t to time t+1 and return to S1.

[0017] Furthermore, the specific steps of S1 include:

[0018] S11. The deflection of the wind turbine blades is obtained based on the measuring instrument. The deflection of the wind turbine blades includes the blade tip flapping displacement, the blade tip edge displacement, and the blade radial displacement.

[0019] S12. Based on the measuring instruments, obtain the wind turbine blade torsion, wind turbine blade pitch angle, wind turbine aerodynamic power, wind turbine angular velocity and wind turbine power generation, and at the same time obtain the effective wind speed prediction value.

[0020] Furthermore, the specific training process for the LSTM-based effective wind speed estimation model of S2 includes:

[0021] S21. Design an effective wind speed estimation network based on LSTM;

[0022] S22. Set up a training dataset, which includes the LSTM input values ​​and LSTM target values ​​of the effective wind speed estimation network;

[0023] S23. Define the sample estimation error of the effective wind speed estimation network, and calculate the objective function that minimizes the sample estimation error.

[0024] S24. Define the update calculation formula for the effective wind speed estimation network;

[0025] S25. Based on the training dataset and the update calculation formula of the effective wind speed estimation network, train and update the effective wind speed estimation network. When the training of the training dataset is completed, if the objective function for minimizing the sample estimation error is less than the set error threshold, stop the update iteration and obtain the trained LSTM-based effective wind speed estimation model.

[0026] Furthermore, the specific steps of S4 are as follows:

[0027] S41. Obtain the wind turbine aerodynamic power, wind turbine angular velocity, effective wind speed estimate, and operating status marker value at time t;

[0028] S42. Obtain the reference value of the wind turbine rotation speed at time t based on the acquired data;

[0029] S43, Based on the data obtained in S41 and the wind turbine rotation speed reference value in S42, the input of the first network at time t and the input of the external reward calculation network are obtained;

[0030] S44. Combining the training experience at time t-1, the input of the first network at time t, the output of the first network at time t-1, the reward signal at time t-1, and the input of the first network at time t-1 are stored in the experience pool of the first network as the training experience at time t.

[0031] Furthermore, the specific steps of S5 are as follows:

[0032] S51. Design a first network based on the deep deterministic policy gradient algorithm. The first network includes an action neural network, an evaluation neural network, a first target network corresponding to the action neural network, and a second target network corresponding to the evaluation neural network.

[0033] S52. The network designed in S51 is pre-trained by simulating turbulent wind speed environmental sequences. The specific steps of the pre-training are as follows:

[0034] Obtain the input of the first network with index k in the sequence, use this input as the input vector of the action neural network, obtain the output action, and use the input vector and output action of the action neural network as the input vector of the evaluation neural network.

[0035] Obtain the input of the first network with the next sequence number, which is k+1. Use this input as the input vector of the first target network to obtain the predicted output action. Use the input vector of the first target network and the predicted output action as the input vector of the second target network to complete the pre-training.

[0036] S53. Construct an external reward calculation model and an internal reward calculation model. The external reward calculation model includes a reward action network and a reward evaluation network, and calculates the external reward.

[0037] Based on the internal reward calculation model, feature extraction is performed on the input with index k and the input with index k+1 in S52 to obtain the state features corresponding to the two inputs respectively. A feedforward network is set up. The feedforward network predicts the first feature of the input with index k+1 based on the output action of S52 and the state features of the input with index k. At the same time, a feature network is learned. The feature network extracts features related to the input with index k and the output action in S52.

[0038] S54. Based on the time difference algorithm, design the training process of the external reward calculation model, and set the prediction error and minimization function of the reward evaluation network;

[0039] S55. Set the weight update rule for the reward evaluation network based on the minimization function, and iteratively update the weights of the reward evaluation network based on the weight update rule. When the number of iterations reaches the preset upper limit of the reward evaluation network, or the prediction error of the reward evaluation network is less than the preset first error threshold, stop the iteration, complete the training of the reward evaluation network, and input the output of the reward evaluation network at this time into the reward action network.

[0040] S56. Based on the output of the reward evaluation network in S55, set the prediction error of the reward action network, set the weight update rule of the reward action network, and iteratively update the weight of the reward action network based on the weight update rule. When the number of iterations reaches the preset update upper limit of the reward action network, or the prediction error of the reward action network is less than the preset second error threshold, stop the iteration. At this time, the output of the reward action network is the updated external reward.

[0041] S57. Calculate the prediction error of the feedforward network and the prediction error of the feature network in the internal reward calculation model.

[0042] S58. Set the weight update rules for the feedforward network and the feature network. Iterate and update the feedforward network and the feature network respectively based on the weight update rules of the two networks. When the number of iterations reaches the preset update upper limit of the feedforward network and the feature network, or when the prediction error of the feedforward network is less than the preset third error threshold and the prediction error of the feature network is less than the preset fourth error threshold, stop the iterative update. Obtain the internal reward based on the output of the feedforward network at this time. Obtain the total single-step reward based on the updated external reward and internal reward. Store the single-step reward as a reward signal in the experience pool of the first network.

[0043] S59. Obtain the first mini-batch training set from the experience pool, set the parameter update rules for the action neural network, and then update the first network using the update calculation formula in S24 based on the stochastic optimization algorithm to obtain the first network after training, and obtain the output of the first network at time t.

[0044] Furthermore, the stochastic optimization algorithm is specifically a backpropagation algorithm optimized by a stochastic optimization algorithm with adaptive momentum.

[0045] Furthermore, the specific steps of S7 are as follows:

[0046] S71. Determine the experience pool in the power control process;

[0047] S72. Obtain a second mini-batch training set from the experience pool in the power control process. The second mini-batch training set consists of experience samples from the experience pool. According to the training method in S54, update the parameters of the first network online based on the second mini-batch training set.

[0048] Furthermore, the expression for the running state of S3 is as follows:

[0049]

[0050] Here, f1 and f2 are two different operating states, P g For wind turbine power generation, v e For the effective wind speed estimate, P g,rate For rated power generation, v rate This is the rated wind speed.

[0051] Furthermore, the expression for external rewards is:

[0052]

[0053] Where, r e (k) represents the external reward, and dist(k) is the distance between the input vector of the reward action network and the radial basis function, Φ a For radial basis functions, N a,h To increase the number of hidden layer neurons in the reward action network, W a,i (k) represents the weights from the i-th hidden neuron in the reward action network to the output layer, where i is the sequence number of the hidden neuron.

[0054] Furthermore, the reward evaluation network is a three-layer BP neural network, comprising an input layer, an output layer, and a hidden layer.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) Based on the system status data and the designed operating status mark value, the present invention adjusts the electromagnetic torque of the wind turbine generator and the pitch angle of the wind turbine so that the wind turbine speed runs at the optimal value. Compared with the existing MPPT control below the rated wind speed, the present invention optimizes the power extraction efficiency, stabilizes the wind power generation power, and reduces the wind turbine load and unit loss.

[0057] (2) The present invention calculates the single-step reward and uses it as a reward signal for learning, which can accelerate the learning convergence speed of the neural network.

[0058] (3) The present invention designs an effective wind speed estimation model based on LSTM, which can use the stored long-term effective wind speed memory and the current short-term wind turbine information to estimate the current effective wind speed and predict the effective wind speed at the next moment. Compared with the wind speed acquisition in the prior art, the effective wind speed estimation can make the power control results more accurate, robust and anti-interference stronger. Attached Figure Description

[0059] Figure 1 This is a flowchart of the present invention;

[0060] Figure 2 This is a schematic diagram of the reward-adaptive wind turbine power control system of the present invention;

[0061] Figure 3 This is a graph of the wind energy utilization coefficient function of the present invention;

[0062] Figure 4 This is a schematic diagram of the effective wind speed estimation model for wind turbines based on LSTM according to the present invention.

[0063] Figure 5 This is a schematic diagram of the first network structure and parameter adjustment of the reward-adaptive wind turbine power control method of the present invention.

[0064] Figure 6 This is a schematic diagram of the action network and target network of the reward-adaptive wind turbine power control method of the present invention.

[0065] Figure 7 This is a schematic diagram of the evaluation network and its target network for the reward-adaptive wind turbine power control method of the present invention;

[0066] Figure 8 This is a schematic diagram of the structure and parameter adjustment of the external reward calculation network of the present invention;

[0067] Figure 9 This is a schematic diagram of the internal reward calculation model of the present invention;

[0068] In the figure, wind speed acquisition system 1, wind turbine information acquisition module 2, wind turbine effective wind speed estimation module 3, wind turbine operating status judgment module 4, input data processing module 5, wind turbine power control module 6, output data processing module 7, and control output calculation submodule 61. Detailed Implementation

[0069] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0070] Example 1:

[0071] This invention provides a reward-adaptive wind turbine power control method, the flowchart of which is shown below. Figure 1 As shown. The method of the present invention includes the following steps:

[0072] S1. Obtain the wind turbine data at time t and the effective wind speed prediction value at time t. The wind turbine data includes wind turbine blade deflection, wind turbine blade torsion, wind turbine blade pitch angle, wind turbine aerodynamic power, wind turbine angular velocity, and wind turbine power generation.

[0073] S2. Input the wind turbine data at time t and the predicted effective wind speed at time t into the trained LSTM-based effective wind speed estimation model to obtain the estimated effective wind speed at time t.

[0074] S3. Based on the effective wind speed estimate and wind turbine power generation at time t, determine the operating status at time t and obtain the operating status marker value.

[0075] S4. The wind turbine aerodynamic power, wind turbine angular velocity, effective wind speed estimate and operating status label value at time t are transformed to obtain the input of the first network. The first network is constructed based on the deep deterministic policy gradient algorithm. The input of the first network is combined with the training experience at time t-1 to obtain the training experience at time t. The training experience at time t is stored in the experience pool of the first network.

[0076] S5. Train the first network to obtain the trained first network. Based on the trained first network, obtain the output of the first network at time t and the single-step reward at time t. Store the single-step reward as a reward signal into the experience pool of the first network.

[0077] S6. Convert the output of the first network at time t into the control input signal of the wind power generation system, and perform power control on the wind turbine based on the control input signal.

[0078] S7. Determine the experience pool in the power control process, obtain the second mini-batch training set from the experience pool in the power control process, update the parameters of the first network based on the second mini-batch training set, and execute S8 after the update is completed.

[0079] S8. Update time t to time t+1 and return to S1.

[0080] The principle and specific steps of S1 are as follows:

[0081] In S1, the wind turbine data at time t includes the following information: wind turbine blade deflection u(t), wind turbine blade torsion γ(t), wind turbine blade pitch angle β(t), and wind turbine aerodynamic power P. a (t), wind turbine angular velocity ω(t) and wind turbine power generation Pg (t); where t represents the sampling time. S1 also collects wind speed data of the wind field based on the wind speed acquisition system, as the real-time inflow wind speed value v(t) of the wind field.

[0082] The specific steps of S1 are as follows:

[0083] S11. Obtain the wind turbine blade deflection u(t) based on the measuring instrument, wherein the wind turbine blade deflection u(t) includes the blade tip flapping displacement u x (t), blade tip edge displacement u y (t), blade radial displacement u z (t), that is, u(t) = {u x (t), u y (t), u z (t)}; where the subscripts x, y, z represent the deflection of the blade in different directions by spatial coordinates.

[0084] S12. Obtain the wind turbine blade torsion γ(t), wind turbine blade pitch angle β(t), and wind turbine aerodynamic power P based on measuring instruments. a (t), wind turbine angular velocity ω(t) and wind turbine power generation P g (t); where P a The overall term represents aerodynamic power; the subscript 'a' indicates that the wind turbine's aerodynamic power is based on aerodynamic calculations. (P) g (t) represents the total power generation, with the subscript g indicating generator. Simultaneously, the effective wind speed forecast value v is obtained. p (t-10).

[0085] The wind speed acquisition system also collects real-time wind speed data from the wind field using an anemometer, and takes the average of multiple measurements as the inflow wind speed v(t), where t represents the sampling time.

[0086] The principle and specific steps of S2 are as follows:

[0087] In S2, the wind turbine data at time t includes the rotor blade deflection u(t), rotor blade torsion γ(t), rotor blade pitch angle β(t), and rotor aerodynamic power P. a (t), wind turbine angular velocity ω(t), and effective wind speed prediction value v at time t. p (t-1), the input is the trained LSTM-based effective wind speed estimation model (denoted as V-Net), and the input is denoted as x. v The effective wind speed estimate v at time t is obtained using the trained model. e (t) and the predicted effective wind speed v at the next moment p (t). The structure diagram of the effective wind speed estimation model is shown below. Figure 4 As shown.

[0088] The specific training process of the S2 LSTM-based effective wind speed estimation model includes:

[0089] S21. Design an effective wind speed estimation network based on LSTM;

[0090] S22. Setting up the training dataset (N v D v The number of samples, k = 1, 2, 3, ..., N v The training dataset includes the LSTM input and LSTM target values ​​of the effective wind speed estimation network.

[0091] Dataset The samples in the data are arranged according to the time series of wind turbine operation, k is the sample number, and the subscript v in the variables indicates that the corresponding variable is related to V-Net.

[0092] With x v ={u(k),γ(k),β(k),P a (k),ω(k),v p (k-1)} is used as the input value of the LSTM, y v (k) is the target value of LSTM, that is, the effective wind speed reference value for the current sample and the next sample.

[0093] The output value is obtained after calculation by the effective wind speed estimation network. Among them, v e (k) represents the effective wind speed estimate for sample k, v p (k) represents the effective wind speed prediction for the k+1 sample. The subscripts e and p represent estimation and prediction, respectively, to distinguish different wind speed variables.

[0094] S23. Define the sample estimation error of the effective wind speed estimation network. And calculate the objective function that minimizes it based on the sample estimation error.

[0095] S24. Define the update calculation formula for the effective wind speed estimation network.

[0096] The updated calculation formula is as follows:

[0097] m(k)=l1m(k-1)+(1l1)g(k), v a (k)=l2v a (k-1)+(1l2)g 2 (k), Where, based on experience, the parameters η = 0.001, l1 = 0.9, l2 = 0.999, ∈ = 1e-8; g(k) represents the gradient of the loss function of the k-th sample with respect to the parameter Θ, where Θ represents the parameter set of V-Net; m(k) is the expectation of the gradient g(k), v a (k) is g 2 The expectation of (k), It is the bias correction of m(k). It is v a Bias correction of (k); where, v a The whole represents g 2 The expectation of (k) is given by the superscript a, which is only used for differentiation.

[0098] The above update calculation formula uses a stochastic optimization algorithm, specifically a backpropagation algorithm optimized by the Adam stochastic optimization algorithm with adaptive momentum. The Adam algorithm uses momentum to suppress oscillations and makes the learning rate adaptive over time, thereby accelerating the convergence speed of the neural network.

[0099] S25. Based on the training dataset and the update calculation formula of the effective wind speed estimation network, train and update the effective wind speed estimation network. When the training of the training dataset is completed, if the objective function E minimizes the sample estimation error... v (k) is less than the set error threshold ε v If the update iteration stops, the trained LSTM-based effective wind speed estimation model is obtained.

[0100] In S3, the effective wind speed estimate v at time t is... e (t) and wind turbine power generation P g (t) Determine the running state at time t and obtain the running state flag value.

[0101] The expression for the running state of S3 is as follows:

[0102]

[0103] Here, f = {f1, f2} marks different operating states, and these states determine different control regions, enabling the control algorithm to adaptively learn and achieve control objectives under various operating conditions. Where P... g,rate For rated power generation, v rate This is the rated wind speed.

[0104] In S4, the wind turbine aerodynamic power P at time t is... a (t), wind turbine angular velocity ω(t), effective wind speed estimate v e (t) and the running status flag value f are transformed to obtain the input X of the first network. D(t), the first network is constructed based on the Deep Deterministic Policy Gradient Algorithm (DDGP algorithm), and the input X of the first network is... D The training experience at time t is obtained by combining the training experience at time t(t) and time t-1, and then stored in the experience pool of the first network. The first network is denoted as D-Net. The structure and parameter updates of the first network are as follows: Figure 5 As shown.

[0105] The specific steps of S4 are as follows:

[0106] S41. Obtain the wind turbine aerodynamic power P at time t. a (t), wind turbine angular velocity ω(t), effective wind speed estimate v e (t) and the running status flag value f.

[0107] S42. Obtain the reference value ω of the wind turbine rotation speed at time t based on the acquired data. opt (t).

[0108] S43, Based on the data obtained in S41 and the wind turbine speed reference value ω in S42 opt (t) Obtain the input X of the first network at time t. D (t) and the input x of the external reward calculation network r (t). The external reward calculation network is denoted as RE-Net. The expression is:

[0109] X D (t)={Δv e (t), ΔP g (t), ω e (t), f(t)

[0110] x r (t)={|Δv e (t)|,|ΔP g (t)|,|ω e (t)|,f(t)}

[0111] Where, Δv e (t)=v e (t)-v e (t-1), ΔP g =P g (t)-P g (t-1), ω e =ω r (t)-ω opt (t). ω opt (t) represents the reference value of the wind turbine speed at time t.

[0112] S44. Combining the training experience at time t-1, the input of the first network at time t, the output of the first network at time t-1, the reward signal at time t-1, and the input of the first network at time t-1 are stored in the experience pool of the first network as the training experience at time t.

[0113] The expression for the training experience at time t is: {X D (t-1), Y D (t-1), r(t-1), X D (t)}.

[0114] In S5, the first network D-Net is trained using the experience pool to obtain the trained first network. Based on the trained first network, the output Y of the first network at time t is obtained. D (t)={ΔT e β(t)} and the single-step reward r(t) at time t are stored as reward signals in the experience pool of the first network. Where ΔT e (t) represents the generator torque regulation at time t.

[0115] The specific steps of S5 include:

[0116] S51. Design a first network based on the deep deterministic policy gradient algorithm. The first network includes an action neural network, an evaluation neural network, a first target network corresponding to the action neural network, and a second target network corresponding to the evaluation neural network.

[0117] D-Net is designed based on the DDPG algorithm. The DDPG algorithm is a deep reinforcement algorithm proposed to solve continuous action control problems. It adopts the Actor-Critic (AC) architecture, including two parts: an action neural network (AN) and an evaluation neural network (CN). It also uses a dual-network architecture, meaning each part has its corresponding first target network and second target network (AN′, CN′), thus D-Net consists of AN, CN, AN′, and CN′. Furthermore, an experience replay mechanism is introduced, establishing an experience pool to break sequence correlation and reuse past experience, improving training stability and accelerating network convergence. In this invention, AN′ is the first target network, and CN′ is the second target network.

[0118] AN′ is a copy of AN, and their network structures are identical, such as... Figure 6 As shown, observation represents the input layer of the AN network, i.e., the D-Net 61 input vector X. D (t); Afc1 and Afc2 are fully connected hidden layers, and Arelu and Atanh are ReLU and Tanh function activation layers, respectively. The ReLU function is defined as: frelu (x) = max(0, x), the Tanh function is defined as: AN computational network output Y D The formula for (t) is as follows: O Afc1 (t)=W Afc1 (t)·X D (t), O Arelu (t)=f relu (O Afc1 (t)), O Afc2 (t)=W Afc2 (t)·O Arelu (t), Y D (t)=tanh(O Afc2 (t)), where W Afc1 W Afc2 These are the weight parameters for hidden layers Afc1 and Afc2, respectively. Afc1 (t), O Arelu (t), O Afc2 (t) are the output vectors of the hidden layers of Afc1, Arelu and Afc2, respectively, and the subscripts indicate the corresponding network layers;

[0119] Similarly, CN′ is a replication network of CN, and its structure is as follows: Figure 7 As shown, action represents the output Y of the AN network. D (t), Cfc1, Cfc2, and Cfc3 are fully connected hidden layers, and Crelu1 and Crelu2 are ReLU function activation layers, defined the same as Arelu; CN evaluates the network by calculating Q. μ The formula for (t) is as follows: O Cfc1 (t)=W Cfc1 (t)·X D (t), O Crelu1 (t)=f relu (O Cfc1 (t)), O Cfc3 (t)=W Cfc3 (t)·Y D (t), I add (t)=[I Crelu1 (t), O Cfc3 (t)],O Crelu2 (t)=f relu (O add (t)), Q μ (t)=W Cfc2 (t)·O Crelu2 (t), where W Cfc1 W Cfc2 and W Cfc3 These are the weight parameters for hidden layers Cfc1, Cfc2, and Cfc3, respectively.Cfc1 (t), O Crelu1 (t), O Cfc3 (t), O add and O Crelu2 These are the output vectors of the Cfc1, Crelu1, Cfc3, add, and Crelu2 layers, respectively, with the subscript indicating the corresponding network layer.

[0120] S52. The network designed in S51 is pre-trained by simulating a turbulent wind speed environment sequence. The specific steps of pre-training are as follows: obtain the input of the first network with sequence number k in the sequence, use this input as the input vector of the action neural network to obtain the output action, use the input vector and output action of the action neural network as the input vector of the evaluation neural network, obtain the input of the first network with sequence number k+1, use this input as the input vector of the first target network to obtain the predicted output action, use the input vector of the first target network and the predicted output action as the input vector of the second target network to complete the pre-training.

[0121] The expression for the simulated turbulent wind speed environment sequence is: (N D The number of wind speed sampling points, k = 1, 2, 3, ..., N D Pre-train the D-Net using the system state rating as the initial training value; V D Arranged according to the time series of wind turbine operation, k is the wind speed sampling sample number, and the subscript D in the variable indicates that the corresponding variable is related to D-Net, the same below.

[0122] Obtain the input X of the first network in the sequence. D (k), X D (k) is used as the input vector of AN in D-Net to obtain the output action Y. D (k)=μ(X D (k)|θ μ )={ΔT e θ(k), β(k), μ is the action function, θ μ These are the network parameters of AN; where ΔT e β(k) represents the generator torque regulation of sample k, and β(k) represents the pitch angle of sample k.

[0123] {X D (k), Y D (k)} is used as the input vector of CN to obtain the output action Y. D The value function of (k) is: θ Q These are the network parameters of CN, i.e., θ Q ={W Cfc1 WCfc2 W Cfc3}

[0124] Then take the input X of the first network of the next sequence number D (k+1) is used as the input vector of AN′ to predict the output action Y′ of k+1. D (k+1)=μ'(X D (k+1)|θ' μ μ' is used to fit the action function μ, θ' μ These are the network parameters of AN′, i.e., θ' μ ={W' Afc1 ,W' Afc2}; then {X D (k+1),Y′ D (k+1)} is the input vector of CN′, Q′ μ For Q μ The fitted value, θ' Q The network parameters for CN′ are θ. Q ={W' Cfc1 ,W' Cfc2 ,W' Cfc3}

[0125] S53. Construct an external reward calculation model and an internal reward calculation model. The external reward calculation model includes a reward action network and a reward evaluation network, and calculates the external reward.

[0126] Based on the internal reward calculation model, feature extraction is performed on the input with index k and the input with index k+1 in S52 to obtain the state features corresponding to the two inputs respectively. A feedforward network is set up. Based on the output action of S52 and the state features of the input with index k, the feedforward network predicts the first feature of the input with index k+1. At the same time, a feature network is learned. The feature network extracts features related to the input with index k and the output action in S52.

[0127] In S53, the external reward calculation model is denoted as RE-Net, built on the AC architecture, while the internal reward calculation model is denoted as RI-Net, built on the internal curiosity reward mechanism. The AC architecture of RE-Net consists of an Action network (denoted as RE_AN, i.e., the reward action network) based on RBF (Radial Basis Functions Neural Network) and a Critic network (denoted as RE_CN, i.e., the reward evaluation network) based on BP (Backpropagation Neural Network). Its specific structure is as follows... Figure 8 As shown.

[0128] The input x of the network is calculated using the external reward in S4. r (k)={|Δv e (k)|,ΔPg (k)|,|ω e The input vectors of RE-AN are (k)|,f(k)}, and the external reward r is calculated by RE-AN. e (k); then, with {x r (k),r e (k)} is used as the input vector of RE_CN, and the output r of RE_AN is obtained by RE_CN. e The cumulative return forecast J of (k) c (k); Let r c The enhanced signal for RE-Net is represented by "0" and "-1" to indicate "success" and "failure" respectively, based on the wind power generation control results.

[0129] RI-Net uses the function φ on X D (k) and X D (k+1) is used for feature extraction to obtain the state feature representation φ(X). D (k)) and φ(X) D (k+1)); based on action Y through the feedforward network F. D (k) and φ(X) D (k) Predict X D (k+1) feature representation Simultaneously, a feature network G is learned to extract X. D Zhong and Y D Based on relevant features, obtain the action prediction value P. k This filters out irrelevant noise in the environment; its specific structure is as follows: Figure 9 As shown.

[0130] The φ function in RI-Net is defined as It is a full-time matrix; the feedforward network F and the feature network G are also composed of three-layer BP neural networks, and their calculation formulas are as follows: These are the weights from the input layer nodes to the hidden layer nodes of the k-sample F network. These are the weights from hidden layer nodes to output layer nodes in the k-sample F network. This is the output of the hidden layer node in the F network; similarly, These are the weights from the input layer nodes to the hidden layer nodes of the k-sample G network. These are the weights from hidden layer nodes to output layer nodes in the G network for k samples. This is the output of the hidden layer node in the G network.

[0131] External reward r e (k) is calculated using the following formula: Where dist(k) is the distance between the input vector of RE_AN and the radial basis functions. N a,h c is the number of hidden layer neurons in RE_AN. a,i (k) represents the radial basis function center of the i-th hidden neuron in RE_AN, σ a,i (k) is the radial basis function width of the i-th hidden neuron in RE_AN, W a,i (k) represents the weights from the i-th hidden neuron in RE_AN to the output layer; RE_CN is a three-layer BP neural network, consisting of an input layer, an output layer, and a hidden layer. c (k) is calculated using the following formula:

[0132]

[0133]

[0134]

[0135] in It is the weights from the i-th input layer node to the j-th hidden layer node in RE_CN for k samples. q is the weight from the i-th hidden layer node to the output layer node in sample RE_CN; i (k) is the input of the i-th hidden layer node of RE_CN; p i (k) is the output of the i-th hidden layer node of RE_CN; N c,h n is the total number of hidden layer nodes in RE_CN; n+1 is the total number of inputs to RE_CN including the output r of RE_AN. e (k), in an embodiment of the present invention, n is 5; x j (k)=[x r (k),r e (k)}.

[0136] S54. Based on the time difference algorithm, design the training process of the external reward calculation model, and set the prediction error and minimization function of the reward evaluation network.

[0137] The prediction error of RE_CN is defined as e. c (m)=αJ c (m)-[J c (m-1)-r c [(k)], where α is the discount factor; the objective function to be minimized for RE_CN is defined as: m represents the iteration number; J(m) represents the value of {x} after the m-th iteration. r (k),r i(k)} is the input of RE_CN and the output of RE_CN.

[0138] S55. Set the weight update rule for the reward evaluation network based on the minimization function, and iteratively update the weights of the reward evaluation network based on the weight update rule. When the number of iterations reaches the preset upper limit of the reward evaluation network, or the prediction error of the reward evaluation network is less than the preset first error threshold, stop the iteration, complete the training of the reward evaluation network, and input the output of the reward evaluation network at this time into the reward action network.

[0139] The RE_CN weight update rule is: w c (m+1)=w c (m)+Δw c (m), the RE_CN weights are iteratively updated according to the RE_CN weight update rule; where w c (m) is the result of the RE_CN weights in the m-th iteration, Δw c (m) is the change in the weight of RE_CN during the m-th iteration. l c This is the RE_CN learning step size; when the number of iterations m reaches the set upper limit of RE_CN update, or the prediction error e of RE_CN... c (m) is less than the set first error threshold ε c Stop the iteration; RE_CN outputs J(m) as J(k) to RE_AN.

[0140] S56. Based on the output of the reward evaluation network in S55, set the prediction error of the reward action network, set the weight update rule of the reward action network, and iteratively update the weights of the reward action network based on the weight update rule. When the number of iterations reaches the preset update upper limit of the reward action network, or the prediction error of the reward action network is less than the preset second error threshold, stop the iteration. At this time, the output of the reward action network is the updated external reward.

[0141] Let the prediction error of RE_AN be e. a (m)=J(k)-U c (m), where U c (m) represents the final expected value of RE_AN, which takes the value 0; the objective function of RE_AN is set as: Set the RE_AN weight update rule as follows: c a (m+1)=c a (m)-Δc a (m), σ a (m+1)=σ a (m)-Δσ a (m), w a (m+1)=wa (m)-Δw a (m), iteratively update the RE_AN weights according to the RE_AN weight update rule; where, c a (m) and σ a (m) represent the radial basis functions Φ of the RE_AN hidden neurons at the m-th iteration. a The center and width of (·), Δc a (m) and Δσ a (m) represents the changes in center and width during the m-th iteration; w a (m) is the weight matrix of the fully connected layer from the hidden layer to the output layer of RE_AN at the m-th iteration, Δw a (m) represents the change in the weight matrix; the specific update formula is: Among them, l a It is the learning step size of RE_AN, and (·) represents Φ. a It is a function variable; when the number of iterations m reaches the set upper limit of RE_AN update, or the prediction error e of RE_AN... a (m) is less than the set second error threshold ε a Stop iteration; set x r (k) is used as input to RE_AN, and the updated external reward r is output through RE_AN. e (k).

[0142] S57. Calculate the prediction error of the feedforward network and the prediction error of the feature network in the internal reward calculation model.

[0143] The prediction error of the RI-Net feedforward network F is set as follows: The objective function to be minimized is The prediction error of the feature network G is e G (m)=P k (m)-Y D (k), whose objective function to be minimized is Where m represents the number of iterations. After the m-th iteration, {φ(X)} will still be used. D (k)),Y D (k)} is the input to F, and the result is output by F; P k (m) represents the value after the m-th iteration, where {φ(X)} will still be used. D (k)),φ(X D (k+1))} is the input of G, and the result is the output of G.

[0144] S58. Set the weight update rules for the feedforward network and the feature network. Iterate and update the feedforward network and the feature network respectively based on the weight update rules of the two networks. When the number of iterations reaches the preset update upper limit of the feedforward network and the feature network, or when the prediction error of the feedforward network is less than the preset third error threshold and the prediction error of the feature network is less than the preset fourth error threshold, stop the iterative update. Obtain the internal reward based on the output of the feedforward network at this time. Obtain the total single-step reward based on the external reward and the internal reward. Store the single-step reward as a reward signal in the experience pool of the first network.

[0145] The weight update rules for F and G are set as follows: w F (m+1)=w F (m)+Δw F (m), w G (m+1)=w G (m)+Δw G (m); Iteratively update the network weights of F and G according to the weight update rule; where w F (m), w G (m) represent the weights of F and G in the m-th iteration, Δw F (m), Δw G (m) represents the change in the weights of F and G during the m-th iteration. l F l G These are the learning step sizes of the feedforward network F and the feature network G, respectively; when the number of iterations m reaches the set update upper limit values ​​of F and G, or the prediction error e of F... F (m) is less than the set error threshold ε F The prediction error of G is e G (m) is less than the set error threshold ε G Stop iteration; F will Output, based on Calculate the internal reward, η r >0 is the scaling factor; the total reward is r(k) = r e (k)+r i (k); experience {X D (k),Y D (k),r(k),X D (k+1)} is stored in the training experience pool.

[0146] The weight update rules of RE-Net and RI-Net are derived from the backpropagation algorithm. Backpropagation is a learning algorithm suitable for multi-layer neural networks. It mainly consists of two iterative steps (incentive propagation and weight update), calculating the partial derivatives of the objective function with respect to the weights of each neuron layer by layer. This forms the gradient of the objective function with respect to the weight vector, which serves as the basis for modifying the weights until the network's response to the input reaches the predetermined target range. The theorem is as follows: If functions u = φ(x) and v = ψ(x) are differentiable at point x, and function z = f(u,v) has continuous partial derivatives at the corresponding point (u,v), then the function z = f[φ(x), ψ(x)] is differentiable at the corresponding point x, and its derivative can be calculated using the following formula:

[0147] S59. Obtain the first mini-batch training set from the experience pool, set the parameter update rules for the action neural network, and then update the first network using the update calculation formula in S24 based on the stochastic optimization algorithm to obtain the first network after training, and obtain the output of the first network at time t.

[0148] N is selected from the training experience pool. Q We obtain the first mini-batch training set from a set of empirical samples:

[0149]

[0150] According to step S52, the parameter update rule for CN is set, and the loss function to be minimized is:

[0151]

[0152] Where y i =r(i)+γ Q Q′ μ (X D (i+1),μ'(X D (i+1)|θ μ ')|θ' Q ), γ Q Let CN be the discount factor; based on the parameter update rule of CN, set the action (policy) gradient of AN as follows:

[0153] in, Let θ' represent the expected cumulative reward for the action, and the parameter updates aim to maximize J(μ). The target networks AN′ and CN′ are updated using a moving average method, with θ' set as follows: Q =τθ Q +(1τ)θ' Q ,θ' μ =τθ μ+(1τ)θ' μ τ is the training factor.

[0154] The parameters of AN, CN, AN′, and CN′ are updated using a stochastic optimization algorithm, i.e., updated according to the formula in step S24, where Θ={θ μ ,θ Q ,θ' μ ,θ' Q}

[0155] In S6, the control input signal for the wind power generation system is {T} e (t),β9t)}。 Wherein, generator torque T e 9t)=T e (t-1)+ΔT e (t), β(t) = β(t).

[0156] In S7, the experience pool for the power control process is determined, and the second mini-batch training set is obtained from the experience pool for the power control process:

[0157] {X D (i),Y D (i),r(i),X D (i+1)}(i=1,2,…,N Q )

[0158] The parameters of the first network are updated online based on the second mini-batch training set. After the update is complete, S8 is executed.

[0159] The specific steps of S7 include:

[0160] S71. Determine the experience pool in the power control process. This experience pool is formed during the control of the wind power generation system by… T represents the operating time of the wind power generation system.

[0161] S72. Select N from the experience pool in the power control process. Q 1 empirical sample, to obtain the second mini-batch training set According to the training method of S54, the parameters of the first network are updated online based on the second mini-batch training set.

[0162] The method of this invention can be implemented using a reward-adaptive wind turbine power control system. A schematic diagram of the system is shown below. Figure 2 As shown. The system includes a wind speed acquisition system, a wind turbine information acquisition module, a wind turbine effective wind speed estimation module, a wind turbine operating status judgment module, an input data processing module, a wind turbine power control module, and an output data processing module, as detailed below:

[0163] Wind speed acquisition system 1 collects wind speed values ​​from the wind field;

[0164] Wind turbine information acquisition module 2 is connected to the wind turbine generator and is used to collect wind turbine blade deflection, wind turbine blade torsion, wind turbine blade pitch angle, wind turbine aerodynamic power, wind turbine angular velocity and wind turbine power generation; among which, wind turbine blade deflection includes blade tip flapping displacement, blade tip edge displacement and blade radial displacement.

[0165] The effective wind speed estimation module 3 is a deep learning module, which consists of an effective wind speed estimation model based on LSTM (Long Short-Term Memory Network) that has been trained. Its signal is connected to the wind turbine information acquisition module. First, it estimates the current effective wind speed of the wind turbine and predicts the effective wind speed of the next moment based on the acquired data of wind turbine tip deflection, wind turbine blade torsion, wind turbine blade pitch angle, wind turbine angular velocity, wind speed, wind turbine aerodynamic power and the effective wind speed value of the previous moment.

[0166] The wind turbine operation status judgment module 4 is connected to the wind turbine information acquisition module and the wind turbine effective wind speed estimation module. It judges the operation status of the wind power generation system based on the estimated effective wind speed and the acquired wind turbine power generation, and outputs the operation status mark value.

[0167] Input data processing module 5 is connected to the fan information acquisition module, the fan effective wind speed estimation module and the fan operating status judgment module. It processes the acquired fan information, the estimated effective wind speed and the operating status marker value to obtain the designed input vector.

[0168] The wind turbine power control module 6 is a deep reinforcement learning module, which is a trained DDPG-based reward adaptive model. It includes a control output calculation submodule and a reward signal adaptive calculation submodule based on a dual Actor-Critic (AC) structure. The reward signal is used to update the parameters of the control output calculation submodule. The wind turbine power control module is connected to the input data processing module. After data processing, the input vectors of each submodule are obtained, and the control output and single-step reward signal are calculated.

[0169] The output data processing module 7 is connected to the wind turbine power control module and converts the output control vector into the control input signal of the wind power generation system.

[0170] When using the above system, the working principle of the module is as follows: The effective wind speed estimation module 3 uses the system status information collected by the wind turbine information acquisition module 2, the wind speed of the wind field collected by the wind speed acquisition system 1, and the effective wind speed value of the previous moment to estimate the current effective wind speed of the wind turbine and predict the effective wind speed of the next moment; secondly, the wind turbine operating status mark value is obtained through the wind turbine operating status judgment module 4; then, the system status information, the effective wind speed estimate value, and the wind turbine operating status mark value are passed through the input data processing module 5 to obtain the input vector of the wind turbine power control module 6; then, the wind turbine power control module 6 calculates the system control output and updates the network parameters of the control output calculation submodule (D-Net) 61 online through the experience pool; finally, the control output is passed through the output data processing module 7 to obtain the system control signal for power control of the wind turbine generator.

[0171] Figure 3 The graph shows the function curve of the wind energy utilization coefficient. The dashed line represents the mapping relationship between the tip speed ratio and the wind energy utilization coefficient when the pitch angle is 0, the dotted line represents the mapping relationship when the pitch angle is 5, the single-dot dashed line represents the mapping relationship when the pitch angle is 15, and the double-dot dashed line represents the mapping relationship when the pitch angle is 20.

[0172] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A reward-adaptive wind turbine power control method, characterized in that, Includes the following steps: S1. Obtain the wind turbine data at time t and the effective wind speed prediction value at time t. The wind turbine data includes wind turbine blade deflection, wind turbine blade torsion, wind turbine blade pitch angle, wind turbine aerodynamic power, wind turbine angular velocity, and wind turbine power generation. S2. Input the wind turbine data at time t and the predicted effective wind speed at time t into the trained LSTM-based effective wind speed estimation model to obtain the estimated effective wind speed at time t. S3. Based on the effective wind speed estimate and wind turbine power generation at time t, determine the operating status at time t and obtain the operating status marker value; S4. The wind turbine aerodynamic power, wind turbine angular velocity, effective wind speed estimate and operating status mark value at time t are transformed to obtain the input of the first network. The first network is constructed based on the deep deterministic policy gradient algorithm. The input of the first network is combined with the training experience at time t-1 to obtain the training experience at time t. The training experience at time t is stored in the experience pool of the first network. S5. Train the first network to obtain the trained first network. Based on the trained first network, obtain the output of the first network at time t and the single-step reward at time t. Store the single-step reward as a reward signal into the experience pool of the first network. S6. Convert the output of the first network at time t into the control input signal of the wind power generation system, and perform power control on the wind turbine based on the control input signal. S7. Determine the experience pool in the power control process, obtain the second mini-batch training set from the experience pool in the power control process, update the parameters of the first network based on the second mini-batch training set, and execute S8 after the update is completed. S8. Update time t to time t+1 and return to S1.

2. The reward-adaptive wind turbine power control method according to claim 1, characterized in that, The specific steps of S1 include: S11. The deflection of the wind turbine blades is obtained based on the measuring instrument. The deflection of the wind turbine blades includes the blade tip flapping displacement, the blade tip edge displacement, and the blade radial displacement. S12. Based on the measuring instruments, obtain the wind turbine blade torsion, wind turbine blade pitch angle, wind turbine aerodynamic power, wind turbine angular velocity and wind turbine power generation, and at the same time obtain the effective wind speed prediction value.

3. The reward-adaptive wind turbine power control method according to claim 1, characterized in that, The specific training process of the S2 LSTM-based effective wind speed estimation model includes: S21. Design an effective wind speed estimation network based on LSTM; S22. Set up a training dataset, which includes the LSTM input values ​​and LSTM target values ​​of the effective wind speed estimation network; S23. Define the sample estimation error of the effective wind speed estimation network, and calculate the objective function that minimizes the sample estimation error. S24. Define the update calculation formula for the effective wind speed estimation network; S25. Based on the training dataset and the update calculation formula of the effective wind speed estimation network, train and update the effective wind speed estimation network. When the training of the training dataset is completed, if the objective function for minimizing the sample estimation error is less than the set error threshold, stop the update iteration and obtain the trained LSTM-based effective wind speed estimation model.

4. The reward-adaptive wind turbine power control method according to claim 3, characterized in that, The specific steps of S4 are as follows: S41. Obtain the wind turbine aerodynamic power, wind turbine angular velocity, effective wind speed estimate, and operating status marker value at time t; S42. Obtain the reference value of the wind turbine rotation speed at time t based on the acquired data; S43. Based on the data obtained in S41 and the wind turbine rotation speed reference value in S42, the input of the first network and the input of the external reward calculation network at time t are obtained; S44. Combining the training experience at time t-1, the input of the first network at time t, the output of the first network at time t-1, the reward signal at time t-1, and the input of the first network at time t-1 are stored in the experience pool of the first network as the training experience at time t.

5. The reward-adaptive wind turbine power control method according to claim 3, characterized in that, The specific steps of S5 are as follows: S51. Design a first network based on the deep deterministic policy gradient algorithm. The first network includes an action neural network, an evaluation neural network, a first target network corresponding to the action neural network, and a second target network corresponding to the evaluation neural network. S52. The network designed in S51 is pre-trained by simulating turbulent wind speed environmental sequences. The specific steps of the pre-training are as follows: Obtain the input of the first network with index k in the sequence, use this input as the input vector of the action neural network, obtain the output action, and use the input vector and output action of the action neural network as the input vector of the evaluation neural network. Obtain the input of the first network with the next sequence number, which is k+1. Use this input as the input vector of the first target network to obtain the predicted output action. Use the input vector of the first target network and the predicted output action as the input vector of the second target network to complete the pre-training. S53. Construct an external reward calculation model and an internal reward calculation model. The external reward calculation model includes a reward action network and a reward evaluation network, and calculates the external reward. Based on the internal reward calculation model, feature extraction is performed on the input with index k and the input with index k+1 in S52 to obtain the state features corresponding to the two inputs respectively. A feedforward network is set up. The feedforward network predicts the first feature of the input with index k+1 based on the output action of S52 and the state features of the input with index k. At the same time, a feature network is learned. The feature network extracts features related to the input with index k and the output action in S52. S54. Based on the time difference algorithm, design the training process of the external reward calculation model, and set the prediction error and minimization function of the reward evaluation network; S55. Set the weight update rule for the reward evaluation network based on the minimization function, and iteratively update the weights of the reward evaluation network based on the weight update rule. When the number of iterations reaches the preset upper limit of the reward evaluation network, or the prediction error of the reward evaluation network is less than the preset first error threshold, stop the iteration, complete the training of the reward evaluation network, and input the output of the reward evaluation network at this time into the reward action network. S56. Based on the output of the reward evaluation network in S55, set the prediction error of the reward action network, set the weight update rule of the reward action network, and iteratively update the weight of the reward action network based on the weight update rule. When the number of iterations reaches the preset update upper limit of the reward action network, or the prediction error of the reward action network is less than the preset second error threshold, stop the iteration. At this time, the output of the reward action network is the updated external reward. S57. Calculate the prediction error of the feedforward network and the prediction error of the feature network in the internal reward calculation model. S58. Set the weight update rules for the feedforward network and the feature network. Iterate and update the feedforward network and the feature network respectively based on the weight update rules of the two networks. When the number of iterations reaches the preset update upper limit of the feedforward network and the feature network, or when the prediction error of the feedforward network is less than the preset third error threshold and the prediction error of the feature network is less than the preset fourth error threshold, stop the iterative update. Obtain the internal reward based on the output of the feedforward network at this time. Obtain the total single-step reward based on the updated external reward and internal reward. Store the single-step reward as a reward signal in the experience pool of the first network. S59. Obtain the first mini-batch training set from the experience pool, set the parameter update rules for the action neural network, and then update the first network using the update calculation formula in S24 based on the stochastic optimization algorithm to obtain the first network after training, and obtain the output of the first network at time t.

6. The reward-adaptive wind turbine power control method according to claim 5, characterized in that, The stochastic optimization algorithm is specifically a backpropagation algorithm optimized by the stochastic optimization algorithm with adaptive momentum.

7. The reward-adaptive wind turbine power control method according to claim 6, characterized in that, The specific steps for S7 are as follows: S71. Determine the experience pool in the power control process; S72. Obtain a second mini-batch training set from the experience pool in the power control process. The second mini-batch training set consists of experience samples from the experience pool. According to the training method in S54, update the parameters of the first network online based on the second mini-batch training set.

8. The reward-adaptive wind turbine power control method according to claim 5, characterized in that, The expression for the running state of S3 is as follows: ， in, and These are two different operating states. For wind turbine power generation, This is an effective wind speed estimate. Rated power generation capacity This is the rated wind speed.

9. The reward-adaptive wind turbine power control method according to claim 5, characterized in that, The expression for external rewards is: in, As an external reward, It is the distance between the input vector of the reward action network and the radial basis function. For radial basis functions, To reward the number of hidden layer neurons in the action network, To reward the Action Network The weights of each hidden neuron to the output layer i This represents the sequence number of the hidden layer neurons.

10. The reward-adaptive wind turbine power control method according to claim 5, characterized in that, The reward evaluation network is a three-layer BP neural network, consisting of an input layer, an output layer, and a hidden layer.

Images