Wind power timing adaptive window adjustment method and device based on reinforcement learning

Abstract

The application provides a wind power timing adaptive window adjustment method and device based on reinforcement learning, wherein the method comprises the following steps: collecting multi-source timing data from a wind turbine system or a database, and preprocessing the multi-source timing data to obtain a preprocessing data set; dividing the preprocessing data set according to a time sequence principle to obtain training data; training an initial LSTM model based on the training data to obtain a target LSTM model, and training a reinforcement learning model according to an output value of the target LSTM model and a reinforcement learning strategy; performing prediction based on the reinforcement learning model and the target LSTM model to obtain a prediction value, and inputting the prediction value and a real-time power feedback error into a discrete PID controller to obtain a control amount for driving a wind turbine actuator. The application combines the deep timing modeling capability of LSTM and the dynamic decision-making capability of reinforcement learning to intelligently adjust the length of a historical time window in wind power prediction.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of wind power generation technology, and in particular to a wind power generation timing adaptive window adjustment method based on reinforcement learning. Background Technology

[0002] Wind power output exhibits significant intermittency and random fluctuations, making accurate short-term wind condition forecasts crucial for wind turbine operation and grid stability. In recent years, recurrent neural networks such as LSTM have been widely applied to short-term wind power forecasting due to their strong ability to process time-series data, effectively capturing the complex correlations and long-short-term dependencies in wind speed sequences.

[0003] Traditional short-term power forecasting often uses manually preset fixed-length sliding windows. While these windows provide sufficient historical information when wind conditions are stable, they often fail to capture new dynamic characteristics quickly when faced with drastic changes such as gusts or sudden airflow shifts, leading to a significant drop in forecast accuracy. Furthermore, existing control systems mostly rely on forecasting before decision-making, lacking a feedforward linkage mechanism between the forecast results and the PID or energy storage dispatch controller. Once wind speed changes abruptly, the controller relies solely on feedback signals for compensation, resulting in significant response delays and overshoot risks.

[0004] Although there are patents that utilize reinforcement learning to optimize PID parameters or directly control fan parameters, they do not combine RL with adaptive adjustment of the timing prediction window, nor do they form an integrated "prediction-adjustment" closed-loop solution, making it difficult to simultaneously achieve synergistic optimization of prediction accuracy and control performance.

[0005] Therefore, a better solution is urgently needed. Summary of the Invention

[0006] In view of this, embodiments of this specification provide a wind power generation time-series adaptive window adjustment method based on reinforcement learning. One or more embodiments of this specification also relate to a wind power generation time-series adaptive window adjustment device based on reinforcement learning, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0007] According to a first aspect of the embodiments of this specification, a wind power generation time-series adaptive window adjustment method based on reinforcement learning is provided, comprising:

[0008] Collect multi-source time-series data from wind turbine systems or databases, and preprocess the multi-source time-series data to obtain a preprocessed dataset;

[0009] The preprocessed dataset is divided according to the time order to obtain the training data;

[0010] The initial LSTM model is trained based on the training data to obtain the target LSTM model, and the reinforcement learning model is trained based on the output value of the target LSTM model and the reinforcement learning strategy. The initial LSTM model is trained using a random sampling window length mechanism with a preset window length range. The reinforcement learning model dynamically selects the optimal window length based on the current state.

[0011] Prediction is performed based on a reinforcement learning model and a target LSTM model to obtain predicted values. These predicted values, along with the real-time power feedback error, are then input into a discrete PID controller to obtain the control quantity that drives the fan actuator.

[0012] In one possible implementation, multi-source time-series data is collected from the wind turbine system or a database, including:

[0013] Collect wind speed, wind direction, temperature, humidity, and output power from the wind turbine system or database.

[0014] In one possible implementation, the preprocessed dataset is divided according to a chronological order to obtain training data, including:

[0015] Obtain the data from the first time period of the preprocessed dataset to determine the training set;

[0016] Obtain data from the second time period from the training set to determine the validation set;

[0017] Obtain the data from the third time period of the preprocessed dataset to determine the test set;

[0018] The first time period consists of the initial data of the preprocessed dataset, and the third time period consists of the final data of the preprocessed dataset.

[0019] In one possible implementation, an initial LSTM model is trained based on training data to obtain a target LSTM model, including:

[0020] Set the range of window length;

[0021] Based on the range of window lengths, the window length is randomly sampled in each training batch;

[0022] Construct the input sequence based on the window length;

[0023] The initial LSTM model is trained based on the input sequence to obtain the target LSTM model;

[0024] The initial LSTM model includes an input layer, multiple LSTM units, a Dropout layer, and a fully connected output layer.

[0025] In one possible implementation, a reinforcement learning model is trained based on a reinforcement learning policy, using the output value of the target LSTM model and the reinforcement learning policy, including:

[0026] Based on the action space and state space, a historical sequence is extracted from the training data, the current state is calculated, and an action is selected to determine the length of the window based on the output of the policy network.

[0027] Prediction is performed based on window length adjustment, historical sequences, and the target LSTM model to determine the output value;

[0028] A reinforcement learning model is obtained by training based on the output value and the reward function.

[0029] In one possible implementation, a reinforcement learning model is trained based on the output value and the reward function, including:

[0030] Determine the error between the output value and the actual power;

[0031] Adjust the network parameters of the adjustment strategy based on the error and reward functions;

[0032] The reward function includes:

[0033]

[0034] Where α is the penalty coefficient measuring prediction error; β is the penalty coefficient measuring window adjustment cost; δt is the difference between the true value and the predicted value; W t Adjust the window length; W t-1 The window length used last time.

[0035] In one possible implementation, predictions are made based on a reinforcement learning model and a target LSTM model to obtain predicted values. These predicted values, along with the real-time power feedback error, are then input into a discrete PID controller to obtain the control quantities for the wind turbine actuator, including:

[0036] At each time step, the current window length is determined based on the reinforcement learning model;

[0037] The target LSTM model extracts features from historical sequences based on the current window length and outputs predicted power;

[0038] Determine the current power, compare the current power with the expected power, and obtain the error power;

[0039] The PID control quantity is calculated based on the error power, and the predicted power weighted as the feedforward compensation signal is superimposed to generate the control quantity.

[0040] According to a second aspect of the embodiments of this specification, a wind power generation timing adaptive window adjustment device based on reinforcement learning is provided, comprising:

[0041] The data processing module is configured to collect multi-source time-series data from the wind turbine system or database, and preprocess the multi-source time-series data to obtain a preprocessed dataset.

[0042] The data partitioning module is configured to partition the preprocessed dataset according to a chronological order to obtain training data;

[0043] The model training module is configured to train an initial LSTM model based on training data to obtain a target LSTM model, and to train a reinforcement learning model based on the output value of the target LSTM model and the reinforcement learning strategy. The initial LSTM model is trained using a random sampling window length mechanism, with the window length range being a preset range. The reinforcement learning model dynamically selects the optimal window length based on the current state.

[0044] The collaborative control module is configured to make predictions based on a reinforcement learning model and a target LSTM model, obtain predicted values, and input the predicted values ​​and real-time power feedback error into a discrete PID controller to obtain the control quantity that drives the fan actuator.

[0045] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0046] Memory and processor;

[0047] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-described wind power generation timing adaptive window adjustment method based on reinforcement learning.

[0048] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described reinforcement learning-based wind power generation timing adaptive window adjustment method.

[0049] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is instructed to perform the steps of the above-described wind power generation timing adaptive window adjustment method based on reinforcement learning.

[0050] This specification provides a method and apparatus for adaptive window adjustment of wind power generation time series based on reinforcement learning. The method includes: collecting multi-source time series data from a wind turbine system or database, and preprocessing the multi-source time series data to obtain a preprocessed dataset; dividing the preprocessed dataset according to the time sequence principle to obtain training data; training an initial LSTM model based on the training data to obtain a target LSTM model; training the target LSTM model based on a reinforcement learning strategy to obtain a reinforced LSTM model, so that the reinforced LSTM model dynamically selects the optimal window length according to the current state; making predictions based on the reinforced LSTM model to obtain predicted values, and inputting the predicted values ​​and real-time power feedback error into a discrete PID controller to obtain the control quantity driving the wind turbine actuator. By introducing a reinforcement learning mechanism, adaptive dynamic adjustment of the input time window of the LSTM prediction model is achieved, enabling the prediction system to determine the input data length in real time based on the volatility, trend characteristics, and historical prediction errors of key variables such as wind speed, thereby effectively improving the model's response and generalization capabilities under different wind conditions, and improving short-term prediction accuracy and robustness. The long short-term memory capability of LSTM networks enables the system to more accurately capture the nonlinear dynamic characteristics of wind speed sequences, thereby enhancing predictive capabilities. Simultaneously, using the prediction results for feedforward control significantly improves the wind turbine's response speed to sudden wind speed changes, reducing power and load fluctuations. In summary, the proposed solution significantly improves the stability and robustness of the wind turbine control system, reduces the risk of wind power output fluctuations, and ensures the reliable operation of the power grid. Attached Figure Description

[0051] Figure 1 This is a flowchart of a wind power generation timing adaptive window adjustment method based on reinforcement learning, provided in one embodiment of this specification.

[0052] Figure 2 This is a schematic diagram of a wind power generation timing adaptive window adjustment device based on reinforcement learning, provided in one embodiment of this specification.

[0053] Figure 3 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0054] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0055] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0056] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0057] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0058] LSTM (Long Short-Term Memory) is a special type of recurrent neural network (RNN) that is able to learn long-term dependent information.

[0059] PID (Proportional-Integral-Derivative) is a common control algorithm that is widely used in various control systems.

[0060] This specification provides a wind power generation time-series adaptive window adjustment method based on reinforcement learning. This specification also relates to a wind power generation time-series adaptive window adjustment device based on reinforcement learning, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0061] See Figure 1 , Figure 1 A flowchart of a wind power generation timing adaptive window adjustment method based on reinforcement learning according to an embodiment of this specification is shown, which specifically includes the following steps.

[0062] Step 101: Collect multi-source time-series data from the wind turbine system or database, and preprocess the multi-source time-series data to obtain a preprocessed dataset.

[0063] In one possible implementation, multi-source time-series data is collected from the wind turbine system or database, including wind speed, wind direction, temperature, humidity, and output power.

[0064] In practical applications, the raw data collected includes: wind speed (v) t ): Collected by an anemometer, reflecting the current wind intensity; wind direction (θ) t ): Indicates the prevailing wind direction and can be used as an auxiliary meteorological variable; ambient temperature (T) t ) and humidity (h t ): Can be used to identify the impact of weather conditions on wind energy conversion efficiency; output power (P) t ): The actual power generation of the wind turbine at the current moment; other auxiliary variables: such as nacelle speed, wind turbine load, grid frequency, etc. (expandable).

[0065] Since the original collected data may contain noise, outliers, sensor malfunctions, or missing values, the following processing is required: Outlier identification and removal: Statistical methods such as the 3σ principle and box plots are used, or a median filter within a sliding window is used to identify and remove points that are significantly outside the physical range; Missing value imputation: Linear interpolation, moving average, or other methods are used to imput small-scale missing data, while large-scale missing data blocks are discarded; Normalization or standardization: Min-Max normalization or Z-score standardization is applied to all input features to ensure that the distribution of each dimension of features is of the same magnitude, avoiding bias towards variables with large amplitudes during model training.

[0066] To enable the reinforcement learning model to perceive the fluctuations and trends of wind conditions in real time, a sliding window W is used. s Internally, the following statistical features are maintained in real time to construct the agent's state input: wind speed variance (σ v,t 2 ): Measures the degree of drastic change in wind speed over a current period of time, reflecting the stability of the wind field;

[0067]

[0068] Where vˉ t This represents the average wind speed within the current window.

[0069] Humidity variance (σ) h,t 2 ): Reflects fluctuations in air humidity, which can indirectly reflect the intensity of weather changes.

[0070] Wind speed trend slope (Δ) trend,t This indicates whether the wind speed within the window shows a significant upward or downward trend, which can be achieved using linear regression slope or simple difference.

[0071]

[0072] These features are updated using incremental computation (rolling window updates), which eliminates the need to rescan the entire window each time, enabling high-frequency real-time updates in real-world deployments.

[0073] Step 102: Divide the preprocessed dataset according to the time order to obtain the training data.

[0074] In one possible implementation, the preprocessed dataset is divided according to a time-order principle to obtain training data, including: obtaining the first time period data of the preprocessed dataset to determine the training set; obtaining the second time period data from the training set to determine the validation set; and obtaining the third time period data of the preprocessed dataset to determine the test set; wherein the first time period is the beginning of the preprocessed dataset and the third time period is the end of the preprocessed dataset.

[0075] In practical applications, data in wind power generation scenarios exhibit significant temporal correlation and non-stationarity. This means that current prediction results are highly dependent on historical data, and different time periods (such as day / night, season / monsoon) have different statistical characteristics. Therefore, random partitioning should not be used; instead, a strict temporal order partitioning principle should be followed.

[0076] Training set: Generally, the first 70% of the entire dataset is used to train the LSTM model and the reinforcement learning window policy model. Validation set: Approximately 10% of the time period is further allocated from the training set for: dynamically monitoring the model's performance on non-training data, guiding hyperparameter tuning, and preventing overfitting through strategies such as early stopping. Test set: The last 30% of the dataset is retained for online simulations to evaluate the system's generalization ability on data not seen before actual deployment. The test set is only used for joint testing and closed-loop validation and is prohibited from being used for any training or hyperparameter tuning.

[0077] Step 103: Train the initial LSTM model based on the training data to obtain the target LSTM model, and train the reinforcement learning model based on the output value of the target LSTM model and the reinforcement learning strategy; wherein, the initial LSTM model is trained using a random sampling window length mechanism, the window length range is a preset range, and the reinforcement learning model dynamically selects the optimal window length according to the current state.

[0078] In one possible implementation, training an initial LSTM model based on training data to obtain a target LSTM model includes: setting a range of window lengths; randomly sampling window lengths in each training batch based on the range of window lengths; constructing an input sequence based on the window lengths; and training the initial LSTM model based on the input sequence to obtain the target LSTM model. The initial LSTM model includes an input layer, multiple LSTM units, a Dropout layer, and a fully connected output layer.

[0079] In practical applications, the LSTM model constructed in this application has the following structural features: Input layer: receives multi-dimensional input feature sequences, including wind speed, wind direction, humidity, temperature, historical power, etc.; Multi-layer LSTM units: adopt a stacked structure (such as 2~3 layers), each layer contains several LSTM units to extract deep temporal features; Dropout layer: used to alleviate overfitting and improve model robustness; Fully connected output layer (Dense): maps the hidden state of the last time step of the LSTM to the output power prediction value P^t; Output dimension: a single value, representing the predicted value of the wind turbine output power in the short term.

[0080] In practical applications, the length of the historical sequence input to the LSTM model will be dynamically decided by the reinforcement learning agent (see later sections for details). Therefore, during the model training phase, the LSTM needs to be "seen" with historical data of different lengths in advance to ensure the generalization ability during the deployment phase.

[0081] The training mechanism is as follows:

[0082] The window length can be set to a range of values: W∈[Wmin,Wmax]; for example, from 6 time steps (30 minutes) to 24 time steps (120 minutes), predicting the power for the next five minutes. Smaller sampling frequencies, such as three minutes or one minute, can also be selected based on specific needs, but the computational cost will increase accordingly. In each training batch, a window of length Wsample is randomly sampled and used to construct the input sequence for all samples in that batch.

[0083] For each sample, based on the current Wsample, feature sequences from past Wsample time steps are extracted from the original data and used as input to the LSTM. Through this perturbation mechanism, the model continuously receives contextual inputs of different time lengths during training, thereby learning to extract effective features under different "memory lengths." This mechanism ensures that the LSTM can handle input sequences from different windows, enhancing its adaptability and robustness.

[0084] The mean squared error loss function is used to evaluate the deviation between predicted and true values:

[0085]

[0086] Where: P^i: wind power predicted by the model; Pi: actual observed wind power; N: number of samples in the batch.

[0087] Using the Adam optimizer for backpropagation and parameter updates has the advantages of adaptive learning rate and first and second moment estimation of gradients, making it suitable for processing non-stationary wind power data with a lot of noise.

[0088] Set an appropriate number of training rounds (e.g., 100 rounds), and dynamically monitor the model performance using the aforementioned pre-defined validation set during training. When the validation set MSE does not decrease significantly for several consecutive rounds, early stopping can be triggered to avoid overfitting. Finally, save the model parameters with the lowest validation error as the final LSTM model for subsequent online prediction after integration with reinforcement learning strategies.

[0089] In one possible implementation, a reinforcement learning model is trained based on a reinforcement learning policy, using the output value of the target LSTM model and the reinforcement learning policy. This includes: extracting a historical sequence from the training data based on the action space and state space, calculating the current state, and selecting an action to determine the window adjustment length based on the policy network output; making predictions based on the window adjustment length, the historical sequence, and the target LSTM model to determine the output value; and training the reinforcement learning model based on the output value and the reward function.

[0090] In practical applications, one of the key innovations of this application is the introduction of reinforcement learning strategies to construct an adaptive time window adjustment mechanism, which drives the LSTM prediction model to automatically select the optimal historical sequence length. This allows for dynamic adjustment of the prediction strategy under conditions of drastic wind speed changes or non-stationary data, achieving a dual optimization of robustness and prediction accuracy.

[0091] Action Space Definition: Considering practical applications, to avoid model instability or increased system resource consumption due to frequent switching, this application adopts a discrete action space design. The set of actions the agent can choose at each time step is defined as: A = {-ΔW, 0, +ΔW}

[0092] Where: ΔW: window length adjustment step, for example, 30min; -ΔW: decrease the current window length by one step; 0: keep the current window length unchanged; +ΔW: increase the current window length by one step. To prevent the window length from going out of bounds, the allowed value range is set as: W∈[Wmin,Wmax], for example [30min,120min].

[0093] State space design (State Representation): To support the agent in making reasonable window decisions under different operating conditions, its observed state (State) needs to cover multiple statistical information related to prediction performance, defined as:

[0094] s t =[σ v,t ,Δ trendt ,δ t-1 W t-1 ]

[0095] The meaning is as follows: σ v,tΔ represents the variance of the wind speed sequence within the current window, characterizing the degree of fluctuation. trendt The slope of the wind speed change within the current window indicates whether the wind speed is increasing, decreasing, or remaining constant; δ t-1 : The absolute error of the previous prediction; W t-1 : The length of the window used last time.

[0096] This state vector forms the input to the reinforcement learning policy network. The agent uses this state vector to determine whether the input window needs to be extended or shortened in order to optimize the current and future prediction performance.

[0097] In one possible implementation, a reinforcement learning model is trained based on the output value and a reward function, including: determining the error between the output value and the actual power; adjusting the network parameters based on the error and the reward function; the reward function includes:

[0098]

[0099] Where α is the penalty coefficient measuring prediction error; β is the penalty coefficient measuring window adjustment cost; δt is the difference between the true value and the predicted value; W t Adjust the window length; W t-1 The window length used last time.

[0100] In practical applications, the training process simulates the impact of different window selections on the prediction results, and the process is as follows:

[0101] State sampling: Extract a historical sequence from the training data and calculate the current state s. t Action selection: The agent selects an action 'a' based on the policy network output. t ∈A, adjust window length W t =W t -1+a t Predictive execution: Truncate to length W t The historical sequence is input into the LSTM model to obtain the predicted power P^t; Error calculation: calculate the difference between the true value and the predicted value δt=|Pt-P^t|; Reward function calculation:

[0102]

[0103] The reward function design reflects the optimization goal of "high precision + low frequency switching".

[0104] Policy update: Use DQN or DDQN algorithms to update the Q network, optimize the action value function Q(s,a), and periodically update the target network to improve training stability.

[0105] Set an upper limit on the number of training rounds or use an early stopping strategy to stop training when the average reward does not increase significantly for several consecutive rounds; save the final trained main network parameters as a window adjustment strategy module in the online system; after deployment, the system can dynamically select the optimal window length based on the current state at each prediction time, improving the model's ability to adapt to data mutations and fluctuations.

[0106] Step 104: Make predictions based on the reinforcement learning model and the target LSTM model to obtain the predicted values, and input the predicted values ​​and the real-time power feedback error into the discrete PID controller to obtain the control quantity that drives the fan actuator.

[0107] In one possible implementation, predictions are made based on a reinforcement learning model and a target LSTM model to obtain predicted values. These predicted values, along with the real-time power feedback error, are then input into a discrete PID controller to obtain the control quantity driving the wind turbine actuator. This includes: at each time step, determining the current window length based on the reinforcement learning model; the target LSTM model extracting features from the historical sequence based on the current window length and outputting the predicted power; determining the current power and comparing it with the expected power to obtain the error power; calculating the PID adjustment quantity based on the error power; and using the predicted power as a weighted feedforward compensation signal to generate the control quantity.

[0108] In practical applications, in order to improve the response capability of wind power systems to sudden changes in wind conditions, this application introduces LSTM-based predicted values ​​as feedforward signals on the basis of traditional feedback power control strategies, and constructs a "prediction-adjustment" collaborative control closed-loop structure, so that the controller can intervene in advance before the arrival of future wind conditions, effectively reducing lag adjustment errors and load fluctuations, and improving system stability and power generation efficiency.

[0109] Specifically, it consists of the following three core components: Feedforward channel: power prediction result P^t+1 based on LSTM model; Feedback channel: based on the deviation between the current real-time output power Pt of the wind turbine and the target setpoint Pset; Controller core: discrete PID controller, outputting control quantity ut, controlling actuators (such as pitch control system, speed control, excitation system, etc.) to make dynamic response adjustments.

[0110] The specific collaborative control process is as follows.

[0111] 1. Real-time prediction: At each time step t, the LSTM model uses the window length Wt selected by the reinforcement learning strategy to extract features from the historical sequence and output the prediction power P^t+1;

[0112] 2. Error feedback calculation: The system measures the current power Pt and compares it with the expected power Pset to obtain the error et;

[0113] 3. Control output generation: The PID regulation is calculated using et, and the predicted value P^t+1 is weighted and used as a feedforward compensation signal to generate the control quantity ut.

[0114] 4. Command execution: The control quantity ut is transmitted to the wind turbine control system, such as adjusting the pitch angle, controlling the generator excitation, modifying the speed setting, etc., to drive the wind turbine system to respond to changes in wind conditions in advance.

[0115] This application introduces a window adaptation mechanism based on reinforcement learning: Through reinforcement learning algorithms (such as DQN or DDQN), different historical time window lengths are dynamically selected based on wind condition fluctuation characteristics (such as wind speed variance, trend slope, etc.) for time-series prediction input. This differs from traditional fixed-window prediction, enhancing the model's responsiveness to sudden changes in operating conditions and its flexible modeling ability based on historical dependencies. A state space is constructed based on multi-source statistical features: the state input not only includes historical errors and window length but also incrementally maintains statistical features (such as variance and trend slope) of multiple variables such as wind speed and humidity, used by the reinforcement learning agent to determine whether the window needs adjustment in the current environment. This design improves the discriminative power of the state representation and solves the problem of traditional RL prediction frameworks lacking stable observation signals. An LSTM network supports a prediction model training mechanism with variable-length inputs: during the LSTM network training phase, a window length perturbation mechanism is introduced, i.e., the historical window length is sampled within each batch, allowing the model to learn feature extraction capabilities under both short and long historical conditions. This improves the model's robustness and generalization ability to adaptive window inputs, meeting the dynamic calling requirements of the deployment phase. Cooperative control architecture of predictive feedforward and feedback control: The short-time power prediction result of LSTM is used as a feedforward signal, which, together with the feedback error of PID, drives the controller output; forming a predictive-adjustment closed-loop control structure, realizing early compensation and accelerated response to sudden changes in wind conditions, and improving control accuracy.

[0116] Corresponding to the above method embodiments, this specification also provides an embodiment of a wind power generation timing adaptive window adjustment device based on reinforcement learning. Figure 2 A schematic diagram of a wind power generation timing adaptive window adjustment device based on reinforcement learning, according to one embodiment of this specification, is shown. Figure 2 As shown, the device includes:

[0117] The data processing module 201 is configured to collect multi-source time-series data from the wind turbine system or database, and preprocess the multi-source time-series data to obtain a preprocessed dataset.

[0118] The data partitioning module 202 is configured to partition the preprocessed dataset according to the time order principle to obtain training data;

[0119] The model training module 203 is configured to train an initial LSTM model based on training data to obtain a target LSTM model, and to train a reinforcement learning model based on the output value of the target LSTM model and the reinforcement learning strategy. The initial LSTM model is trained using a random sampling window length mechanism, with the window length range being a preset range. The reinforcement learning model dynamically selects the optimal window length based on the current state.

[0120] The collaborative control module 204 is configured to make predictions based on a reinforcement learning model and a target LSTM model, obtain predicted values, and input the predicted values ​​and real-time power feedback error into a discrete PID controller to obtain the control quantity that drives the fan actuator.

[0121] In one possible implementation, multi-source time-series data is collected from the wind turbine system or a database, including:

[0122] Collect wind speed, wind direction, temperature, humidity, and output power from the wind turbine system or database.

[0123] In one possible implementation, the preprocessed dataset is divided according to a chronological order to obtain training data, including:

[0124] Obtain the data from the first time period of the preprocessed dataset to determine the training set;

[0125] Obtain data from the second time period from the training set to determine the validation set;

[0126] Obtain the data from the third time period of the preprocessed dataset to determine the test set;

[0127] The first time period consists of the initial data of the preprocessed dataset, and the third time period consists of the final data of the preprocessed dataset.

[0128] In one possible implementation, an initial LSTM model is trained based on training data to obtain a target LSTM model, including:

[0129] Set the range of window length;

[0130] Based on the range of window lengths, the window length is randomly sampled in each training batch;

[0131] Construct the input sequence based on the window length;

[0132] The initial LSTM model is trained based on the input sequence to obtain the target LSTM model;

[0133] The initial LSTM model includes an input layer, multiple LSTM units, a Dropout layer, and a fully connected output layer.

[0134] In one possible implementation, a reinforcement learning model is trained based on a reinforcement learning policy, using the output value of the target LSTM model and the reinforcement learning policy, including:

[0135] Based on the action space and state space, a historical sequence is extracted from the training data, the current state is calculated, and an action is selected to determine the length of the window based on the output of the policy network.

[0136] Prediction is performed based on window length adjustment, historical sequences, and the target LSTM model to determine the output value;

[0137] A reinforcement learning model is obtained by training based on the output value and the reward function.

[0138] In one possible implementation, a reinforcement learning model is trained based on the output value and the reward function, including:

[0139] Determine the error between the output value and the actual power;

[0140] Adjust the network parameters of the adjustment strategy based on the error and reward functions;

[0141] The reward function includes:

[0142]

[0143] Where α is the penalty coefficient measuring prediction error; β is the penalty coefficient measuring window adjustment cost; δt is the difference between the true value and the predicted value; W t Adjust the window length; W t-1 The window length used last time.

[0144] In one possible implementation, predictions are made based on a reinforcement learning model and a target LSTM model to obtain predicted values. These predicted values, along with the real-time power feedback error, are then input into a discrete PID controller to obtain the control quantities for the wind turbine actuator, including:

[0145] At each time step, the current window length is determined based on the reinforcement learning model;

[0146] The target LSTM model extracts features from historical sequences based on the current window length and outputs predicted power;

[0147] Determine the current power, compare the current power with the expected power, and obtain the error power;

[0148] The PID control quantity is calculated based on the error power, and the predicted power weighted as the feedforward compensation signal is superimposed to generate the control quantity.

[0149] The above is a schematic scheme of a wind power generation timing adaptive window adjustment device based on reinforcement learning according to this embodiment. It should be noted that the technical solution of this wind power generation timing adaptive window adjustment device based on reinforcement learning belongs to the same concept as the technical solution of the wind power generation timing adaptive window adjustment method based on reinforcement learning described above. Details not described in detail in the technical solution of the wind power generation timing adaptive window adjustment device based on reinforcement learning can be found in the description of the technical solution of the wind power generation timing adaptive window adjustment method based on reinforcement learning described above.

[0150] Figure 3 A structural block diagram of a computing device 300 according to one embodiment of this specification is shown. The components of the computing device 300 include, but are not limited to, a memory 310 and a processor 320. The processor 320 is connected to the memory 310 via a bus 330, and a database 350 is used to store data.

[0151] The computing device 300 also includes an access device 340, which enables the computing device 300 to communicate via one or more networks 360. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 340 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0152] In one embodiment of this specification, the aforementioned components of the computing device 300 and Figure 3 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 3The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0153] The computing device 300 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 300 can also be a mobile or stationary server.

[0154] The processor 320 executes the following computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned wind power generation timing adaptive window adjustment method based on reinforcement learning. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned wind power generation timing adaptive window adjustment method based on reinforcement learning belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the aforementioned wind power generation timing adaptive window adjustment method based on reinforcement learning.

[0155] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described reinforcement learning-based wind power generation timing adaptive window adjustment method.

[0156] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the wind power generation time-series adaptive window adjustment method based on reinforcement learning described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the wind power generation time-series adaptive window adjustment method based on reinforcement learning described above.

[0157] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is instructed to perform the steps of the above-described wind power generation timing adaptive window adjustment method based on reinforcement learning.

[0158] The above is an illustrative scheme of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the wind power generation time-series adaptive window adjustment method based on reinforcement learning described above. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solution of the wind power generation time-series adaptive window adjustment method based on reinforcement learning described above.

[0159] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0160] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0161] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0162] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0163] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A wind power generation time-series adaptive window adjustment method based on reinforcement learning, characterized in that, include: Collect multi-source time-series data from wind turbine systems or databases, and preprocess the multi-source time-series data to obtain a preprocessed dataset; The preprocessed dataset is divided according to the time order to obtain training data; The initial LSTM model is trained based on the training data to obtain the target LSTM model, and a reinforcement learning model is trained based on the output value of the target LSTM model and the reinforcement learning strategy; wherein, the initial LSTM model is trained using a random sampling window length mechanism, the window length range is a preset range, and the reinforcement learning model dynamically selects the optimal window length according to the current state; Based on the reinforcement learning model and the target LSTM model, a prediction is made to obtain a predicted value. The predicted value and the real-time power feedback error are then input into a discrete PID controller to obtain the control quantity that drives the wind turbine actuator. The step of training a reinforcement learning model based on the output value of the target LSTM model and the reinforcement learning policy includes: Based on the action space and state space, a historical sequence is extracted from the training data, the current state is calculated, and an action is selected to determine the length of the window based on the output of the policy network. Based on the adjusted window length, the historical sequence, and the target LSTM model, a prediction is made to determine the output value; A reinforcement learning model is obtained by training based on the output value and the reward function; The reinforcement learning model trained based on the output value and the reward function includes: Determine the error between the output value and the actual power; Adjust the policy network parameters based on the error and the reward function; The reward function includes: Where α is the penalty coefficient measuring prediction error; β is the penalty coefficient measuring window adjustment cost; δt is the difference between the true value and the predicted value; W t Adjust the length of the window; W t 1 represents the length of the window used last time.

2. The method according to claim 1, characterized in that, The acquisition of multi-source time-series data from the wind turbine system or database includes: Collect wind speed, wind direction, temperature, humidity, and output power from the wind turbine system or database.

3. The method according to claim 1, characterized in that, The preprocessed dataset is divided according to a chronological order to obtain training data, including: Obtain the data from the first time period of the preprocessed dataset to determine the training set; Obtain data from the second time period from the training set to determine the validation set; Obtain the data from the third time period of the preprocessed dataset to determine the test set; Wherein, the first time period is the beginning of the preprocessed dataset, and the third time period is the end of the preprocessed dataset.

4. The method according to claim 1, characterized in that, The step of training the initial LSTM model based on the training data to obtain the target LSTM model includes: Set the range of window length; Based on the range of the window length, the window length is randomly sampled in each training batch; Construct the input sequence based on the window length; The initial LSTM model is trained based on the input sequence to obtain the target LSTM model; The initial LSTM model includes an input layer, multiple LSTM units, a Dropout layer, and a fully connected output layer.

5. The method according to claim 1, characterized in that, The process of making predictions based on the reinforcement learning model and the target LSTM model to obtain predicted values, and then inputting the predicted values ​​and the real-time power feedback error into a discrete PID controller to obtain the control quantity driving the wind turbine actuator, includes: At each time step, the current window length is determined based on the reinforcement learning model; The target LSTM model extracts features from the historical sequence based on the current window length and outputs the predicted power. Determine the current power, compare the current power with the expected power, and obtain the error power; The PID control quantity is calculated based on the error power, and the predicted power is weighted as a feedforward compensation signal and superimposed to generate the control quantity.

6. A wind power generation time-series adaptive window adjustment device based on reinforcement learning, used to implement the wind power generation time-series adaptive window adjustment method based on reinforcement learning as described in any one of claims 1 to 5, characterized in that, include: The data processing module is configured to collect multi-source time-series data from the wind turbine system or database, and preprocess the multi-source time-series data to obtain a preprocessed dataset. The data partitioning module is configured to partition the preprocessed dataset according to a time-order partitioning principle to obtain training data; The model training module is configured to train an initial LSTM model based on the training data to obtain a target LSTM model, and to train a reinforcement learning model based on the output value of the target LSTM model and a reinforcement learning strategy; wherein, the initial LSTM model is trained using a random sampling window length mechanism, the window length range is a preset range, and the reinforcement learning model dynamically selects the optimal window length according to the current state; The collaborative control module is configured to make predictions based on the reinforcement learning model and the target LSTM model, obtain predicted values, and input the predicted values ​​and real-time power feedback error into a discrete PID controller to obtain the control quantity that drives the fan actuator.

7. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the wind power generation timing adaptive window adjustment method based on reinforcement learning as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the wind power generation timing adaptive window adjustment method based on reinforcement learning as described in any one of claims 1 to 5.

Images