A multi-time scale fusion wind speed prediction method based on a double-encoder UNet model

Abstract

The application provides a kind of multi-time scale fusion wind speed prediction method based on double encoder UNet model, comprising the following steps: S1, constructing double encoder;S2, encoder feature fusion: realize the feature fusion between numerical prediction wind speed encoder and historical wind speed encoder by cross attention mechanism;S3, constructing multi-time scale decoder;S4, multi-time scale prediction result fusion: the prediction results of different time scales are fused by multi-scale fusion module to generate the final wind speed prediction result;S5, wind speed prediction UNet model is constructed based on steps S1-S4;S6, training data collection, training database construction, model training;S7, wind speed prediction.The application can not only effectively capture the multi-time scale characteristics of wind speed, but also realize the online coupling of historical wind speed data and numerical weather prediction model, significantly improve the accuracy and robustness of wind speed prediction.

Description

Technical Field

[0001] This invention belongs to the field of machine learning, specifically relating to a multi-timescale fusion wind speed prediction method based on a dual-encoder UNet model. Background Technology

[0002] With the continuous expansion of wind power capacity, the demand for short- and medium-term wind speed forecasting is increasing. Traditional wind speed forecasting methods mainly rely on numerical weather prediction models, which simulate atmospheric motion through physical models to predict wind speed. However, these methods often ignore the time dependence and multi-timescale characteristics of historical wind speed data, resulting in limited prediction accuracy. To improve prediction accuracy, more and more research is beginning to combine historical wind speed data and numerical weather prediction models, utilizing machine learning methods for wind speed forecasting. For example, Chinese patent CN110674965A proposes an intelligent hybrid model based on a dynamic feature extraction algorithm, which solves the problem of wind speed forecasting at different time scales by mining historical wind speed time series and numerical weather prediction data. Chinese patent CN112529282A discloses a wind speed forecasting method based on a spatiotemporal graph convolutional neural network, which uses a bidirectional gated recurrent unit (BiGRU) and a graph convolutional network (GCN) to jointly train historical wind speed data and weather forecast parameters, significantly improving the accuracy of wind speed forecasting.

[0003] However, existing wind speed prediction methods still have the following shortcomings: First, most methods only focus on wind speed prediction at a single time scale and fail to make full use of the rich information in wind speed data at multiple time scales; second, traditional methods usually couple historical wind speed data with numerical weather prediction models offline, failing to achieve dynamic interaction and feature fusion between the two; finally, when dealing with the time dependence and nonlinear characteristics in wind speed data, existing methods often rely on simple neural network structures, failing to fully capture the complex patterns of wind speed changes. Summary of the Invention

[0004] The main objective of this invention is to provide a multi-timescale fusion wind speed prediction method based on a dual-encoder UNet model, addressing the aforementioned problems.

[0005] Therefore, the above-mentioned objective of the present invention is achieved through the following technical solution:

[0006] A multi-timescale fusion wind speed prediction method based on a dual-encoder UNet model includes the following steps:

[0007] S1. Constructing a dual encoder: The dual encoder mainly consists of two modules: a numerical prediction wind speed encoder and a historical wind speed encoder. Each encoder contains a multi-layer feature extraction module.

[0008] S2, Encoder Feature Fusion: Feature fusion between the numerical prediction wind speed encoder and the historical wind speed encoder is achieved through a cross-attention mechanism;

[0009] S3. Construct a multi-timescale decoder: decode the generated fused features;

[0010] S4. Multi-timescale prediction result fusion: The multi-scale fusion module fuses prediction results from different time scales to generate the final wind speed prediction result.

[0011] S5. Based on steps S1-S4, construct the UNet model for wind speed prediction, and collect training data, construct the training database, and train the model.

[0012] S6. Wind speed prediction: Use the trained UNet model for wind speed prediction to predict wind speed.

[0013] While adopting the above technical solutions, the present invention may also adopt or combine the following technical solutions:

[0014] As a preferred technical solution of the present invention: In step S1, both encoders are composed of multiple two-layer convolutional modules. Each two-layer convolutional module consists of two convolutional layers, and each convolutional layer is followed by a ReLU activation function and layer normalization.

[0015] As a preferred technical solution of the present invention: In step S1, a time attention mechanism module is introduced, which consists of three convolutional layers: query, key, and value. By calculating the attention weights between different time steps in the time series, the time dependence is captured.

[0016] As a preferred technical solution of the present invention: In step S2, the cross-attention mechanism adopts a multi-head attention mechanism, wherein the features of the numerical prediction wind speed encoder are used as queries, and the features of the historical wind speed encoder are used as keys and values, and the fusion between the features of the two encoders is achieved by calculating attention weights.

[0017] As a preferred technical solution of the present invention: In step S3, the multi-timescale decoder includes three upsampling modules. Each upsampling module gradually improves the temporal resolution of the feature map through deconvolution operation and further extracts features through a double-layer convolution module.

[0018] As a preferred technical solution of the present invention: In step S4, the multi-scale fusion module performs linear interpolation on the prediction results of different time scales, unifies them to the same time resolution, and fuses them through a convolutional layer to finally output a comprehensive wind speed prediction result.

[0019] As a preferred technical solution of the present invention: In step S5, the training data collection first requires collecting historical wind speed data and numerical prediction wind speed data. The prediction wind speed data comes from the numerical weather prediction model, and the historical wind speed data comes from the observation data of the meteorological station or wind farm.

[0020] As a preferred technical solution of the present invention: In step S5, during the construction of the training database, multiple training samples are generated by means of a sliding window. The size and step size of the sliding window can be adjusted according to the prediction requirements. When the amount of data is small, a smaller time step can be selected to ensure that the amount of training samples is sufficient.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] This invention constructs a dual encoder to extract multi-level features of historical wind speed and numerically predicted wind speed, and introduces a temporal attention mechanism to capture the temporal dependence in the wind speed data. A cross-attention mechanism is used to dynamically fuse the features of historical and numerically predicted wind speed, and a multi-timescale decoder is combined to generate wind speed prediction results at different time scales. Finally, a multi-scale fusion module integrates the prediction results from different time scales to output a high-precision comprehensive wind speed prediction. This invention not only effectively captures the multi-timescale features of wind speed but also achieves online coupling of historical wind speed data with numerical weather prediction models, significantly improving the accuracy and robustness of wind speed prediction. Attached Figure Description

[0023] Figure 1 The flowchart shows the multi-timescale fusion wind speed prediction method provided by this invention.

[0024] Figure 2 This is a schematic diagram of a specific UNet model with dual encoders.

[0025] Figure 3 This is a comparison chart of predicted and measured wind speeds. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0027] like Figure 1 As shown, a multi-timescale fusion wind speed prediction method based on a dual-encoder UNet model includes the following steps:

[0028] S1. Constructing a Dual Encoder: The dual encoder mainly consists of two modules: a predictive wind speed encoder and a historical wind speed encoder. Each encoder contains a multi-layer feature extraction module, used to extract multi-layer feature information from the input data. For example... Figure 2As shown, the predictive wind speed encoder processes predicted wind speed data, with input being time-series data of predicted wind speed, typically from numerical weather prediction models (such as data from the European Centre for Meteorological Research). The historical wind speed encoder processes historical wind speed data, with input being time-series data of historical wind speed, typically from observation data from weather stations or wind farms. Its structure is the same as the predictive wind speed encoder, also containing multiple double-layer convolutional modules and a temporal attention mechanism module. Both encoders consist of multiple double-layer convolutional modules, each containing two convolutional layers with a kernel size of 3 and a stride of 1. Each convolutional layer is followed by a ReLU activation function and layer normalization to extract local features. Simultaneously, a temporal attention mechanism module is introduced, which captures the temporal dependencies in the time series by calculating attention weights between different time steps. The temporal attention mechanism consists of three convolutional layers: query (Q), key (K), and value (V), calculated using the following formula:

[0029] ;

[0030] In the formula, d k is the dimension of the key vector.

[0031] The dual-layer convolution module consists of two convolutional layers, each followed by a ReLU activation function and layer normalization, used to extract local features from the input data. The temporal attention mechanism module consists of three convolutional layers: query, key, and value. It captures temporal dependencies by calculating attention weights between different time steps in the time series.

[0032] S2. Encoder Feature Fusion: After feature extraction by dual encoders, the feature fusion between the predictive wind speed encoder and the historical wind speed encoder is achieved through a cross-attention mechanism, which enhances the model's ability to perceive wind speed changes.

[0033] like Figure 2 As shown, the cross-attention mechanism employs a multi-head attention layer, where the features of the predicted wind speed encoder serve as the query (Q), and the features of the historical wind speed encoder serve as the key (K) and value (V). The calculation formula is the same as in step S1. Through the cross-attention mechanism, the features of predicted and historical wind speeds can be dynamically fused, enhancing the model's ability to perceive wind speed changes. The fused features are added to the original predicted wind speed features via residual connections, preserving the original information while enhancing the feature expressive power, resulting in the final fused features.

[0034] S3. Construct a multi-timescale decoder: The multi-timescale decoder is used to decode the generated fused features and generate wind speed prediction results at different time scales. For example... Figure 2As shown, the decoder consists of multiple upsampling modules and a two-layer convolutional module. Each upsampling module progressively increases the temporal resolution of the feature map through deconvolution operations, and further extracts features through the two-layer convolutional module. Each layer of the decoder receives features from the corresponding encoder through skip connections, fusing the encoder and decoder features to ensure information integrity. The decoder outputs wind speed predictions at different time scales at each layer. Through the multi-time scale design, it can capture the changing trend of wind speed at different time scales.

[0035] The multi-timescale decoder contains three upsampling modules. Each upsampling module gradually increases the temporal resolution of the feature map through deconvolution operations and further extracts features through a double-layer convolution module.

[0036] S4. Multi-timescale prediction result fusion: After the decoder generates multi-timescale prediction results, the multi-scale fusion module fuses the prediction results from different timescales to generate the final wind speed prediction result. For example... Figure 2 As shown, the multi-scale fusion module linearly interpolates the prediction results from different time scales to unify them to the same time resolution, and then fuses them through a convolutional layer. The final output is a comprehensive wind speed prediction result that simultaneously considers wind speed trends across different time scales.

[0037] S5. Construct a UNet model for wind speed prediction: (e.g.) Figure 2 As shown, a complete UNet model for wind speed prediction is constructed based on a dual encoder, a feature fusion module, a multi-timescale decoder, and a multi-scale fusion module. A classic U-shaped network structure is adopted, including a dual encoder, a feature fusion module, a multi-timescale decoder, and a multi-scale fusion module. Features of the predicted wind speed and historical wind speed are extracted by the dual encoder, and feature fusion is performed through a cross-attention mechanism. The decoder generates multi-timescale prediction results, and finally, the multi-scale fusion module outputs a comprehensive wind speed prediction result.

[0038] Training Data Collection: To train the UNet wind speed prediction model, historical and predicted wind speed data must first be collected. Historical wind speed data is obtained from weather stations or wind farms, and is typically stored in time series format. Predicted wind speed data is obtained from weather forecasting models (such as data from the European Centre for Weather Research), and the temporal resolution of the predicted data is consistent with that of the historical data.

[0039] Training Database Construction: A training database is constructed based on the collected data for model training. Historical and predicted wind speed data are preprocessed, including data cleaning, missing value imputation, and normalization. Multiple training samples are generated using a sliding window approach, with a window size of 8 time steps and a step size of 2 time steps, increasing data diversity. Each sample contains wind speed data within a single time window, used as model input.

[0040] During the construction of the training database, multiple training samples are generated using a sliding window. The size and step size of the sliding window can be adjusted according to the prediction requirements. When the amount of data is small, a smaller time step can be selected to ensure that the amount of training samples is sufficient.

[0041] Model Training: After building the training database, the constructed UNet wind speed prediction model was trained using gradient descent. A custom loss function was used, which combines the mean bias and standard deviation bias.

[0042] ;

[0043] In the formula, , They are 1 and 0.5 respectively. For predicted values, This is the target value.

[0044] The AdamW optimizer is used, combined with a cosine annealing learning rate scheduler to dynamically adjust the learning rate, and mixed precision training is used. GradScaler is used to scale the gradients to avoid gradient overflow.

[0045] It can effectively measure the difference between predicted results and true values. The AdamW optimizer is used for model optimization, combined with a cosine annealing learning rate scheduler to dynamically adjust the learning rate, improving the model's convergence speed. To accelerate the training process, mixed-precision training is used, and gradients are scaled using GradScaler to avoid gradient overflow.

[0046] S6. Wind Speed ​​Prediction: After model training is complete, the wind speed prediction UNet model trained in step S8 is used to predict wind speed. Predicted wind speed data and historical wind speed data are input into the model to predict wind speed. The prediction results can be used in fields such as wind farm power prediction and weather forecasting.

[0047] This study investigates the implementation and advantages of a multi-timescale fusion wind speed prediction method based on a dual-encoder UNet model, using wind speed records from a 100-m height on an onshore anemometer. The anemometer's wind measurement period was from 16:00 on February 9, 2023 to 23:00 on February 27, 2024, with a recording step size of one hour and a total data length of 9216. The numerically predicted wind speed was generated using an ECMWF numerical model containing wind speed prediction data from four grid points at a 100-m height on the anemometer, with a time resolution of one hour, and the time period was the same as the wind measurement period.

[0048] The specific inputs for model training are the predicted wind speeds from four grid points every 8 hours and the historical wind speeds from the meteorological tower for the previous 32 hours. The target is the actual wind speed measured by the meteorological tower every 8 hours. The model training parameters are set as follows: the learning rate is set to 1×10. -4 The weight decay factor is set to 3×10. -5 The model was trained for a total of 50 epochs, with an early stopping tolerance value of 10 and a gradient clipping threshold of 1.0. After training, the model's final error converged to 0.0250. Figure 3 As shown, the comparison between predicted and measured wind speeds over four 8-hour (32-hour) periods is presented. The results indicate that the multi-timescale fusion wind speed prediction method has high wind speed prediction accuracy, with an average error of 0.5 m / s.

[0049] The technical solution of the present invention has been described in conjunction with the specific experimental procedures shown in the accompanying drawings. However, the scope of protection of the present invention is not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions resulting from such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A multi-timescale fusion wind speed prediction method based on a dual-encoder UNet model, characterized in that, Includes the following steps: S1. Constructing a dual encoder: The dual encoder mainly consists of two modules: a numerical prediction wind speed encoder and a historical wind speed encoder. Each encoder contains a multi-layer feature extraction module. S2, Encoder Feature Fusion: Feature fusion between the numerical prediction wind speed encoder and the historical wind speed encoder is achieved through a cross-attention mechanism; S3. Construct a multi-timescale decoder: decode the generated fused features; S4. Multi-timescale prediction result fusion: The multi-scale fusion module fuses prediction results from different time scales to generate the final wind speed prediction result. S5. Based on steps S1-S4, construct the UNet model for wind speed prediction, and collect training data, construct the training database, and train the model. S6. Wind speed prediction: Use the trained UNet wind speed prediction model to predict wind speed. In step S1, both encoders consist of multiple two-layer convolutional modules. Each two-layer convolutional module consists of two convolutional layers, with each convolutional layer followed by a ReLU activation function and layer normalization. In step S1, a time attention mechanism module is introduced, consisting of three convolutional layers: query, key, and value. By calculating the attention weights between different time steps in the time series, the time dependency is captured. In step S2, the cross-attention mechanism adopts a multi-head attention mechanism, in which the features of the numerical prediction wind speed encoder are used as queries, and the features of the historical wind speed encoder are used as keys and values. The fusion between the features of the two encoders is achieved by calculating attention weights. In step S3, the multi-timescale decoder includes three upsampling modules. Each upsampling module gradually increases the temporal resolution of the feature map through deconvolution operation and further extracts features through a double-layer convolution module. In step S4, the multi-scale fusion module performs linear interpolation on the prediction results at different time scales to unify them to the same time resolution, and then fuses them through a convolutional layer to finally output a comprehensive wind speed prediction result. In step S5, the training data collection first requires collecting historical wind speed data and numerical prediction wind speed data. The predicted wind speed data comes from the numerical weather prediction model, and the historical wind speed data comes from the observation data of the weather station or wind farm. In step S5, during the construction of the training database, multiple training samples are generated using a sliding window. The size and step size of the sliding window can be adjusted according to the prediction requirements. When the amount of data is small, a smaller time step can be selected to ensure that the amount of training samples is sufficient.

Images