A batch wind speed interpolation method based on 3D SCA-Transformer

Abstract

The present invention discloses a batch wind speed interpolation method based on three-dimensional SCA-Transformer. The method is based on a hybrid three-dimensional moving window attention mechanism and a channel attention mechanism to capture the spatial correlation and variable correlation of large-area meteorological elements. The method adopts a three-dimensional SCA-Transformer architecture. Unlike the point-to-point interpolation method of traditional reanalysis data, the method of the present invention embeds meteorological information of a larger area, thereby better characterizing the impact of meteorological conditions at different spatial scales on the wind speed of wind measuring points. The present invention has the excellent ability to accurately interpolate missing data and batch interpolate different wind measuring equipment and wind measuring heights, providing more accurate resource assessment for the development and utilization of wind energy resources.

Description

Technical Field

[0001] The present invention belongs to the intersection field of machine learning and wind power generation, and specifically relates to a batch wind measurement data interpolation method based on a novel three-dimensional SCA-Transformer. Background Art

[0002] As a clean and renewable energy source, wind energy plays an increasingly important role in the global energy structure. However, the development and utilization of wind energy resources are restricted by many factors, one of which is the accuracy and completeness of wind measurement equipment data. In practical applications, wind measurement equipment often loses data due to equipment failure, improper maintenance or environmental factors, which brings great trouble to the evaluation and utilization of wind energy resources.

[0003] Traditional wind speed interpolation methods are mostly based on point-to-point data interpolation techniques, such as linear interpolation and polynomial interpolation. Although these methods are simple and easy to use, they often ignore the spatial correlation and variable correlation between meteorological elements, and cannot handle complex meteorological conditions, resulting in inaccurate interpolation results. In recent years, with the rapid development of deep learning technology, more and more researchers have begun to explore the use of deep learning models for wind speed interpolation. Among them, the Transformer-based model has attracted much attention due to its excellent sequence modeling capabilities and self-attention mechanism. However, most of the existing Transformer-based wind speed interpolation methods only focus on one-dimensional time series data and cannot effectively capture the spatial correlation of meteorological elements. Summary of the invention

[0004] The purpose of the present invention is to solve the problems existing in the existing technology and provide a batch wind speed interpolation and method based on a novel three-dimensional SCA-Transformer to cope with the wind speed interpolation work of wind measuring equipment in complex meteorological environments.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A batch wind speed interpolation method based on three-dimensional SCA-Transformer, characterized by:

[0007] Step S1, collecting wind speed measured data and mesoscale meteorological data within a target time range;

[0008] Step S2: preprocess the data obtained in step S1, build a database and a verification data set required for model training, and generate a data loader from the database;

[0009] Step S3, constructing an SCA module that couples the channel attention mechanism and the three-dimensional moving window multi-head self-attention mechanism, wherein the SCA module consists of a pooling layer, a linear layer, a channel attention layer, a moving window self-attention layer, a multi-layer feedforward neural network layer, and an additive normalization layer from bottom to top;

[0010] Step S4, building a Transformer framework based on the SCA module in step S3, that is, a SCA-Transformer model, wherein the SCA-Transformer model includes a data embedding layer, an encoding module and an output layer, wherein the encoding module includes multiple layers of SCA modules and a downsampling layer, and the output layer includes a normalization layer, a 3D global pooling layer and a linear layer;

[0011] Step S5: Based on the training data set generated in step S2, the SCA-Transformer model is trained using an optimizer;

[0012] Step S6, verifying the accuracy of the SCA-Transformer model trained in step S5 in the validation data set;

[0013] Step S7: interpolate the missing data in the wind speed measured data collected in step S1 using the SCA-Transformer model verified in step S6.

[0014] On the basis of adopting the above technical solutions, the present invention may also adopt the following further technical solutions, or use these further technical solutions in combination:

[0015] In step S1, the measured wind speed data include wind speed and wind direction data of different height layers within a target time range of multiple target wind towers; the mesoscale meteorological data include ERA5 mesoscale meteorological data within the target time range, which includes multiple pressure layers and main physical variables affecting the characteristics of wind profiles near the ground. The ERA5 is the fifth-generation global atmospheric reanalysis data set of the European Centre for Medium-Range Weather Forecasts, with a spatial resolution of 0.25° and a temporal resolution of every hour.

[0016] In step S1, the main physical variables are the horizontal wind speed vector, air pressure, temperature and potential height of different pressure layers, the horizontal velocity component at a height of 10m, the temperature at a height of 2m, and the surface roughness length of the near-surface layer.

[0017] In step S2, first, the missing values ​​and unreasonable data of the wind tower measured data are eliminated; secondly, the remaining wind tower measured data are converted into horizontal wind speed vector data using wind direction and wind speed data; then, the training data set and the validation data set are divided; finally, a training database is constructed with ERA5 mesoscale meteorological data as input data and wind tower data as target data, and the data loader required for model training is generated.

[0018] In step S2, the batch size of the data loader is preferably set to 2, and can also be adjusted according to specific data conditions.

[0019] In step S3, the operation logic of the SCA module is as follows:

[0020] 1). The SCA module input enters the channel attention layer and the moving window self-attention layer SW-MSA after pooling and linear processing, calculates the channel attention and spatial self-attention, and captures the variable correlation and spatial correlation of the ERA5 input data;

[0021] 2). The output of the channel attention layer is reduced put Then add it to the output of the moving window self-attention layer to get the mixed attention;

[0022] 3). Mixed attention ensures that the output of the SCA module is a coded tensor with spatial correlation and variable correlation through multiple layers of feedforward neural network layers and additive normalization layers.

[0023] In step S4, the SCA-Transformer model operation logic is as follows:

[0024] 1). The input data is convolved through the data embedding layer to obtain the encoding module input with a channel dimension of C;

[0025] 2). The encoding module composed of multiple layers of SCA modules and downsampling layers performs information encoding, that is, the input is first encoded by the SCA module, then the resolution is reduced by the downsampling layer, and the correlation information is encoded again by the SCA module, and the "SCA module-downsampling layer-SCA module" operation is cyclically performed. To ensure information circulation, the output of the first-layer SCA module is proportionally connected to the SCA modules of the remaining layers for residual error;

[0026] 3). The output of the encoding module passes through the normalization layer, 3D global pooling layer and linear layer to obtain the output of the SCA-Transformer model.

[0027] In step S5, before training the SCA-Transformer model, the SCA-Transformer model is initialized first; and a masked loss function is set, that is, the model loss at the target missing point is not calculated; and the number of model cycles, that is, the number of epochs, is set; when training the model, the model error under each model cycle is recorded. If the model error converges, the training is valid, and the trained SCA-Transformer model is saved.

[0028] In step S5, the optimizer is an Adam (Adaptive Moment Estimation) optimizer.

[0029] In step S5, the loss function is set to the mean absolute error function.

[0030] In step S7, the regional mesoscale data at the time corresponding to the missing wind speed value is used as input to the SCA-transformer model that has been trained and verified in step S6, and its output is denormalized and used as the interpolation value of the missing wind speed.

[0031] The batch wind speed interpolation method based on the new three-dimensional SCA-Transformer proposed in this paper combines the three-dimensional moving window attention mechanism and the channel attention mechanism, which can simultaneously capture the spatial correlation and variable correlation of large-area meteorological elements. Through the three-dimensional SCA-Transformer architecture, meteorological information of a larger area is embedded, and the impact of meteorological conditions at different spatial scales on the wind speed of the wind measuring point is better characterized. It has strong adaptability to complex atmospheric environments, and the model method has higher accuracy and practical efficiency.

[0032] The batch wind speed interpolation method based on the novel three-dimensional SCA-Transformer established in the present invention is an innovation and improvement of the traditional numerical prediction method. It can interpolate the wind speed data of multiple wind measuring devices and multiple height layers at one time, and can realize the accurate interpolation of the wind speed of batch wind measuring devices, thereby solving the problem of data loss caused by equipment failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of the method running steps.

[0034] Figure 2 This is a schematic diagram of the SCA module architecture.

[0035] Figure 3 The following is a schematic diagram of the SCA-Transformer architecture.

[0036] Figure 4 This is a schematic diagram of the interpolation effect of wind measurement data. DETAILED DESCRIPTION

[0037] The present invention will be further described and illustrated below in conjunction with the accompanying drawings and specific implementation methods.

[0038] This paper mainly constructs a batch wind measurement data interpolation method based on a new three-dimensional SCA-Transformer, which is used to interpolate missing wind measurement data in wind power projects and help the preliminary assessment of wind energy resources in the project. The overall method steps are as follows Figure 1 The specific steps are as follows:

[0039] Step S1, collect wind speed measured data and mesoscale meteorological data within the target time range: First, collect wind speed and wind direction data from multiple wind measurement data sources at different altitudes, with dimensions (N, m, t), where N represents different wind measurement data sources, m represents different altitudes, and t is the time series. Secondly, with the target site as the center, download the regional open source ERA5 mesoscale meteorological data with a grid resolution of , the number of grids is , that is, the coverage area is approximately The time resolution is 1h. Specifically, it includes horizontal wind speed components (u, v), air pressure (p), temperature (t) and geopotential height (geo) of different pressure layers, as well as horizontal speed components (u10, v10) at 10m height near the surface layer, temperature (t2) at 2m, surface roughness length (z 0 ). Therefore, the ERA5 data dimensions of different pressure layers are (Z, H, W, n, t), and the data dimensions of the surface layer are (H, W, n, t), where Z is the number of vertical pressure layers, 7 layers near the surface; H and W are the horizontal data widths, which are equal to 40; and n is the number of variables, that is, n=5 (pressure layer data) or 4 (surface layer data).

[0040] Step S2: preprocess the data obtained in step S1, build the training data set and verification data set required for model training, and generate a data loader from the database: first, remove the missing values ​​and unreasonable data of the wind tower measured data; second, use the wind direction and wind speed data to convert the wind speed into the horizontal wind speed vector u , v ,Right now:

[0041] ,

[0042] ,

[0043] in ， They are wind speed and wind direction, respectively. Then, the training data set and the validation data set are divided into training data set and validation data set according to the division ratio of 8:2. Finally, Pytorch's Dataset and Dataloader libraries are used to construct a training database with ERA5 mesoscale meteorological data as input data (pressure layer data and surface layer data) and wind measurement data as target data, and generate the data loader required for model training with a step size of 2, denoted as B=2.

[0044] Step S3: construct an SCA module that couples the channel attention mechanism and the three-dimensional moving window multi-head self-attention mechanism. The SCA module consists of a pooling layer, a linear layer, a channel attention layer, a moving window self-attention layer, a multi-layer feedforward neural network layer, and an additive normalization layer from bottom to top. Its operation logic is as follows: Figure 2 As shown in the figure, the core SCA module scales the output of the channel attention layer and adds it to the output of the moving window self-attention layer to better capture the spatial correlation of variables and the correlation between variables. Specifically, the spatial dimension is eliminated by adopting the global maximum pooling layer, and the channel attention layer is used to capture the attention between channel layers. At the same time, spatial attention is captured by the linear layer and the moving window self-attention layer. The output of the channel attention layer is scaled and added to the output of the moving window self-attention layer, and then passed through the additive normalization layer, the multi-layer feedforward neural network layer and the additive normalization layer to obtain a mixed attention tensor containing spatial features and channel features. Its mixed attention calculation can be expressed as follows:

[0045] ,

[0046] Among them, Q, K, and V represent the Query, Key, and Value of multi-head attention respectively, and the superscript T represents the matrix transpose. d m is a multi-head dimension, Softmax and ReLu are softmax and relu activation functions respectively, Conv is a convolution operation, and GMP is a global maximum pooling operation. is the scaling factor, x For input.

[0047] Step S4: Build a Transformer framework based on the SCA module in step S3, i.e., a SCA-Transformer model. The SCA-Transformer model includes a data embedding layer, an encoding module, and an output layer. The encoding module includes multiple layers of SCA modules and a downsampling layer. The output layer includes a normalization layer, a 3D global pooling layer, and a linear layer. The operation logic is as follows: Figure 3As shown, the data embedding layer performs convolution processing on the ERA5 input data to obtain embedded data with the same channel dimension of C, which are (B, H, W, C) and (B, Z, H, W, C), respectively. They are spliced ​​in the vertical direction to obtain an embedding tensor with a dimension of (B, Z+1, H, W, C); secondly, the tensor passes through an encoding module consisting of a 4-layer SCA module, a downsampling layer, a 2-layer SCA module, a downsampling layer, and a 1-layer SCA module in turn to obtain an encoding tensor of set space and variable correlation. Among them, downsampling is a 3D average pooling operation with a window size of [2, 2, 2], that is, each time it passes through the pooling layer, the spatial resolution (dimension) of the tensor is reduced by 2 times. Secondly, the output of the first SCA module is respectively proportional to α 1 = 0.6 and α 2 =0.2 for residual connection with the second and third SCA modules in the form of output addition. Finally, the tensor continues to pass through the output layer consisting of a normalization layer, a 2D global average pooling layer, and a linear layer to obtain an SCA-Transformer output of dimension (B, N, m).

[0048] Step S5, based on the training data set generated in step S2, use the optimizer to train the SCA-Transformer model: Use the data loader generated in step S2 to train the SCA-Transformer constructed in step S4. First, load the input data and target data and initialize the SCA-Transformer model. Then, select the Adam optimizer and set the learning rate. Secondly, define the loss function as the MAE loss function and mask the missing positions of the target data. Finally, set the number of model cycles epoch, use a multi-core GPU to train the model, and print the model error under each epoch. When the error converges and the total number of cycles is completed, save the optimized SCA-Transformer model.

[0049] Step S6, verify the accuracy of the SCA-Transformer model trained in step S5 in the validation data set: Based on the SCA-Transformer trained in step S5, the ERA5 data in the validation set is used as input, the wind speed data is interpolated for the wind measurement data, and the model accuracy is verified by comparing with the measured wind data in the validation set.

[0050] Step S7: interpolate the missing data in the wind speed measured data collected in step S1 using the SCA-Transformer model verified in step S6. Example

[0051] In this embodiment, the wind speed record data of two wind towers on land are taken as the research object, and the specific implementation effect and advantages of a batch wind speed interpolation method based on a new three-dimensional SCA-Transformer are explored. The wind measurement period of the two wind towers is from 2017.01.21-10:00 to 2017.11.22-03:00, the recording step is every hour, and the total data length is 7314. The wind tower has a total of 6 wind measurement heights, namely 100m, 90m, 70m, 50m, 30m, and 10m.

[0052] First, download the ERA5 pressure layer reanalysis dataset for the wind measurement period, with a longitude range of 112.5°E~122.5°E, a latitude range of 36°N~46°N, and 9 vertical layers. At the same time, download the surface layer reanalysis dataset in the same range. Secondly, the wind measurement data of the two wind towers were cleaned to remove unreasonable data and data during long-term stagnation periods (periods when the wind measurement equipment was not working). After cleaning, the data integrity rates were 96.5% and 98.5%, respectively. The wind tower data was used as the target data, and the ERA5 data was used as the input data. A data loader with a batch size of 2 was generated, and the test set and validation set were divided into 8:2. Finally, the model was trained on 8 NVIDIA A100 graphics cards, with the training learning rate set to 1e-3 and the weight decay set to 3e-4. The epoch was set to 50, and the Adam optimizer was used.

[0053] like Figure 4 As shown in the figure, by comparing the wind speed predicted (interpolated) by the SCA-transformer model and the measured wind speed in the training set (training 1, training 2) and test set (test 1, test 2) of the two wind turbines, the results show that the batch wind speed interpolation method using the new SCA-transformer model has high accuracy, with a correlation coefficient above 0.9, an average error below 0.6m / s, and a root mean square error of about 0.1m / s. Therefore, this method can accurately realize the interpolation of wind speeds of different wind measuring equipment and different altitude layers in batches.

[0054] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A batch wind speed interpolation method based on three-dimensional SCA-Transformer, characterized by: Step S1, collecting wind speed measured data and mesoscale meteorological data within a target time range; Step S2: preprocess the data obtained in step S1, build a database and a verification data set required for model training, and generate a data loader from the database; Step S3, constructing an SCA module that couples the channel attention mechanism and the three-dimensional moving window multi-head self-attention mechanism, wherein the SCA module consists of a pooling layer, a linear layer, a channel attention layer, a moving window self-attention layer, a multi-layer feedforward neural network layer, and an additive normalization layer from bottom to top; Step S4, building a Transformer framework based on the SCA module in step S3, that is, a SCA-Transformer model, wherein the SCA-Transformer model includes a data embedding layer, an encoding module and an output layer, wherein the encoding module includes multiple layers of SCA modules and a downsampling layer, and the output layer includes a normalization layer, a 3D global pooling layer and a linear layer; Step S5: Based on the training data set generated in step S2, the SCA-Transformer model is trained using an optimizer; Step S6, verifying the accuracy of the SCA-Transformer model trained in step S5 in the validation data set; Step S7, using the SCA-Transformer model verified in step S6 to interpolate the missing data in the wind speed measured data collected in step S1; In step S3, the operation logic of the SCA module is as follows: 1) The SCA module input enters the channel attention layer and the moving window self-attention layer after pooling and linear processing, calculates the channel attention and spatial self-attention, and captures the variable correlation and spatial correlation of the ERA5 input data; 2) The output of the channel attention layer CA is scaled and added to the output of the moving window self-attention layer to obtain mixed attention; 3) Hybrid attention ensures that the output of the SCA module is a coded tensor with spatial correlation and variable correlation through multiple layers of feedforward neural network layers and additive normalization layers; In step S4, the SCA-Transformer model operation logic is as follows: 1) The input data is convolved through the data embedding layer to obtain the encoding module input with a channel dimension of C; 2) The encoding module composed of multiple layers of SCA modules and downsampling layers performs information encoding, that is, the input is first encoded by the SCA module, then the resolution is reduced by the downsampling layer, and then the information is encoded again by the SCA module, and the "SCA module-downsampling layer-SCA module" operation is cyclically performed. To ensure the circulation of information, the output of the first-layer SCA module is proportionally connected to the SCA modules of the remaining layers for residual error; 3). The output of the encoding module passes through the normalization layer, 3D global pooling layer and linear layer to obtain the output of the SCA-Transformer model.

2. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 1 is characterized in that: In step S1, the measured wind speed data include wind speed and wind direction data of different height layers within a target time range of multiple target wind towers; the mesoscale meteorological data include ERA5 mesoscale meteorological data within the target time range, which includes multiple pressure layers and main physical variables affecting the characteristics of wind profiles near the ground. The ERA5 is the fifth-generation global atmospheric reanalysis data set of the European Centre for Medium-Range Weather Forecasts, with a spatial resolution of 0.25° and a temporal resolution of every hour.

3. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 2 is characterized in that: In step S1, the main physical variables are the horizontal wind speed vector, air pressure, temperature and potential height of different pressure layers, the horizontal velocity component at a height of 10m, the temperature at a height of 2m, and the surface roughness length of the near-surface layer.

4. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 2 is characterized in that: In step S2, first, the missing values ​​and unreasonable data of the wind tower measured data are eliminated; secondly, the remaining wind tower measured data are converted into horizontal wind speed vector data using wind direction and wind speed data; then, the training data set and the validation data set are divided; finally, a training database is constructed with ERA5 mesoscale meteorological data as input data and wind tower data as target data, and the data loader required for model training is generated.

5. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 1 is characterized in that: In step S5, before training the SCA-Transformer model, the SCA-Transformer model is initialized first; and a masked loss function is set, that is, the model loss at the target missing point is not calculated; and the number of model cycles, that is, the number of epochs, is set; when training the model, the model error under each model cycle is recorded. If the model error converges, the training is valid, and the trained SCA-Transformer model is saved.

6. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 5 is characterized in that: In step S5, the optimizer is the Adam optimizer.

7. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 5 is characterized in that: In step S5, the loss function is set to the mean absolute error (MAE) function.

8. The batch wind speed interpolation method based on three-dimensional SCA-Transformer according to claim 1 is characterized in that: In step S7, the regional mesoscale data at the time corresponding to the missing wind speed value is used as input to the SCA-transformer model that has been trained and verified in step S6, and its output is denormalized and used as the interpolation value of the missing wind speed.

Images