Multi-modal feature fusion model construction method and multi-step wind power prediction method

Abstract

The invention discloses a multi-modal feature fusion model construction method and a multi-step wind power prediction method, and belongs to the field of wind power prediction.The multi-modal feature fusion model construction method comprises the steps that firstly, two data conversion channels are added on the basis of a Grubrum angle field to construct a multi-channel Grubrum angle field, and wind power data are converted into RGB color images with color information; secondly, the time text features are embedded into the data features and the image features respectively, and the time sequence expression ability of the data features and the image features is improved; fusing the data features and the image features by using a cross-modal fusion method, and adjusting dynamic weights of the data features and the image features during fusion by using an adaptive decision mode to obtain fusion features containing different modal information; and finally, extracting association information between the fusion features and the wind power by using an encoder of the Transform, and decoding the input data into the wind power to be predicted by a decoder according to the association information and the prior data. A simulation experiment shows that the method effectively improves the prediction precision of the wind power.

Description

Technical Field

[0001] The present invention relates to the field of wind power prediction, and in particular to a multi-modal feature fusion model construction method and a multi-step wind power prediction method. Background Art

[0002] With the rapid development of global industry, the contradiction between the surge in electricity demand and the reduction in traditional energy reserves is becoming increasingly severe, and it is imperative to promote the transformation of energy structure. Many countries have introduced relevant industrial support policies and formulated green development strategies to promote the rapid development of renewable energy. Among many renewable energy sources, wind energy has become one of the most promising new energy sources with its advantages of environmental protection, cleanliness, wide distribution, and significant cost-effectiveness. However, the high volatility and low controllability of wind energy make it difficult to absorb wind power when it is connected to the grid, and its utilization rate is limited. Therefore, accurate wind power prediction is of great significance to alleviate the pressure of grid dispatching and improve the comprehensive utilization rate of wind energy.

[0003] In order to achieve accurate wind power forecasting, some studies have constructed physical models for forecasting through meteorological factors and geographical parameters. This method has strong interpretability, but it has strict requirements on modeling parameters and calculations, and relies on numerical weather forecast (NWP) information. There are also studies that use historical data to mine the nonlinear mapping relationship between wind power and historical feature sequences to construct statistical models for forecasting. The most representative ones are the autoregressive moving average model (ARMA) and the autoregressive integrated moving average model (ARIMA). This type of model has a simple structure and is easy to implement, but it is difficult to accurately predict wind power. In order to improve the prediction accuracy, some researchers have turned to wind power forecasting based on deep learning models. Commonly used models include convolutional neural networks (CNN), long short-term memory networks (LSTM), gated recurrent units (GRU) and temporal convolutional networks (TCN). Considering the advantages and disadvantages of different models, most existing studies use integrated methods to predict wind power in order to obtain more accurate results. Abou Houran M et al. proposed a photovoltaic / wind power forecasting method that uses the Brown Bear optimization algorithm to optimize the key hyperparameters of LSTM-CNN, achieving high prediction accuracy. Liu S et al. proposed a method to predict the input sequence after Savitzky-Golay (SG) filter denoising through TCN-LSTM, and achieved high prediction accuracy. HuM et al. proposed a variational mode decomposition method based on sparse prior knowledge to reduce the volatility of original wind power, and then used LASSO to screen key features and used BiGRU to predict short-term wind power, which improved the prediction accuracy of wind power.

[0004] With the development of deep learning models, single-step wind power prediction technology has become relatively mature, but power grid dispatching often requires the support of multi-step wind power prediction. Existing multi-step prediction methods include direct prediction, recursive prediction, and multi-input multi-output prediction (MIMO). Among them, the direct prediction method models each time step, and the model complexity increases with the number of steps; the recursive prediction uses the output of the previous time step as the input of the next time step, resulting in cumulative prediction errors; MIMO predicts multiple time steps at the same time, simplifies the model and avoids error accumulation, and is currently the main multi-step wind power prediction method. Jiang Z et al. used quadratic decomposition and multi-label specific XGBoost methods for feature selection, and used CGR (Convolutional gated recurrent unit network) to achieve multi-step prediction, achieving good prediction performance. Qu Z et al. proposed a dual-fusion spatiotemporal graph deep neural network model based on a superimposed multi-integration framework, which superimposed bagging, long short-term memory network and random forest, effectively improving the prediction accuracy. Saeed A et al. proposed a gated multi-scale convolutional sequence model (GMSCSM), which uses the recursive ability of LSTM and the parallel input ability of CNN to improve the prediction accuracy while reducing the prediction time. Yu C et al. proposed a multi-step wind speed prediction model that combines the attention sampling interaction module and the multi-resolution interaction module, which can simultaneously analyze and fuse wind speed data of different frequencies, effectively improving the prediction accuracy of the model.

[0005] Although the above research has improved the accuracy of multi-step wind power prediction, there are still some problems: most of the time text labels are discarded during data preprocessing, and only the data input order and masking technology are used to reflect the time series information of the data, which limits the model's ability to capture the long-term seasonal and periodic changes of wind power sequences. When converting data into images, most of the grayscale image conversion methods are used. In essence, it is still a data matrix, and only the brightness of the pixels is used to reflect the features, which cannot take advantage of the richer image feature information. The decoder of the Transformer model relies too much on the context vector generated by the encoder, ignoring the influence of prior data on the decoder. Therefore, it is of great significance to further study the multimodal feature fusion for wind power prediction. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-modal feature fusion model construction method and a multi-step wind power prediction method to solve the problem that in multi-step wind power prediction, the strong volatility and weak long-term memory of wind power make it difficult for traditional models to achieve accurate prediction.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] The present invention provides a method for constructing a multimodal feature fusion model, wherein the multimodal feature fusion model is used for multi-step wind power prediction, and comprises the following steps:

[0009] S1. Construct a multi-channel Gram angle field, convert wind power data into RGB color images with color information, and obtain wind power image features;

[0010] S2, converting the time text into time data features, and then embedding them into the meteorological data features and the wind power image features described in step S1, and extracting the hidden data features and image features;

[0011] S3, using a cross-modal fusion method to fuse the data features extracted in step S2 with the image features, and using an adaptive decision-making method to adjust the dynamic weights when the data features and the image features are fused to obtain fused features containing information of different modalities;

[0012] S4, using the Transformer encoder to extract the correlation information between the fusion features described in step S3 and the wind power, and outputting the context vector to the decoder through a multi-head attention mechanism;

[0013] S5. The decoder decodes the input data into wind power to be predicted according to the context vector and the priori data described in step S4.

[0014] Furthermore, the S1 specifically includes: in the S1, the multi-channel Gram angle field is constructed by converting the grayscale image I generated by the Gram angle sum field and the grayscale image II generated by the Gram angle difference field into a grayscale image III in a randomly weighted manner, and assigning the grayscale image I, the grayscale image II, and the grayscale image III to the red channel, the green channel, and the blue channel, respectively, to obtain an RGB color image.

[0015] Furthermore, the process of constructing the grayscale image III is as follows: IGAF i,j =R j GASF i,j +(1-R j )GADF i,j

[0016] In the formula, R j Represents a random number in the interval [0,1], IGAF i,j Represents grayscale image III, GASF i,j Represents a grayscale image I, GADF i,j represents grayscale image II;

[0017] The resulting RGB color image is: I=(I R ,I G ,I B )=(GASF,GADF,IGAF)

[0018] Where I represents the final color image, I R ,I G and I B Represent the red channel, green channel and blue channel in the color image respectively.

[0019] Furthermore, S2 specifically includes: after the time data features are embedded into the meteorological data features, 1DCNN is used to capture the hidden features of the data; after the time data features are embedded into the wind power image features, the convolution layer in 2DCNN is used to extract the image features, and the output of the convolution layer is downsampled through a pooling operation.

[0020] Furthermore, in S2, the method of converting the time text into the time data feature is as follows:

[0021] (1) To ensure the uniqueness of the year, use One-Hot encoding: n year (i)=year(i)-year min i=0,1,2...,l year binary =concat(b i )

[0022] In the formula, l and i represent the length of the time text sequence and the position of the point to be processed, respectively, and j represents n i The j-th binary bit of mod and concat are the floor operator, modulus operator and concatenation operator respectively;

[0023] (2) In order to ensure the uniqueness of the month and day in the year, position codes are used:

[0024] In the formula, month_day(i) and year_day(i) represent the total number of days in the month and year of the point to be processed respectively;

[0025] (3) In view of the periodicity of the moment in the year, a cyclic coding method is used to represent it:

[0026] In the formula, hour(i) and m(i) represent the hour and minute of the point to be processed respectively;

[0027] (4) Perform dimensional transformation so that the time data feature is consistent with the other feature dimensions, as shown in the following formula:

[0028] In the formula, p and q represent the input feature dimension and output feature dimension of time data respectively, σ is the activation function, ω qp With b q denote weight and bias respectively.

[0029] Furthermore, in S3, the implementation process of cross-modal fusion and adaptive decision fusion is as follows:

[0030] A multi-head cross-attention mechanism is constructed with image features as query vectors and data features as key vectors and value vectors to obtain fusion features with image features as the main body;

[0031] A multi-head cross-attention mechanism is constructed with data features as query vectors, image features as key vectors and value vectors to obtain fusion features with data features as the main body;

[0032] Dynamically generate the weights of image-based fusion features and data-based fusion features to achieve adaptive feature fusion.

[0033] Furthermore, in S5, the input of the decoder includes the data to be predicted and prior data, wherein the prior data is a part of the time text, meteorological data and wind power data that have been used by the encoder in step S4.

[0034] Furthermore, the data to be predicted and the prior data are sequentially subjected to temporal feature embedding, masked attention mechanism, multi-head attention mechanism and cross attention mechanism in the decoder. The cross attention mechanism combines the context vector output by the decoder with the context vector output by the encoder, passes through a fully connected layer, and then outputs a prediction result.

[0035] The present invention also provides a multi-step wind power prediction method, which is based on a model obtained by the multi-modal feature fusion model construction method described above. The multi-step wind power prediction method comprises the following steps:

[0036] (1) Data collection and processing

[0037] Step 1: Collect historical data of wind farms to obtain wind power data sets;

[0038] Step 2: Filter the features in the data set that are highly correlated with wind power as the input features of the model;

[0039] Step 3: Divide the data set into two parts: training set and test set;

[0040] (2) Model training

[0041] Step 4: Use the training set data to construct the encoder input of the model, then select a part of the encoder input data as prior data, and combine these prior data with the data to be predicted as the decoder input;

[0042] Step 5: Calculate the prediction error and save the model parameters with the smallest error as the optimal model parameters;

[0043] (3) Model testing

[0044] Step 6: Use the test set data to build the encoder input of the model, then select a part of the encoder input data as prior data, and combine these prior data with the data to be predicted as the decoder input;

[0045] Step 7: Load the optimal model parameters to perform multi-step wind power prediction and obtain the prediction results.

[0046] By adopting the above technical solution, the present invention can achieve the following beneficial effects:

[0047] 1. The wind power map obtained by multi-channel Gram angle field conversion in the present invention has more characteristic information than the wind power map obtained by traditional Gram angle field conversion, which allows the model to observe data from a higher dimension, can more effectively reduce the impact of wind power volatility, and greatly improve the prediction accuracy.

[0048] 2) MMFformer effectively integrates time text, meteorological data and wind power images through multimodal feature fusion technology, which enhances the expressiveness of input features and effectively improves the prediction accuracy of the model. Using time text labels as features can effectively enhance the temporal nature of data and improve the model's prediction ability for long time series data.

[0049] 3) Adding prior data to the decoder enhances the interaction between the encoder and the decoder, effectively reducing the impact of weak long-term memory on wind power prediction. Prior data can provide prior knowledge for the decoder, enhance the decoding ability of the decoder, and improve the prediction accuracy of the model in long sequence prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is the overall framework diagram of the MMFformer model in the embodiment of the present invention;

[0051] Figure 2 It is an overall framework diagram of multimodal feature fusion in an embodiment of the present invention;

[0052] Figure 3 A schematic diagram of injecting prior data into a decoder according to an embodiment of the present invention;

[0053] Figure 4 Flow chart of a multi-step wind power prediction method according to an embodiment of the present invention;

[0054] Figure 5 : This is the Spearman correlation heat map of the data set 1 used in the embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] In multi-step wind power prediction, the strong volatility and weak long-term memory of wind power make it difficult for traditional models to achieve accurate prediction. In response to the above problems, this application proposes a multi-modal feature fusion model construction method and a multi-step wind power prediction method. The multi-modal feature fusion model construction method specifically includes the following steps:

[0057] S1. Construct a multi-channel Gram angle field, convert wind power data into RGB color images with color information, and obtain wind power image features;

[0058] S2, converting the time text into time data features, and then embedding them into the meteorological data features and the wind power image features described in step S1, and extracting the hidden data features and image features;

[0059] S3, using a cross-modal fusion method to fuse the data features extracted in step S2 with the image features, and using an adaptive decision-making method to adjust the dynamic weights when the data features and the image features are fused to obtain fused features containing information of different modalities;

[0060] S4, using the Transformer encoder to extract the correlation information between the fusion features described in step S3 and the wind power, and outputting the context vector to the decoder through a multi-head attention mechanism;

[0061] S5. The decoder decodes the input data into wind power to be predicted according to the context vector and the priori data described in step S4.

[0062] In this embodiment, the multimodal feature fusion model is defined as an MMFformer model. The overall framework of the MMFformer model is as follows: Figure 1As shown. The MMFformer model is mainly composed of multi-channel Gram angular field, multimodal feature fusion, multi-head attention mechanism, temporal feature embedding, mask attention mechanism and cross attention mechanism. Among them, the multi-head attention mechanism, mask attention mechanism and cross attention mechanism are classic modules in Transformer, and this application will not go into details. The following is a detailed description of the technologies and specific implementation measures used in the multi-channel Gram angular field, multimodal feature fusion and temporal feature embedding parts.

[0063] In step S1, the specific process of constructing the multi-channel Gram angle field is as follows:

[0064] Gram Angular Field is a method for converting time series data into image features. It can be divided into Gram Angular Summation Field (GASF) and Gram Angular Difference Field (GADF) according to the different trigonometric functions used. GASF calculates the sum of the corresponding angles at different time points so that each element in the matrix contains the cumulative information of two time points. The generated image can reflect the cumulative relationship between time series and help capture global features, as shown in formula (2); GADF calculates the difference of the corresponding angles at different time points so that each element in the matrix reflects the difference between two time points. The generated image can reflect the difference relationship between time series and help capture local features, as shown in formula (3):

[0065] In the formula, yes The angle in the polar coordinate system; i, j represent the row index and column index of the matrix respectively.

[0066] The image generated by GAF conversion is the result of mapping the size of matrix element values, and is essentially a grayscale image. Color images usually consist of three channels: red (R), green (G), and blue (B). Each pixel contains the characteristics of the pixels corresponding to the three channels. On this basis, the three grayscale images are assigned to the three R, G, and B channels to reconstruct the color image. Considering that the grayscale images converted by GASF and GADF emphasize the global features and local features of the original data respectively, and ignore the comprehensive consideration of the two, this application introduces a random weighted method to construct a third grayscale image, that is: the grayscale image I generated by the Gram angle sum field and the grayscale image II generated by the Gram angle difference field are converted into a grayscale image III by a random weighted method, and the grayscale image I, grayscale image II, and grayscale image III are assigned to the red channel, green channel, and blue channel, respectively, to obtain an RGB color image. The conversion process is shown in formula (4): IGAF i,j =R j GASF i,j +(1-R j )GADF i,j (4)

[0067] In the formula, R j Represents a random number in the interval [0,1], IGAF i,j Represents grayscale image III, GASF i,j Represents a grayscale image I, GADF i,j Represents a grayscale image II.

[0068] The final color image is shown in formula (5): I=(I R ,I G ,I B )=(GASF,GADF,IGAF) (5)

[0069] Where I represents the final color image, I R ,I G and I B Represent the red channel, green channel and blue channel in the color image respectively.

[0070] The multimodal feature fusion process is as follows:

[0071] The introduction of wind power image features and temporal text features improves the robustness of the model, but the differences between different modal features make the model unable to directly process them. To address this problem, this application proposes a multimodal feature fusion module, the overall framework of which is as follows: Figure 2 shown.

[0072] Depend on Figure 2 It can be seen that the proposed module is mainly composed of temporal feature embedding, one-dimensional convolutional network (1DCNN), two-dimensional convolutional network (2DCNN), cross-modal attention mechanism and adaptive decision fusion. Their specific implementation process is described in detail below.

[0073] In step S2, the temporal feature embedding implementation process is as follows:

[0074] To address the problem that time text features cannot be directly used by the model, this application adopts a method of converting time text into time data features. The main steps are as follows:

[0075] 1) To ensure the uniqueness of the year, use One-Hot encoding: n year (i)=year(i)-year min i=0,1,2...,l (6) year binary =concat(b i ) (9)

[0076] In the formula, l and i represent the length of the time text sequence and the position of the point to be processed, respectively, and j represents n i The j-th binary bit of . mod and concat are the floor operator, modulus operator, and concatenation operator, respectively.

[0077] 2) The uniqueness of the month and day in the year is expressed using positioning codes:

[0078] Where month_day(i) and year_day(i) represent the total number of days in the month and year of the point to be processed, respectively.

[0079] 3) In view of the periodicity of the time in the year, a cyclic coding method is used to represent it, as shown in formula (12):

[0080] In the formula, hour(i) and m(i) represent the hour and minute of the point to be processed respectively.

[0081] 4) Perform dimension transformation to make the time data feature consistent with the other feature dimensions, as shown in formula (13):

[0082] In the formula, p and q represent the input feature dimension and output feature dimension of time data respectively, σ is the activation function, ω qp With b q Respectively represent weight and bias.

[0083] The implementation process of 1DCNN and 2DCNN is as follows:

[0084] 1) Use 1DCNN to capture the hidden features of the data.

[0085] Where I and K represent the input data and one-dimensional convolution kernel respectively, M is the length of the convolution kernel, and i represents the index of the output feature. P, N, and Y represent the input feature map, pooling window size, and output feature map respectively.

[0086] 2) Use the convolutional layer in 2DCNN to extract image features.

[0087] Among them, f and g represent the input image and the two-dimensional convolution kernel respectively, m and n represent the width and height of the convolution kernel respectively, and (x, y) is the coordinate of a pixel point in the output feature.

[0088] The output of the convolutional layer is downsampled through the pooling operation, which reduces the network parameters and computational complexity and improves the computational efficiency of the model.

[0089] Among them, Z represents the input feature map, k is the size of the pooling window, and X is the output feature map.

[0090] The implementation process of cross-modal fusion and adaptive decision fusion is as follows:

[0091] 1) A multi-head cross-attention mechanism is constructed with image features as query vectors and data features as key vectors and value vectors to obtain fusion features with image features as the main body. Q=AW Q ,K=BW K ,V=BW V (18) O=concat(Attention(Q1,K1,V1),Attention(Q1,K1,V1),...,Attention(Q h ,K h ,V h )) (20)

[0092] Where A and B represent image features and data features respectively. Q , W K , and W V They represent the weights of the query matrix Q, the key matrix K, and the value matrix V. Attention(Q,K,V) represents a single-head attention mechanism, and concat is a matrix concatenation operation.

[0093] 2) A multi-head cross-attention mechanism is constructed with data features as query vectors, image features as key vectors and value vectors to obtain fused features with data features as the main body.

[0094] 3) Considering the different impacts of the fusion characteristics of different subjects on the model operation, the weights of the fusion features are dynamically generated to achieve adaptive feature fusion. Result fuse =Δ⊙result map +(1-Δ)result data (twenty one)

[0095] In the formula, resultfuse 、result map and result data They represent the final fusion feature, the image-based fusion feature, and the data-based fusion feature, respectively. Δ represents the adaptive weight, and ⊙ represents the Hadamard product.

[0096] The decoder injects prior data:

[0097] The weak long-term memory of wind power is the key reason why it is difficult to achieve accurate multi-step wind power prediction. The existing sequence-to-sequence model only interacts with the decoder through the context vector generated in the encoder, ignoring the influence of prior data on the decoder. Based on the existing encoder-decoder interaction mode, this application implements a new interaction mode by adding part of the data in the encoder to the decoder.

[0098] Depend on Figure 3 It can be seen that the input of the encoder consists of the known part of the original data, and the input of the decoder consists of the data to be predicted and a part of the prior data. The prior data is a part of the time text, meteorological data and wind power data that have been used in the encoder. The encoder's context vector contains the encoding information of these data, which provides a basis for decoding the data to be predicted.

[0099] The data to be predicted and the prior data are sequentially subjected to temporal feature embedding, masked attention mechanism, multi-head attention mechanism and cross-attention mechanism in the decoder. The cross-attention mechanism combines the context vector output by the decoder with the context vector output by the encoder, passes through the fully connected layer, and then outputs the prediction result. Among them, the implementation process of temporal feature embedding is as described in step S2, converting the time text into time data features, and then embedding it into meteorological data and wind power data; the multi-head attention mechanism, masked attention mechanism and cross-attention mechanism adopt the classic modules in Transformer, which will not be repeated.

[0100] Based on the above model, this application also provides a multi-step wind power prediction method, the prediction process is as follows: Figure 4 As shown. Figure 4 It can be seen that the overall prediction process can be divided into three parts: data collection and processing, model training and model testing. The multi-step wind power prediction method based on MMFformer includes the following steps:

[0101] (1) Data collection and processing

[0102] Step 1: Obtain a wind power data set by collecting historical data of wind farms.

[0103] Step 2: Filter the features in the data set that are highly correlated with wind power as the input features of the model.

[0104] Step 3: Divide the data set into two parts: training set and test set.

[0105] (2) Model training

[0106] Step 4: Use the training set data to construct the encoder input of MMFformer, then select a part of the encoder input data as prior data, and combine these prior data with the data to be predicted as the decoder input of MMFformer.

[0107] Step 5: Calculate the prediction error and save the model parameters with the smallest error as the optimal model parameters.

[0108] (3) Model testing

[0109] Step 6: Use the test set data to construct the encoder input of the MMFformer, then select a part of the encoder input data as prior data, and combine these prior data with the data to be predicted as the decoder input of the MMFformer.

[0110] Step 7: Load the optimal model parameters to perform multi-step wind power prediction and obtain the prediction results.

[0111] In order to illustrate the superiority of the method of multimodal feature fusion for multi-step wind power prediction described in this application, a comparative analysis was conducted through experiments. The data set, evaluation indicators and comparison models used are introduced below. The computer used in the experiment is Windows 10 operating system, the CPU is Intel (R) Xeon (R) Gold 5320, the main frequency is 2.20GHz, the memory size is 32GB, the GPU is RTXA4000, and the video memory is 16GB. In addition, the models used in the experiment are all built with the PyTroch framework, and the programming environment is PyCharm. The specific process is as follows:

[0112] (1) Data description and preprocessing

[0113] This application selects meteorological data and power data of a wind farm in Fujian from January 3, 2022 to December 31, 2022 with a sampling interval of 15 minutes as the research object. There are 12 sample labels including time, air pressure, relative humidity, cloud cover, wind speed and direction at 10 meters, temperature, irradiation intensity, precipitation, wind speed and direction at 100 meters, and wind power, and data set 1 is obtained. The first 90% of the samples are selected as the training set, and the remaining data are used as the test set. In order to eliminate the gap between different sample label data, normalization is performed by the following formula.

[0114] In the formula, is the normalized data, x i is the original data, x min is the minimum value in the data, x max is the maximum value in the data.

[0115] The objective loss function during model training is the mean square error (MSE), as shown in formula (23):

[0116] In the formula, y i represents the real wind power, It represents the wind power predicted by the model, and n represents the total number of predicted time points.

[0117] (2) Feature screening

[0118] Too many input features will reduce the prediction accuracy of the model and increase the risk of overfitting. Selecting features with high correlation with the target object as input can allow the model to more accurately capture the dynamic changes of the time series and improve the accuracy of prediction. In order to extract features with high correlation, the Spearman correlation coefficient between each feature is calculated, as shown in formula (24):

[0119] Where Cov(rank(X),rank(Y)) is the covariance between the ranks of the two variables, σ rank(X) and σ rank(Y) are the standard deviations of the ranks of the two variables.

[0120] The Spearman correlation heat map of the data set 1 used is as follows Figure 5 As shown. Figure 5 It can be seen that the three features of air pressure, wind speed at 10m and wind speed at 100m have the highest correlation with wind power and can be used as input features for multi-step wind power prediction.

[0121] (3) Evaluation indicators

[0122] In order to evaluate the model effect, two model evaluation indicators, root mean square error (RMSE) and mean absolute error (MAE), were used to test the performance of the prediction model.

[0123] In the formula, p i 、p' i and are the true power, the predicted value of power and the average value of power respectively, and n is the number of data.

[0124] (4) Simulation settings

[0125] To verify the effectiveness of each module of the model, MMFformer is compared with traditional Transformer, Timeformer with time tag features, GAFformer with Gram's angle field, McGAFformer with multi-channel Gram's angle field, and Priorformer with validation data. To verify the superiority of the overall performance of the proposed model, MMFformer is compared with LSTM, LSTM-CNN, and the more advanced Transformer variant model Informer.

[0126] The btach_size of each model is 64, the learning_rate is 0.0001, the hidden_size is 128, the CNN convolution kernel size is 3*3, the num_heads is 8, the dim_feedforeard is 256, the epoch is 200, and the length of the prior data and the length of the input sequence are 2 times and 4 times the prediction time step respectively. The prediction errors of each model at different time steps are shown in Table 1. Table 1 Prediction error of each model in data set 1 at different time steps

[0127] 1) The impact of adding a multi-channel Gram angle field module on model performance

[0128] Comparing #1 and #3 under three time steps, it can be seen that in the multi-step wind power prediction task, the prediction error RMSE is reduced by an average of 0.1789KW by converting the time-series wind power data into grayscale images through the Gram angle field. This is because the grayscale image has a stronger ability to express information than the data, making it easier for the model to observe the essence of wind power volatility. Comparing #3 and #4 under three time steps, it can be seen that the prediction error RMSE of the model with the multi-channel Gram angle field is reduced by an average of 0.1217KW compared to the model with the single-channel Gram angle field. The reason is that the multi-channel Gram angle field converts the data into an RGB image containing color information, which has a stronger feature expression ability than the grayscale image, allowing the model to observe the data from a higher dimension, reducing the impact of wind power volatility and greatly improving the prediction accuracy.

[0129] 2) Impact of adding the time text label module on model performance

[0130] Comparing #1 and #2 under three time steps, we can see that the prediction error RMSE of the model using time text as input feature is reduced by 0.1205KW on average compared with the original model. It can be seen that judging the order of data only by mask-assisted model cannot well highlight the temporal correlation between data. Using time text labels as features can effectively enhance the temporal nature of data and improve the model's prediction ability for long time series data.

[0131] 3) The impact of adding prior data to the decoder on model performance

[0132] Comparing #1 and #5 under three time steps, we can see that the model with prior data injected into the decoder has lower prediction error RMSE and MAE than the original model. The reason is that the encoder and decoder in the traditional model only rely on ContextVector for information exchange, which causes the decoding difficulty to gradually increase with the increase of time steps, reducing the prediction accuracy of long sequence data. The decoder with prior data injected has more information exchange with the encoder. Prior data can provide prior knowledge for the decoder, enhance the decoding ability of the decoder, and improve the prediction accuracy of the model in long sequence prediction tasks.

[0133] Based on the prediction errors of the models in Table 1, it can be seen that compared with #1 and #6 under three time steps, the prediction error RMSE of MMFformer decreased by an average of 7.03% compared with the original model. The reason is that MMFformer integrates the features of three different modes, namely time text, wind power image and meteorological data, through a multimodal fusion method, providing the model with more comprehensive and diverse information. Comparing #6, #7, #8 and #9 under three time steps, it can be seen that compared with LSTM, LSTM-CNN and Informer, the prediction error RMSE of MMFformer decreased by an average of 46%, 30.90% and 3.79% respectively. It can be seen that MMFformer has superior prediction performance compared with other types of models.

[0134] In order to verify the generalization ability of the proposed model, this section adds a data set of a 80KW wind farm in Germany to test MMFformer. From this data set, the data from January 1 to December 31, 2023 are selected as data set 2 for the same operation. The first 90% of the points are selected as the training set, and the last 10% of the points are selected as the test set. The model number and prediction error are shown in Table 2. Table 2 Prediction error of each model in data set 2 at different time steps

[0135] By comparing #1 and #2, we can see that using time text as input features can reduce the prediction error of the model. This is because only considering the temporal nature of the data cannot fully capture the temporal characteristics of wind power in different seasons and at different times. By comparing #1, #3 and #4, we can see that converting wind power into image features can effectively reduce the interference caused by wind power volatility to the model. Multi-channel color images have stronger feature expression capabilities than single-channel grayscale images. By comparing #5 and #6, we can see that injecting prior data into the decoder can effectively improve the prediction accuracy of the model. This is because the prior data provides prior guidance for the decoder's work.

[0136] Combining the prediction curves of each model, it can be seen that the prediction accuracy of MMFformer is higher than any other model; by comparing #6, #7, #8 and #9, it can be seen that compared with the recurrent network model and the Transformer variant model, the RMSE and MAE of MMFformer are the smallest, which shows that MMFformer has a better performance in the multi-step wind power prediction task.

[0137] In summary, in order to address the problems of strong volatility and weak long-term memory in multi-step wind power prediction, this application proposes a method of multimodal feature fusion for multi-step wind power prediction, which effectively improves the prediction accuracy of multi-step wind power.

[0138] Finally, it should be noted that the parts of the present invention that are not described in detail are all prior art. Those of ordinary skill in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the aforementioned examples, those of ordinary skill in the art can still modify the technical solutions recorded in the aforementioned examples, or replace some of the technical features therein with equivalents. Any modifications, equivalent replacements, etc. made within the spirit and principles of the invention should be included in the scope of protection of the invention.

Claims

1. A method for constructing a multimodal feature fusion model, characterized in that: The multimodal feature fusion model is used for multi-step wind power prediction, comprising the following steps: S1. Construct a multi-channel Gram angle field, convert wind power data into RGB color images with color information, and obtain wind power image features; S2, converting the time text into time data features, and then embedding them into the meteorological data features and the wind power image features described in step S1, and extracting the hidden data features and image features; S3, using a cross-modal fusion method to fuse the data features extracted in step S2 with the image features, and using an adaptive decision-making method to adjust the dynamic weights when the data features and the image features are fused to obtain fused features containing information of different modalities; S4, using the Transformer encoder to extract the correlation information between the fusion features described in step S3 and the wind power, and outputting the context vector to the decoder through a multi-head attention mechanism; S5. The decoder decodes the input data into wind power to be predicted according to the context vector and the priori data described in step S4.

2. The method for constructing a multimodal feature fusion model according to claim 1, characterized in that: The S1 specifically includes: In S1, the multi-channel Gram angle field is constructed by converting the grayscale image I generated by the Gram angle sum field and the grayscale image II generated by the Gram angle difference field into a grayscale image III in a randomly weighted manner, and assigning the grayscale image I, the grayscale image II, and the grayscale image III to the red channel, the green channel, and the blue channel, respectively, to obtain an RGB color image.

3. The method for constructing a multimodal feature fusion model according to claim 2, characterized in that: The process of constructing grayscale image III is as follows: IGAF i，j =R j GASP i，j +(1-R j ) KEEP i，j In the formula, R j Represents a random number in the interval [0, 1], IGAF i，j Represents grayscale image III, GASF i，j Represents a grayscale image I, GADF i，j represents grayscale image II; The resulting RGB color image is: I=(I R ,I G ,I B )=(GASF,GADF,IGAF) Where I represents the final color image, I R ,I G and I B Represent the red channel, green channel and blue channel in the color image respectively.

4. The method for constructing a multimodal feature fusion model according to claim 1, characterized in that: The S2 specifically includes: after the time data features are embedded into the meteorological data features, 1DCNN is used to capture the hidden features of the data; after the time data features are embedded into the wind power image features, the convolution layer in 2DCNN is used to extract the image features, and the output of the convolution layer is downsampled through a pooling operation.

5. The method for constructing a multimodal feature fusion model according to claim 1, characterized in that: In S2, the method of converting the time text into time data features is as follows: (1) To ensure the uniqueness of the year, use One-Hot encoding: n year (i)=year(i)-year min i=0,1,2…,l year binary =concat(b i ) In the formula, l and i represent the length of the time text sequence and the position of the point to be processed, respectively, and j represents n i The j-th binary bit of mod and concat are the floor operator, modulus operator and concatenation operator respectively; (2) In order to ensure the uniqueness of the month and day in the year, position codes are used to represent them: In the formula, month_day(i) and year_day(i) represent the total number of days in the month and year of the point to be processed respectively; (3) In view of the periodicity of the time in the year, a cyclic coding method is used to represent it: In the formula, hour(i) and m(i) represent the hour and minute of the point to be processed respectively; (4) Perform dimensional transformation so that the time data feature is consistent with the other feature dimensions, as shown in the following formula: In the formula, p and q represent the input feature dimension and output feature dimension of time data respectively, σ is the activation function, ω qp With b q denote weight and bias respectively.

6. The method for constructing a multimodal feature fusion model according to claim 1, characterized in that: In S3, the implementation process of cross-modal fusion and adaptive decision fusion is as follows: A multi-head cross-attention mechanism is constructed with image features as query vectors and data features as key vectors and value vectors to obtain fusion features with image features as the main body; A multi-head cross-attention mechanism is constructed with data features as query vectors, image features as key vectors and value vectors to obtain fusion features with data features as the main body; Dynamically generate the weights of image-based fusion features and data-based fusion features to achieve adaptive feature fusion.

7. The method for constructing a multimodal feature fusion model according to claim 1, characterized in that: In step S5, the input of the decoder includes the data to be predicted and the priori data, wherein the priori data is a part of the time text, meteorological data and wind power data that have been used by the encoder in step S4.

8. The method for constructing a multimodal feature fusion model according to claim 7, characterized in that: The data to be predicted and the prior data are sequentially subjected to temporal feature embedding, masked attention mechanism, multi-head attention mechanism and cross attention mechanism in the decoder. The cross attention mechanism combines the context vector output by the decoder with the context vector output by the encoder, passes through a fully connected layer, and then outputs a prediction result.

9. A multi-step wind power prediction method, characterized in that: The method is based on a model obtained by the multimodal feature fusion model construction method according to any one of claims 1 to 8, and comprises the following steps: (1) Data collection and processing Step 1: Collect historical data of wind farms to obtain wind power data sets; Step 2: Filter the features in the data set that are highly correlated with wind power as the input features of the model; Step 3: Divide the data set into two parts: training set and test set; (2) Model training Step 4: Use the training set data to construct the encoder input of the model, then select a part of the encoder input data as prior data, and combine these prior data with the data to be predicted as the decoder input; Step 5: Calculate the prediction error and save the model parameters with the smallest error as the optimal model parameters; (3) Model testing Step 6: Use the test set data to build the encoder input of the model, then select a part of the encoder input data as prior data, and combine these prior data with the data to be predicted as the decoder input; Step 7: Load the optimal model parameters to perform multi-step wind power prediction and obtain the prediction results.

Images