Spatiotemporal wind power prediction method based on meta-learning and content matching

Abstract

The application belongs to the technical field of wind power prediction, and particularly relates to a spatio-temporal wind power prediction method based on meta-learning and content matching. First, a source domain dataset formed by multiple source domain tasks is obtained, and a target domain task is obtained; then, a wind power prediction model is constructed, including a periodic embedding layer, an encoder, a content matching storage network and a prediction layer; the periodic embedding layer is used to capture the periodic information of wind power data, the encoder extracts time features and space features respectively and fuses them to obtain spatio-temporal fusion features; the content matching storage network matches the spatio-temporal fusion features extracted by the encoder with memory node feature vectors according to similarity to obtain matched node feature vectors; the prediction layer uses the matched node feature vectors and the spatio-temporal fusion features extracted by the encoder for prediction to output wind power prediction results; finally, the wind power prediction model is trained based on meta-learning using the source domain dataset, and the wind power prediction model trained in the source domain is migrated to the target domain for prediction of the target domain task. The method fully considers the spatio-temporal correlation of wind power, and the model has good generalization ability and prediction performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of wind power prediction, and particularly relates to a spatio-temporal wind power prediction method based on meta-learning and content matching. BACKGROUND

[0002] Wind power prediction of a wind farm can effectively alleviate the peak regulation pressure of a power grid and realize reasonable dispatching of a power system. Due to the randomness and uncertainty of wind energy, great challenges are brought to effective prediction of wind power. In particular, for some newly built wind farms, due to the lack of effective historical data, it is more difficult to accurately predict the wind power in a future time period.

[0003] In recent years, deep learning technology is widely applied in the field of wind power prediction due to its strong nonlinear data processing ability and the ability to extract implicit features. However, model training is usually based on a large number of data samples, and the model cannot be effectively trained when facing small sample data, resulting in poor prediction effect. Transfer learning can solve the small sample problem. Specifically, the model is pre-trained using source domain data, and then the pre-trained model is fine-tuned using target domain data. Usually, it is based on the premise that the source domain and target domain data have certain similarities. However, different data sets will have different data distributions. In the case of different data distributions between the source domain and the target domain, the knowledge learned from the source domain may not be directly applicable to the target domain, resulting in negative transfer, which leads to poor generalization ability of the model and cannot achieve the expected prediction effect on the target domain. Meta-learning can also solve the small sample problem. Through training, the model has the ability to learn, so that the model can quickly adapt to the target domain data in the case of different data distributions between the source domain and the target domain, and improve the generalization ability of the model on new tasks. SUMMARY

[0004] In view of the deficiencies of the prior art, the technical problem to be solved by the present application is to provide a spatio-temporal wind power prediction method based on meta-learning and content matching.

[0005] The technical problem solved by the present application adopts the following technical solution:

[0006] A spatio-temporal wind power prediction method based on meta-learning and content matching, characterized in that it comprises the following steps:

[0007] First step: data sampling is performed on a plurality of source domain wind farms to obtain a source domain data set formed by a plurality of source domain tasks; data of a target wind farm is obtained as a target domain task;

[0008] Second step: a wind power prediction model is constructed; the wind power prediction model comprises a periodic embedding layer, an encoder, a content matching storage network and a prediction layer;

[0009] The encoder comprises a plurality of encoder layers, each of which comprises three modules of time feature extraction, space feature extraction and space-time feature fusion; in the space feature extraction module, first, the parameterized adjacency matrix and the dynamic adjacency matrix are constructed by formula (5) and (6) and the dynamic adjacency matrix :

[0010] (5)

[0011] (6)

[0012] In the formula, denotes a normalization operation, denotes a ReLU activation function, denotes a learnable parameter matrix, denotes matrix transposition, denotes the feature of the nth task after periodic embedding, , denotes a learnable parameter matrix;

[0013] Then, the parameterized adjacency matrix , the dynamic adjacency matrix and the input feature of the space feature extraction module are input into the graph convolution network to extract the space feature, and the process is represented as:

[0014] (7)

[0015] (8)

[0016] In the formula, denotes a space feature, denotes a graph convolution operation, denotes a root mean square layer normalization operation;

[0017] The space-time fusion feature extracted by the encoder is input into the content matching storage network to match information, and the matched node feature vector is obtained, and the process is represented as:

[0018] (11)

[0019] (12)

[0020] (13)

[0021] In the formula, denotes a query vector, denotes a weight matrix, denotes a bias, represents the similarity between the query vector and the jth memory node feature vector, represents the number of memory node feature vectors;

[0022] the matched node feature vector and the spatio-temporal fusion feature extracted by the encoder are input into the prediction layer for prediction, and a prediction result is output;

[0023] Step 3: training the wind power prediction model based on meta-learning;

[0024] The task model has the same network architecture as the wind power prediction model but different parameters. The task model is trained and updated using the source domain dataset, and the parameters of the wind power prediction model are updated using the parameters of the task model, thereby obtaining the source domain trained wind power prediction model.

[0025] Step 4: training the source domain trained wind power prediction model using the target domain task to obtain the target domain trained wind power prediction model; and using the target domain trained wind power prediction model for prediction of the target domain task.

[0026] Further, the periodic embedding layer is represented as:

[0027] (1)

[0028] wherein, is the wind power data sequence of the nth task, , , are the intra-day embedding vector, the day embedding vector and the month embedding vector, respectively, represents a weight vector, represents a bias, represents a GELU activation function, represents a splicing operation.

[0029] Further, the time feature extraction module comprises a plurality of time feature extraction layers. In each time feature extraction layer, the input feature is subjected to padding processing through dilated convolution, and the padded feature is subjected to four convolution operations with different convolution kernel sizes to obtain features , , and . By cropping, the features , , and are aligned to obtain cropped features , , ​and ; and the cropped features are processed by formula (3) to obtain output features of the time feature extraction layer 、 、 and ;

[0030] (3)

[0031] In the formula, is the output feature of the i-th time feature extraction layer, represents a batch normalization operation, represents a random inactivation operation, is the input feature of the time feature extraction layer;

[0032] The output feature of the last time feature extraction layer is added to the input feature of the time feature extraction module after an activation operation, to obtain the time feature , and the process is represented as:

[0033] (4)

[0034] In the formula, m represents the number of time feature extraction layers.

[0035] Further, the space-time feature fusion module fuses the time feature and the space feature to obtain the space-time fusion feature, and the process is represented as:

[0036] (9)

[0037] (10)

[0038] In the formula, is the space-time fusion feature, represents a linear operation, represents a Hadamard product, represents a Sigmoid activation function.

[0039] Further, the encoder comprises two encoder layers, and the time feature extraction module comprises three time feature extraction layers.

[0040] Compared with the prior art, the present application has the following advantages:

[0041] 1. Existing research mainly considers the temporal correlation of wind power. However, in reality, there is also a spatial dependency between wind turbines, which also affects wind power. Therefore, this invention fully considers the temporal and spatial correlation of wind power, extracts temporal and spatial features separately, and then fuses them to obtain spatiotemporal fused features. The memory node feature vectors recorded and stored in the content matching storage network are used to enhance the model's long-term memory and contextual understanding capabilities. By querying the vectors and matching them with the memory node feature vectors, more useful information for the prediction task is found, improving the model's prediction performance.

[0042] 2. This invention addresses the difficulty in predicting wind power output in newly constructed wind farms due to scarce sample data. It employs a meta-learning strategy, treating different source domain data as different tasks for model training. Furthermore, it introduces a content-matching storage network to learn typical features from the data, mitigating the discrepancy between source and target domain data and improving the model's generalization ability and prediction performance in the target domain. Compared to transfer learning, this invention's method can adapt to target domain data more quickly, improving model efficiency and preventing overfitting. Attached Figure Description

[0043] Figure 1 This is a structural diagram of the wind power prediction model of the present invention;

[0044] Figure 2 It is a comparison of MAE of different models;

[0045] Figure 3 It is a comparison of RMSE of different models;

[0046] Figure 4 This is a comparison of the prediction results of the present invention. Detailed Implementation

[0047] Specific embodiments are given below with reference to the accompanying drawings. These specific embodiments are only used to describe the technical solution of the present invention in detail, and are not intended to limit the scope of protection of this application.

[0048] This invention provides a spatiotemporal wind power prediction method based on meta-learning and content matching (hereinafter referred to as the method, see below). Figures 1-4 ), including the following steps:

[0049] Step 1: Data sampling is performed from multiple source wind farms with abundant data. The data collected from each source wind farm is treated as a task to obtain the source domain dataset. Each source domain task is divided into a support set and a query set. The support set is used to train the model, and the query set is used to evaluate the model's performance. Represents the number of source domain tasks; data from the target wind farm is obtained as the target domain tasks. The target domain task is also divided into two parts: the support set and the query set. The target wind farm is a newly built wind farm with a small amount of data.

[0050] Step 2: Construct a wind power prediction model; such as Figure 1 As shown, the wind power prediction model includes a periodic embedding layer, an encoder, a content matching storage network, and a prediction layer. The periodic embedding layer is used to capture the periodic information of wind power data. The encoder extracts temporal and spatial features and fuses them to obtain spatiotemporal fusion features. The content matching storage network matches the spatiotemporal fusion features extracted by the encoder with the memory node feature vectors based on similarity to obtain matched node feature vectors. The prediction layer uses the matched node feature vectors and the spatiotemporal fusion features extracted by the encoder to make predictions and output the prediction results.

[0051] Considering the significant periodicity of meteorological conditions (such as wind speed, wind direction, and temperature) affecting wind turbine power generation performance—for example, similar meteorological conditions may exist at the same time of day, on the same day of the year, or in the same month of the year—and these meteorological conditions are important factors affecting wind power output, a periodic embedding layer is used to enhance the task, enabling the model to learn the periodic changes in the data. The operation of the periodic embedding layer is represented as follows:

[0052] (1)

[0053] In the formula, It is the feature of the nth task after periodic embedding. This is the wind power data sequence for the nth task. , , These are the intraday embedding vector, the daily embedding vector, and the monthly embedding vector, respectively. Represents the weight vector. Indicates bias. This represents the GELU activation function. This indicates a splicing operation.

[0054] The encoder contains multiple encoder layers (two in this embodiment). The output features of the previous encoder layer are the input features of the next encoder layer. The encoder layer includes a time feature extraction module, a spatial feature extraction module, and a spatiotemporal feature fusion module.

[0055] The temporal feature extraction module is used to extract long-term temporal dependencies between data. It includes multiple temporal feature extraction layers (three in this embodiment), where the output feature of the previous temporal feature extraction layer becomes the input feature of the next temporal feature extraction layer. In the temporal feature extraction layer, the input features are padded through dilated convolution to maintain the sequence length during convolution. The padding size is expressed as:

[0056] (2)

[0057] In the formula, represents the padding size, represents the expansion factor, represents the convolution kernel size of the expanded convolution; in this embodiment, the expansion factors of the three time feature extraction layers are 1, 2 and 4 respectively;

[0058] The padded features are respectively subjected to convolution operations with four convolution kernel sizes of 1x2, 1x3, 1x5 and 1x6, to obtain features , , and ; the features , , and are cropped to align the four features, to obtain cropped features , , and ; the cropped features , , and are processed by formula (3) to obtain the output features of the time feature extraction layer;

[0059] (3)

[0060] In the formula, is the output feature of the i-th time feature extraction layer, represents the batch normalization operation, represents the random inactivation operation, is the input feature of the time feature extraction layer;

[0061] The output feature of the last time feature extraction layer is subjected to an activation operation, and then is connected in residual with the input feature of the time feature extraction module to obtain the time feature , which is represented as:

[0062] (4)

[0063] In the formula, is the output feature of the m-th time feature extraction layer, represents the ReLU activation function.

[0064] The spatial dependency between the wind turbines affects the power of the wind turbines. On one hand, different wind turbines may have similar weather conditions such as wind conditions, temperature and atmospheric pressure due to similar spatial positions and similar geographical environments, and these weather conditions are important factors affecting the power. On the other hand, the performance of the wind turbines is affected by the wake effect, which refers to the fact that when the wind turbines are adjacent in geographical position, the wake generated by the operation of the front row of wind turbines will cause the inflow wind speed of the rear row of wind turbines to decrease, further affecting the power output of the rear row of wind turbines. Therefore, the spatial feature extraction module is used to extract the spatial dependency between the data. In the spatial feature extraction module, a parameterized adjacency matrix and a dynamic adjacency matrix are first constructed, which are respectively used to represent the inherent spatial relationship and the dynamically changing spatial relationship between the wind turbines;

[0065] (5)

[0066] (6)

[0067] wherein, represents the parameterized adjacency matrix, represents the dynamic adjacency matrix, represents the normalization operation, is a learnable parameter matrix, , represents the learnable parameter matrix;

[0068] Then, the parameterized adjacency matrix , the dynamic adjacency matrix and the input features of the spatial feature extraction module are input into the graph convolution network GCN to aggregate the features of the nodes and their neighbor nodes, to obtain the spatial features, i.e. the output features of the spatial feature extraction module, represented as:

[0069] (7)

[0070] (8)

[0071] wherein, represents the spatial feature, represents the graph convolution operation, represents the root mean square layer normalization operation.

[0072] The temporal features and the spatial features are input into the spatio-temporal feature fusion module for fusion, to capture more potential patterns and trends between the data, to obtain the spatio-temporal fusion features, i.e. the output features of the encoder layer;

[0073] (9)

[0074] (10)

[0075] wherein, is the spatio-temporal fusion feature, denotes a linear operation, represents a Hadamard product, denotes a Sigmoid activation function.

[0076] The content matching storage network is used to learn typical features in the data, and the spatio-temporal fusion feature extracted by the encoder is input into the content matching storage network to match the memory node feature vector; first, the spatio-temporal fusion feature extracted by the encoder is converted into a query vector, and then the similarity between the query vector and the memory node feature vector is calculated, the similarity is taken as the weight of the memory node feature vector, and the weighted sum of all memory node feature vectors is obtained to obtain the matched node feature vector , the process is represented as:

[0077] (11)

[0078] (12)

[0079] (13)

[0080] wherein, denotes a query vector, is the spatio-temporal fusion feature extracted by the encoder, denotes a weight matrix, denotes a bias, denotes the similarity between the query vector and the jth memory node feature vector, denotes the jth memory node feature vector, denotes the number of memory node feature vectors.

[0081] The initial memory node feature vector is obtained by random initialization, and is updated in the model training process, and the updated memory node feature vector learns the typical features in the data. The matched node feature vector and the spatio-temporal fusion feature extracted by the encoder are input into the prediction layer for prediction to obtain the wind power prediction value. The prediction layer is a multilayer perceptron (MLP).

[0082] Step 3: training the wind power prediction model based on meta-learning;

[0083] ​The task model and the wind power prediction model are trained on the source domain dataset, and the network architecture of the task model is the same as that of the wind power prediction model, but the model parameters are different. The support set and the query set of the source domain task are input into the task model in turn for training, and the parameters of the task model are updated by gradient backpropagation. The parameters of the task model are used to update the parameters of the wind power prediction model. After the task model completes the training of a source domain task, the parameters of the task model are used to update the wind power prediction model, and the training of all source domain tasks is completed, that is, the trained wind power prediction model of the source domain is obtained.

[0084] The performance of the trained model is evaluated using the mean absolute error (MAE) and the root mean square error (RMSE). When the mean absolute error and the root mean square error are both minimum, the model performance is best.

[0085] (14)

[0086] (15)

[0087] wherein, represents the number of tasks in the domain, represents the number of samples in the query set of the task, , respectively represent the true value and the predicted value of the wind power of the u-th sample in the v-th task.

[0088] Fourth step: The source domain trained wind power prediction model is migrated to the target domain, and the source domain trained wind power prediction model is trained using the support set of the target domain task, and the parameters of the source domain trained wind power prediction model are fine-tuned to obtain the target domain trained wind power prediction model. The target domain trained wind power prediction model is used for prediction of the target domain task, that is, the query set of the target domain task is input into the target domain trained wind power prediction model to obtain the prediction result.

[0089] Embodiment

[0090] In this embodiment, SDWPF, GEFCom2014 and Engie are selected as source domain datasets, and the target domain task is a self-owned dataset. The target domain is considered to contain ten days of historical data, and the wind power in the future 12 steps (2 hours) is predicted based on the wind power in the past 36 steps (6 hours).

[0091] Under the same conditions, a diffusion convolution recurrent neural network (DCRNN, see Li Y, Yu R, Shahabi C, et al. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting [J]. 2017. DOI: 10.48550 / arXiv.1707.01926.) without transfer learning, a CNN-LSTM-based transfer learning method (CNN-LSTM-TL) and the wind power prediction model of the application are selected for comparison. The mean absolute error comparison is shown in Figure 2 , the root mean square error comparison is shown in Figure 3 , Figure 4 is a comparison chart of the prediction results of the application.

[0092] From Figure 2 and Figure 3 , it can be seen that the MAE and RMSE of the wind power prediction model of the application are lower than those of the other two models, and the prediction effect is better. The DCRNN model uses non-transfer learning technology and needs to rely on a large amount of training data that is homologous to the target domain task. However, the target domain task is derived from a newly built wind farm, and the data samples are limited, resulting in poor model training effect. The application uses a meta-learning training strategy combined with content matching to train the model on the source domain dataset, so that the model learns the prior knowledge of multiple source domain tasks and has the ability to quickly adapt to new tasks, so that a small amount of data on the target domain can achieve good prediction effect. The CNN-LSTM-TL model improves the prediction effect of the model through transfer learning compared with the DCRNN, but the CNN-LSTM-TL only extracts features from the time dimension and does not consider the spatial correlation between wind turbines, while the wind power prediction model of the application fully considers the inherent and dynamically changing spatial relationship between wind turbines, fully extracts spatial features, and further improves the prediction performance.

[0093] The unmentioned parts of the application apply to the prior art.

Claims

1. A spatio-temporal wind power prediction method based on meta-learning and content matching, characterized in that, The method comprises the following steps: Step 1: sampling data of multiple source domain wind farms to obtain a source domain data set formed by multiple source domain tasks; obtaining data of a target wind farm as a target domain task; Step 2: constructing a wind power prediction model; the wind power prediction model comprises a periodic embedding layer, an encoder, a content matching storage network and a prediction layer; The encoder comprises a plurality of encoder layers, each of which comprises three modules of time feature extraction, space feature extraction and space-time feature fusion; in the space feature extraction module, first, the parameterized adjacency matrix and the dynamic adjacency matrix are constructed by formula (5) and (6) and formula (6) : （5） （6） In the formula, This indicates a normalization operation. Represents the ReLU activation function. This represents the learnable parameter matrix. Indicates matrix transpose. This represents the feature of the nth task after periodic embedding. , Represents the learnable parameter matrix; Then, the parameterized adjacency matrix , the dynamic adjacency matrix and the input features of the spatial feature extraction module are input into the graph convolution network to extract spatial features, and the process is represented as: （7） （8） In the formula, denotes a spatial feature, denotes a graph convolution operation, denotes a root mean square layer normalization operation; Spatio-temporal fusion features extracted by the encoder Input into the content matching storage network for information matching to obtain a matched node feature vector The process is represented as: （11） （12） （13） wherein, represents a query vector, represents a weight matrix, represents a bias, represents a similarity between the query vector and the jth memory node feature vector represents a number of memory node feature vectors;​ The matched node feature vector and the spatio-temporal fusion feature extracted by the encoder is input into a prediction layer for prediction, and a prediction result is output. Step 3: training the wind power prediction model based on meta-learning; The task model has the same network architecture as the wind power prediction model but different parameters, the task model is trained and updated using the source domain data set, the parameters of the wind power prediction model are updated using the parameters of the task model, and a source domain trained wind power prediction model is obtained; Step 4: training the source domain trained wind power prediction model using the target domain task to obtain a target domain trained wind power prediction model; and using the target domain trained wind power prediction model for prediction of the target domain task.

2. The spatio-temporal wind power prediction method based on meta-learning and content matching according to claim 1, characterized in that, The periodic embedding layer is represented as: （1） wherein, is the wind power data sequence of the nth task, , , are the intra-day embedding vector, the day embedding vector and the month embedding vector, respectively, denotes a weight vector, denotes a bias, denotes a GELU activation function, denotes a concatenation operation.

3. The spatio-temporal wind power prediction method based on meta-learning and content matching according to claim 1 or 2, characterized in that, The time feature extraction module comprises a plurality of time feature extraction layers, in each time feature extraction layer, the input features are subjected to padding processing through dilated convolution, the padded features are subjected to four convolution operations with different convolution kernel sizes respectively, and features , , and are obtained; cropping the feature , , and aligning, to obtain a cropped feature , , and ; The cropped features are processed by formula (3) again to obtain output features of the time feature extraction layer. , , and ​ （3） In the formula, is the output feature of the i-th time feature extraction layer, represents a batch normalization operation, represents a random inactivation operation, is the input feature of the time feature extraction layer; output features of the last temporal feature extraction layer After the activation operation, the input features of the temporal feature extraction module are added to obtain the temporal features The process is represented as: （4） In the formula, m represents the number of time feature extraction layers.

4. The spatio-temporal wind power prediction method based on meta-learning and content matching according to claim 3, characterized in that, The space-time feature fusion module fuses the time feature and the space feature to obtain space-time fusion features, and the process is represented as: （9） （10） wherein, is a spatio-temporal fusion feature, denotes a linear operation, represents a Hadamard product, denotes a Sigmoid activation function.

5. The spatio-temporal wind power prediction method based on meta-learning and content matching according to claim 4, characterized in that, The encoder comprises two encoder layers, and the time feature extraction module comprises three time feature extraction layers.

Images