Space-time wind power prediction method based on meta learning and content matching

Abstract

The invention belongs to the technical field of wind power prediction, and particularly relates to a space-time wind power prediction method based on meta learning and content matching. The method comprises the following steps: firstly, acquiring a source domain data set formed by a plurality of source domain tasks, and acquiring a target domain task; then, constructing a wind power prediction model which comprises a periodic embedding layer, an encoder, a content matching storage network and a prediction layer; the periodic embedding layer is used for capturing periodic information of wind power data, and the encoder extracts time features and space features and fuses the time features and the space features to obtain space-time fusion features; the content matching storage network matches the space-time fusion feature extracted by the encoder with the memory node feature vector according to similarity to obtain a matched node feature vector; the prediction layer performs prediction by using the matched node feature vector and the space-time fusion feature extracted by the encoder, and outputs a wind power prediction result; and finally, training the wind power prediction model by using the source domain data set and based on meta learning, and migrating the wind power prediction model after source domain training to a target domain for prediction of a target domain task. According to the method, the space-time correlation of the wind power is fully considered, and the model has good generalization ability and prediction performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of wind power prediction, and in particular relates to a spatiotemporal wind power prediction method based on meta-learning and content matching. Background Art

[0002] Wind power forecasting of wind farms can effectively alleviate the peak load pressure of power grids and realize the reasonable dispatch of power systems. Due to the randomness and uncertainty of wind energy itself, it brings great challenges to the effective forecasting of wind power. Especially for some newly built wind farms, due to the lack of effective historical data, it is more difficult to accurately estimate the wind power in the future time period.

[0003] In recent years, deep learning technology has been widely used in the field of wind power prediction due to its powerful nonlinear data processing capabilities and the ability to extract implicit features. However, model training is often based on a large number of data samples. When faced with small sample data, the model cannot be effectively trained, resulting in poor prediction results. Transfer learning can solve the problem of small samples. Specifically, the model is pre-trained using source domain data, and then the pre-trained model is fine-tuned using target domain data. This is usually based on the premise that the source domain and target domain data have a certain similarity. However, different data sets have different data distributions. When the source domain and target domain data are distributed differently, the knowledge learned from the source domain may not be directly applied to the target domain, resulting in negative transfer, resulting in poor generalization of the model and failure to achieve the expected prediction results in the target domain. Meta-learning can also solve the problem of small samples. Through training, the model has the ability to learn, so that the model can quickly adapt to the target domain data when the source domain and target domain data are distributed differently, thereby improving the generalization ability of the model on new tasks. Summary of the invention

[0004] In view of the deficiencies in the prior art, the technical problem that the present invention intends to solve is to provide a spatiotemporal wind power prediction method based on meta-learning and content matching.

[0005] The present invention solves the technical problem by adopting the following technical solution: A spatiotemporal wind power prediction method based on meta-learning and content matching, characterized by comprising the following steps: Step 1: Sample data from multiple source domain wind farms to obtain a source domain data set formed by multiple source domain tasks; obtain data from the target wind farm as the target domain task; Step 2: construct a wind power prediction model; the wind power prediction model includes a periodic embedding layer, an encoder, a content matching storage network and a prediction layer; The encoder comprises a plurality of encoder layers, each of which comprises three modules: temporal feature extraction, spatial feature extraction and spatiotemporal feature fusion. In the spatial feature extraction module, a parameterized adjacency matrix is ​​first constructed by equations (5) and (6): and the dynamic adjacency matrix : (5) (6) In the formula, represents the normalization operation, represents the ReLU activation function, represents the learnable parameter matrix, represents the matrix transpose, represents the features of the nth task after periodic embedding, , represents the learnable parameter matrix; Then, the parameterized adjacency matrix , Dynamic adjacency matrix And the input features of the spatial feature extraction module are input into the graph convolutional network to extract spatial features. The process is expressed as: (7) (8) In the formula, Represents spatial features, represents the graph convolution operation, represents the root mean square layer normalization operation; The spatiotemporal fusion features extracted by the encoder Input into the content matching storage network for information matching to obtain the matching node feature vector , the process is expressed as: (11) (12) (13) In the formula, represents the query vector, represents the weight matrix, represents the bias, Represents the query vector and the feature vector of the jth memory node The similarity between Represents the number of memory node feature vectors; Match the node feature vector And the spatiotemporal fusion features extracted by the encoder Input into the prediction layer for prediction and output the prediction result; Step 3: Train the wind power prediction model based on meta-learning; The task model and the wind power prediction model have the same network architecture but different parameters. The task model is trained and updated using the source domain data set, and the parameters of the wind power prediction model are updated using the parameters of the task model to obtain the wind power prediction model after source domain training. Step 4: Use the target domain task to train the wind power prediction model trained in the source domain to obtain the wind power prediction model trained in the target domain; use the wind power prediction model trained in the target domain for the prediction of the target domain task.

[0006] Further, the periodic embedding layer is expressed as: (1) In the formula, is the wind power data sequence of the nth task, , , They are intra-day embedding vector, daily embedding vector and monthly embedding vector, represents the weight vector, represents the bias, represents the GELU activation function, Represents a concatenation operation.

[0007] Furthermore, the temporal feature extraction module comprises a plurality of temporal feature extraction layers. In each temporal feature extraction layer, the input feature is filled by dilated convolution, and the filled feature is respectively subjected to four convolution operations with different convolution kernel sizes to obtain the feature , , and ; By cutting out the features , , and Align and get the clipping features , , and ; Then use formula (3) to cut the features , , and Processing is performed to obtain the output features of the time feature extraction layer; (3) In the formula, is the output feature of the i-th temporal feature extraction layer, represents the batch normalization operation, represents the random dropout operation, It is the input feature of the temporal feature extraction layer; Output features of the last temporal feature extraction layer After the activation operation, it is added to the input features of the time feature extraction module to obtain the time feature , the process is expressed as: (4) Where m represents the number of temporal feature extraction layers.

[0008] Furthermore, the spatiotemporal feature fusion module fuses the temporal features and spatial features to obtain spatiotemporal fusion features. The process is expressed as follows: (9) (10) In the formula, It is the time-space fusion feature. represents a linear operation, Representing Hadamard, Represents the Sigmoid activation function.

[0009] Furthermore, the encoder includes two encoder layers, and the time feature extraction module includes three time feature extraction layers.

[0010] Compared with the prior art, the present invention has the following beneficial effects: 1. Existing studies mainly consider the temporal correlation of wind power, but in fact there is also a spatial dependency between wind turbines, which will also affect wind power. Therefore, the present invention fully considers the temporal and spatial correlation of wind power, extracts temporal features and spatial features respectively, and then fuses them to obtain spatiotemporal fusion features. The memory node feature vectors recorded and stored in the content matching storage network are used to enhance the long-term memory and context understanding capabilities of the model. By matching the query vector with the information of the memory node feature vector, more useful information for the prediction task is obtained, thereby improving the prediction performance of the model.

[0011] 2. Aiming at the problem that wind power prediction is difficult in newly built wind farms due to the scarcity of sample data, the present invention uses a meta-learning strategy to treat different source domain data as different task training models, and introduces typical features in content matching storage network learning data to alleviate the difference between source domain data and target domain data, thereby improving the generalization ability of the model and the prediction effect in the target domain. Compared with transfer learning, the method of the present invention can adapt to the target domain data more quickly, improve the operating efficiency of the model, and prevent overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1is a structural diagram of the wind power prediction model of the present invention; Figure 2 It is the MAE comparison of different models; Figure 3 It is the RMSE comparison of different models; Figure 4 It is a comparison of the prediction results of the present invention. DETAILED DESCRIPTION

[0013] Specific embodiments are given below in conjunction with the accompanying drawings. The specific embodiments are only used to introduce the technical solutions of the present invention in detail and are not intended to limit the protection scope of the present application.

[0014] The present invention provides a spatiotemporal wind power prediction method based on meta-learning and content matching (hereinafter referred to as method, see Figures 1 to 4 ), including the following steps: Step 1: Sample data from multiple data-rich source domain wind farms, treat the data collected from each source domain wind farm as a task, and obtain the source domain dataset ,Each source domain task is divided into two parts: support set and query set. The support set is used to train the model, and the query set is used to evaluate the model performance. Represents the number of source domain tasks; obtains the data of the target wind farm as the target domain task ,The target domain task is also divided into two parts including support set and query set. ,The target wind farm is a newly built wind farm with a small amount of ,data.

[0015] Step 2: Construct a wind power prediction model; Figure 1 As shown in the figure, the wind power prediction model includes a periodic embedding layer, an encoder, a content matching storage network and a prediction layer; the periodic embedding layer is used to capture the periodic information of wind power data, the encoder extracts time features and space features respectively and fuses them to obtain time-space fusion features; the content matching storage network matches the time-space fusion features extracted by the encoder with the memory node feature vector according to similarity to obtain a matched node feature vector; the prediction layer uses the matched node feature vector and the time-space fusion features extracted by the encoder to make predictions and output the prediction results.

[0016] Considering that the meteorological conditions that affect the power generation performance of wind turbines (such as wind speed, wind direction, temperature, etc.) have obvious periodicity, for example, similar meteorological conditions may exist in the same time period of a day, the same day of a year, and the same month of a year, and these meteorological conditions are important factors affecting wind power, the task is enhanced through the periodic embedding layer so that the model can learn the periodic changes of the data; the operation of the periodic embedding layer is expressed as: (1) In the formula, is the feature of the nth task after periodic embedding, is the wind power data sequence of the nth task, , , They are intra-day embedding vector, daily embedding vector and monthly embedding vector, represents the weight vector, represents the bias, represents the GELU activation function, Represents a concatenation operation.

[0017] The encoder includes multiple encoder layers (two in this embodiment), the output features of the previous encoder layer are the input features of the next encoder layer, and the encoder layer includes a time feature extraction module, a space feature extraction module and a time-space feature fusion module.

[0018] The temporal feature extraction module is used to extract the long-term temporal dependency between data, and includes multiple temporal feature extraction layers (three in this embodiment). The output feature of the previous temporal feature extraction layer is the input feature of the next temporal feature extraction layer. In the temporal feature extraction layer, the input feature is padded by dilated convolution to keep the sequence length unchanged during convolution. The padding size is expressed as: (2) In the formula, Indicates the padding size, represents the expansion factor, represents the convolution kernel size of the dilated convolution; in this embodiment, the dilation factors of the three temporal feature extraction layers are 1, 2, and 4 respectively; The padded features are subjected to four convolution operations with kernel sizes of 1×2, 1×3, 1×5, and 1×6 to obtain the features , , and ; for features , , and Perform cropping to align the four features and obtain the cropping feature , , and ; Then use formula (3) to cut the features , , and Processing is performed to obtain the output features of the time feature extraction layer; (3) In the formula, is the output feature of the i-th temporal feature extraction layer, represents the batch normalization operation, represents the random dropout operation, It is the input feature of the temporal feature extraction layer; The output features of the last time feature extraction layer are activated and then residually connected with the input features of the time feature extraction module to obtain the time feature. , expressed as: (4) In the formula, is the output feature of the mth temporal feature extraction layer, Represents the ReLU activation function.

[0019] There is a spatial dependency between wind turbines, which will affect the power of wind turbines. On the one hand, different wind turbines may have similar meteorological conditions such as wind conditions, temperature and atmospheric pressure due to their close spatial locations and similar geographical environments, and these meteorological conditions are important factors affecting power; on the other hand, the power generation performance of wind turbines is affected by the wake effect. The wake effect means that when wind turbines are geographically adjacent, the wake generated by the operation of the front row of wind turbines will cause the inflow wind speed of the rear row of wind turbines to decrease, further affecting the power output of the rear row of wind turbines. Therefore, the spatial feature extraction module is used to extract the spatial dependency between data. In the spatial feature extraction module, the parameterized adjacency matrix and the dynamic adjacency matrix are first constructed to represent the inherent spatial relationship and the dynamically changing spatial relationship between wind turbines, respectively; (5) (6) In the formula, represents the parameterized adjacency matrix, represents the dynamic adjacency matrix, represents the normalization operation, is the learnable parameter matrix, , represents the learnable parameter matrix; Then, the parameterized adjacency matrix , dynamic adjacency matrix As well as the input features of the spatial feature extraction module, the features of the aggregation node and its neighboring nodes in the graph convolutional network GCN are input to obtain the spatial features, that is, the output features of the spatial feature extraction module, which are expressed as: (7) (8) In the formula, Represents spatial features, represents the graph convolution operation, Represents the RMS layer normalization operation.

[0020] The temporal features and spatial features are input into the spatiotemporal feature fusion module for fusion to capture more potential patterns and trends between data and obtain the spatiotemporal fusion features, which are the output features of the encoder layer. (9) (10) In the formula, It is the time-space fusion feature. represents a linear operation, Representing Hadamard, Represents the Sigmoid activation function.

[0021] The content matching storage network is used to learn typical features in the data and integrate the spatiotemporal features extracted by the encoder. Input into the content matching storage network to match with the memory node feature vector; first, convert the spatiotemporal fusion features extracted by the encoder into a query vector, then calculate the similarity between the query vector and the memory node feature vector, use the similarity as the weight of the memory node feature vector, perform weighted summation on all memory node feature vectors, and obtain the matching node feature vector , the process is expressed as: (11) (12) (13) In the formula, represents the query vector, is the spatiotemporal fusion feature extracted by the encoder, represents the weight matrix, represents the bias, represents the similarity between the query vector and the feature vector of the jth memory node, represents the jth memory node feature vector, Represents the number of memory node feature vectors.

[0022] The initial memory node feature vector is randomly initialized and updated during the model training process. The updated memory node feature vector learns the typical features in the data. And the spatiotemporal fusion features extracted by the encoder The prediction layer is used to predict the wind power and obtain the wind power prediction value.

[0023] Step 3: Train the wind power prediction model based on meta-learning; The task model and wind power prediction model are trained on the source domain dataset. The task model and the wind power prediction model have the same network architecture, but different model parameters. The support set and query set of the source domain task are input into the task model in turn for training, and the parameters of the task model are updated through gradient reverse; the parameters of the wind power prediction model are updated using the parameters of the task model; after the task model completes the training of a source domain task, the wind power prediction model must be updated using the parameters of the task model until all source domain task training is completed, and the wind power prediction model after source domain training can be obtained; The performance of the trained model is evaluated using the mean absolute error (MAE) and root mean square error (RMSE). When both the mean absolute error and the root mean square error are minimized, the model performance is optimal. (14) (15) In the formula, represents the number of tasks in the domain, Represents the number of samples in the query set in this task, , They represent the true value and predicted value of wind power of the u-th sample in the v-th task respectively.

[0024] Step 4: Migrate the wind power prediction model trained in the source domain to the target domain, use the support set of the target domain task to train the wind power prediction model trained in the source domain, and then fine-tune the parameters of the wind power prediction model trained in the source domain to obtain the wind power prediction model trained in the target domain; use the wind power prediction model trained in the target domain for the prediction of the target domain task, that is, input the query set of the target domain task into the wind power prediction model trained in the target domain to obtain the prediction result.

[0025] Example This embodiment selects three public datasets, SDWPF, GEFCom2014, and Engie, as source domain datasets. The target domain task is a proprietary dataset. The target domain is considered to contain ten days of historical data, and the wind power of the past 36 time steps (6 hours) is used to predict the wind power of the next 12 steps (2 hours).

[0026] Under the same conditions, the diffusion convolutional recurrent neural network (DCRNN, see Li Y, Yu R, Shahabi C, et al. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting [J]. 2017. DOI: 10.48550 / arXiv.1707.01926.) without transfer learning, the transfer learning method based on CNN-LSTM (CNN-LSTM-TL) and the wind power prediction model of the present invention are compared. Among them, the comparison of mean absolute error is shown in Figure 2 , see RMS error comparison Figure 3 , Figure 4 It is a comparison chart of the prediction results of the present invention.

[0027] from Figure 2 and Figure 3 It can be seen that the MAE and RMSE of the wind power prediction model of the present invention are lower than those of the other two models, and it has better prediction effect. The DCRNN model adopts non-transfer learning technology and needs to rely on a large amount of training data with the same source as the target domain task. However, the target domain task comes from a newly built wind farm, and the data samples are limited, resulting in poor model training effect. The present invention uses a meta-learning training strategy combined with content matching to train the model on the source domain data set, so that the model learns the prior knowledge of various source domain tasks and has the ability to quickly adapt to new tasks. Therefore, a small amount of data can be used in the target domain to achieve a good prediction effect. The CNN-LSTM-TL model improves the prediction effect of the model compared to DCRNN through transfer learning, but CNN-LSTM-TL only extracts features from the time dimension and does not consider the spatial correlation between wind turbines. The wind power prediction model of the present invention fully considers the inherent and dynamically changing spatial relationship between wind turbines, fully extracts spatial features, and further improves the prediction performance. Any matters not described in the present invention are applicable to the prior art.

Claims

1. A spatiotemporal wind power prediction method based on meta-learning and content matching, characterized in that: The following steps are involved: Step 1: Sample data from multiple source domain wind farms to obtain a source domain data set formed by multiple source domain tasks; obtain data from the target wind farm as the target domain task; Step 2: construct a wind power prediction model; the wind power prediction model includes a periodic embedding layer, an encoder, a content matching storage network and a prediction layer; The encoder comprises a plurality of encoder layers, each of which comprises three modules: temporal feature extraction, spatial feature extraction and spatiotemporal feature fusion. In the spatial feature extraction module, a parameterized adjacency matrix is ​​first constructed by equations (5) and (6): and the dynamic adjacency matrix : （5） （6） In the formula, represents the normalization operation, represents the ReLU activation function, represents the learnable parameter matrix, represents the matrix transpose, represents the features of the nth task after periodic embedding, , represents the learnable parameter matrix; Then, the parameterized adjacency matrix , dynamic adjacency matrix And the input features of the spatial feature extraction module are input into the graph convolutional network to extract spatial features. The process is expressed as: （7） （8） In the formula, Represents spatial features, represents the graph convolution operation, represents the root mean square layer normalization operation; The spatiotemporal fusion features extracted by the encoder Input into the content matching storage network for information matching to obtain the matching node feature vector , the process is expressed as: （11） （12） （13） In the formula, represents the query vector, represents the weight matrix, Indicates bias, Represents the query vector and the feature vector of the jth memory node The similarity between Represents the number of memory node feature vectors; Match the node feature vector And the spatiotemporal fusion features extracted by the encoder Input into the prediction layer for prediction and output the prediction result; Step 3: Train the wind power prediction model based on meta-learning; The task model and the wind power prediction model have the same network architecture but different parameters. The task model is trained and updated using the source domain data set, and the parameters of the wind power prediction model are updated using the parameters of the task model to obtain the wind power prediction model after source domain training. Step 4: Use the target domain task to train the wind power prediction model trained in the source domain to obtain the wind power prediction model trained in the target domain; use the wind power prediction model trained in the target domain for the prediction of the target domain task.

2. The spatiotemporal wind power prediction method based on meta-learning and content matching according to claim 1 is characterized in that: The periodic embedding layer is expressed as: （1） In the formula, is the wind power data sequence of the nth task, , , They are intra-day embedding vector, daily embedding vector and monthly embedding vector, represents the weight vector, Indicates bias, represents the GELU activation function, Represents a concatenation operation.

3. The spatiotemporal wind power prediction method based on meta-learning and content matching according to claim 1 or 2, characterized in that: The temporal feature extraction module includes multiple temporal feature extraction layers. In each temporal feature extraction layer, the input feature is filled by dilated convolution. The filled feature is respectively subjected to four convolution operations with different convolution kernel sizes to obtain the feature. , , and ; By cutting out the features , , and Align and get the clipping features , , and ; Then, the clipping feature is calculated by formula (3). , , and Processing is performed to obtain the output features of the time feature extraction layer; （3） In the formula, is the output feature of the i-th temporal feature extraction layer, represents the batch normalization operation, represents the random dropout operation, It is the input feature of the temporal feature extraction layer; Output features of the last temporal feature extraction layer After the activation operation, it is added to the input features of the time feature extraction module to obtain the time feature , the process is expressed as: （4） Where m represents the number of temporal feature extraction layers.

4. The spatiotemporal wind power prediction method based on meta-learning and content matching according to claim 3 is characterized in that: The spatiotemporal feature fusion module fuses the temporal features and spatial features to obtain spatiotemporal fusion features. The process is expressed as follows: （9） （10） In the formula, It is the time-space fusion feature. represents a linear operation, Representing Hadamard, Represents the Sigmoid activation function.

5. The spatiotemporal wind power prediction method based on meta-learning and content matching according to claim 4 is characterized in that: The encoder includes two encoder layers, and the temporal feature extraction module includes three temporal feature extraction layers.

Images