Wind power prediction method based on pre-training model and low-rank self-adaption

Abstract

The invention discloses a wind power prediction method based on a pre-training model and low-rank self-adaption, and relates to the technical field of wind power generation, and the method comprises the steps: obtaining historical meteorological data and numerical values of a wind power plant, obtaining weather forecast data, and obtaining historical operation data and numerical weather forecast data of a wind motor; cleaning and normalizing the acquired data, dividing historical meteorological data and historical operation data into a training set and a test set, selecting a pre-training model according to a wind power prediction demand, and training the pre-training model by using the training set. According to the method, the pre-training model is combined with low-rank self-adaption, so that the model can accurately capture the change rule of wind power, prediction errors are reduced, prediction precision is improved, low-rank decomposition is carried out on a model parameter matrix, the parameter quantity of model training is reduced, calculation complexity is reduced, rapid training and real-time prediction of the model are facilitated, and the method is suitable for large-scale popularization and application. The stability of the power system of the wind power plant is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power generation, and in particular to a wind power prediction method based on a pre-training model and low-rank adaptation. Background Art

[0002] In the field of wind power technology, accurate prediction of wind power is of great significance for the operation and management of wind farms and the dispatch optimization of power systems. Traditional wind power prediction methods often rely on statistical models or simple physical models. These methods have many limitations when dealing with complex and changeable meteorological conditions and wind farm operation data, resulting in insufficient prediction accuracy. With the rapid development of big data and artificial intelligence technologies, methods based on machine learning, especially deep learning, have shown great potential in wind power prediction. Deep learning models can automatically extract complex feature relationships from massive data, thereby improving prediction accuracy. However, traditional deep learning models are prone to overfitting and large computational complexity during training, and have poor adaptability to changes in real-time data, making it difficult to maintain stable prediction performance in a complex and changeable wind power environment. In this regard, we propose a wind power prediction method based on pre-training models and low-rank adaptation. Summary of the invention

[0003] In order to solve the above technical problems, a wind power prediction method based on a pre-trained model and low-rank adaptation is provided. This technical solution solves the above problems.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0005] Wind power prediction method based on pre-trained model and low-rank adaptation, including:

[0006] Obtain historical meteorological data and values ​​of wind farms, obtain weather forecast data, obtain historical operation data of wind turbines and numerical weather forecast data;

[0007] Clean and normalize the acquired data, and divide the historical meteorological data and historical operation data into training sets and test sets;

[0008] Select a pre-trained model based on the wind power forecast demand, and train the pre-trained model using the training set data;

[0009] The parameter matrix in the pre-trained model is decomposed into a low-rank matrix. During the model adaptive training process, the parameters after the low-rank decomposition are adaptively adjusted according to the real-time data.

[0010] Input numerical weather forecast data and operation data into the model, and the model calculates the predicted value of wind power based on the input data and outputs the predicted result;

[0011] Use the test set data to evaluate the model prediction results and adjust the model based on the evaluation results;

[0012] Deploy the trained model to the wind farm to achieve real-time prediction of wind power.

[0013] Preferably, the method for obtaining the historical meteorological data and numerical weather forecast data of the wind farm and the historical operation data and numerical weather forecast data of the wind turbine is:

[0014] Historical meteorological data of wind farms: through the meteorological data platform, select the geographical location of the wind farm, enter the start and end time of the query, select the required meteorological data, and download and export;

[0015] Numerical weather forecast data: through the meteorological data platform, select the geographical location of the wind farm and obtain the required numerical weather forecast data in real time;

[0016] Historical operation data of wind turbines: select the wind turbines to be queried through the data management system of the wind farm, enter the start and end time of the query, obtain the historical operation data within the time period and export it. The historical operation data includes power generation, power output and operation status;

[0017] Real-time operation data of wind turbines: the operation status and data of wind turbines are obtained in real time through the monitoring system of the wind farm.

[0018] Preferably, the pre-trained model is selected based on a transformer network with an attention mechanism according to the wind power prediction demand, wherein the mathematical representation of the attention mechanism is:

[0019]

[0020] Where Q, K and V represent query, key and value matrices respectively, softmax() is the function, T represents the scaling factor, and d k The dimension of the vector representing the key K;

[0021] Use the deep learning framework to build a pre-trained model and use the training set data to train the pre-trained model;

[0022] During the training process, an algorithm is used to update the network weights, where the calculation method for updating the network weights is:

[0023]

[0024] In the formula, w t+1 represents the network weight w at the t+1th iteration, w t represents the network weight w at the tth iteration, α and β are the control parameters of the algorithm, represents the gradient of the network weight w at the tth iteration

[0025] Preferably, the loss functions of the pre-trained model are root mean square error and mean absolute error;

[0026] Among them, the loss function root mean square error calculation formula is:

[0027]

[0028] In the formula, RMSE represents the difference between the predicted value and the actual value, n represents the total number of samples, and y i represents the actual value y of the i-th sample, Represents the predicted value of the i-th sample

[0029] Among them, the loss function mean absolute error calculation formula is:

[0030]

[0031] In the formula, MAE represents the absolute value of the difference between the predicted value and the actual value;

[0032] The loss function is used to measure model performance, guide model training and optimization, and monitor overfitting.

[0033] Preferably, the low-rank decomposition selects a suitable intermediate layer dimension R:

[0034] Based on the pre-trained model, the query (Q) and key (K) layer weight matrices of the model's attention mechanism layer are selected for low-rank decomposition;

[0035] Decompose a matrix into the product of two matrices, that is, a layer with D*D parameters, into a layer with D*R parameters and a layer with R*D parameters. Then the number of parameters to be trained changes from D*D to 2*R*D (R is much smaller than D).

[0036] The output of each low-rank decomposition layer is superimposed on the output of the corresponding layer in the original model, and the performance of the model superimposed with the low-rank adaptive layer is evaluated using the test set.

[0037] Preferably, the method for adaptively adjusting the parameters after low-rank decomposition according to real-time data is:

[0038] During the model prediction process, continuously monitor the performance of real-time data, including prediction accuracy and loss function value;

[0039] Update the parameter weights after low-rank decomposition. For the newly arrived real-time data, treat it as incremental data and only update the model locally.

[0040] Dynamically adjust the weight distribution of the corresponding layers in the attention mechanism according to the feature distribution and importance of real-time data;

[0041] Use the transformer model to process the normalized features of the input;

[0042] During the adaptive adjustment process, the performance of the model is continuously monitored and the strategy is adjusted based on the feedback results.

[0043] Preferably, the method for low-rank decomposition of the matrix is:

[0044] Select the query (Q) and key (K) layer weight matrices of the model's attention mechanism layer for low-rank decomposition;

[0045] For the D*D parameter layer to be decomposed, select an appropriate intermediate layer dimension R and decompose it into a D*R parameter layer and an R*D parameter layer;

[0046] Initialize the weight parameter value of the D*R layer through the Gaussian function, and initialize the weight parameter value of the R*D layer to zero;

[0047] During training, the weight parameters of the pre-trained model are frozen, and only the weight parameters of the low-rank decomposition layer are adjusted.

[0048] Preferably, the wind power prediction method is deployed on a cloud platform to realize the rights of remote monitoring and prediction services, and users remotely access and use the prediction services through the Internet.

[0049] Preferably, the model parameters are dynamically adjusted through the newly collected wind power data, and the prediction model is updated in real time and learned online.

[0050] Preferably, during the data preprocessing stage, the prediction capability of the model is improved by combining multi-source data including neighboring wind farms, weather stations and satellite images.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention proposes a combination of a pre-trained model and low-rank adaptation, so that the model can more accurately capture the changing law of wind power, reduce prediction errors and improve prediction accuracy. By performing low-rank decomposition on the model parameter matrix, the amount of model parameters is reduced and the calculation complexity is reduced, which is conducive to rapid training and real-time prediction of the model. The parameters after low-rank decomposition are adaptively adjusted according to real-time data, so that the model can better adapt to new data and environmental changes, and the generalization ability of the model is improved. By deploying the model on a cloud platform, users can remotely access and use prediction services through the Internet, which is convenient and fast, and realizes remote monitoring and prediction of wind power, which is of great significance to improving the operating efficiency of wind farms and the stability of power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The flowchart of the wind power prediction method based on pre-trained model and low-rank adaptation. DETAILED DESCRIPTION

[0054] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0055] Reference Figure 1 As shown, the wind power prediction method based on the pre-trained model and low-rank adaptation includes:

[0056] Through the meteorological data platform, according to the specific geographical location of the wind farm, the start and end time range of the query is set to accurately obtain historical meteorological data including wind speed, wind direction, temperature, and humidity, real-time updated meteorological data, and weather forecast data. The data management system and monitoring system of the wind farm provide detailed operating data of the wind turbine, including historical power generation, power output curves, and various operating status information. These data are crucial for understanding the performance changes of the wind turbine. Real-time operating data is captured in real time through the monitoring system to ensure that the prediction model can make predictions based on the latest information;

[0057] The collected raw data often contain noise, missing values ​​or outliers, which will affect the performance of the prediction model. Therefore, in the data preprocessing stage, the data is first cleaned to remove invalid or erroneous data points, fill reasonable missing values, and smooth outliers. Then, in order to eliminate the influence of different dimensions on model training, all data are converted to the same scale using normalization technology. Finally, the processed historical meteorological data and historical operation data are divided into training sets and test sets for subsequent model training and evaluation.

[0058] Select a pre-trained model based on the wind power forecast demand, and train the pre-trained model using the training set data;

[0059] In order to further improve the prediction efficiency of the model and reduce the computational complexity, the parameter matrix of the corresponding layer in the pre-trained model is decomposed into low rank. During the model prediction process, the parameters after low rank decomposition are adaptively adjusted according to real-time data.

[0060] Input numerical weather forecast data and operation data into the trained model. The model calculates the predicted value of wind power based on the input data. The prediction results are output to wind farm managers or power system dispatchers through the user interface or API interface to provide them with decision support.

[0061] Use the test set data to comprehensively evaluate the model prediction results. By comparing the differences between the actual values ​​and the predicted values, analyze the model's prediction accuracy and error distribution. Based on the evaluation results, further adjust and optimize the model, including adjusting the network structure, optimizing hyperparameters, and introducing new features to improve the model's prediction performance.

[0062] The trained and fully evaluated model is deployed in the wind farm. By inputting weather forecast data and operation data in real time, real-time prediction of wind power can be achieved. At the same time, the prediction system is connected to the cloud platform to realize remote monitoring and prediction services. Users can remotely access and use the prediction service through the Internet to obtain the latest wind power prediction results.

[0063] The method for obtaining the historical meteorological data and numerical weather forecast data of the wind farm and the historical operation data and numerical weather forecast data of the wind turbine is as follows:

[0064] Historical meteorological data of wind farms: select the geographical location of the wind farm through the meteorological data platform, enter the longitude and latitude of the wind farm accurately or select the corresponding geographical location on the map in the platform's geographic positioning tool, enter the start and end time of the query, and this time range should cover the key period of wind farm operation, so as to analyze the wind power output under different seasons and weather conditions, select the required meteorological data including but not limited to wind speed, wind direction, temperature, humidity, air pressure and precipitation, and download and export;

[0065] Numerical weather forecast data: select the geographical location of the wind farm through the meteorological data platform to obtain the required numerical weather forecast data in real time. The meteorological data platform usually provides a real-time data interface to allow users to obtain the latest meteorological data in real time through programming. In order to maintain the real-time nature of the data, write scripts or programs to automatically poll the meteorological data platform through the API regularly to obtain the latest numerical weather forecast data;

[0066] Historical operation data of wind turbines: through the data management system inside the wind farm, enter the user name and password to log in to the system. On the system interface, according to the layout of the wind farm and the unit number, select the wind turbine to be queried, enter the start and end time of the query to obtain the historical operation data within the time period, obtain the historical operation data within the time period and export it. The historical operation data includes power generation, power output and operation status;

[0067] Real-time operation data of wind turbines: The operation status and data of wind turbines can be obtained in real time through the monitoring system of the wind farm. The monitoring system has an intuitive interface and powerful data processing capabilities. In the monitoring system, various operating parameters and operating status of wind turbines can be viewed in real time.

[0068] According to the wind power forecast demand, the pre-trained model is selected based on the transformer network with attention mechanism, where the mathematical representation of the attention mechanism is:

[0069]

[0070] Where Q, K and V represent query, key and value matrices respectively, softmax() is the function, T represents the scaling factor, and d k The dimension of the vector representing the key K;

[0071] Use the deep learning framework to build a pre-trained model and use the training set data to train the pre-trained model;

[0072] During the training process, an algorithm is used to update the network weights, where the calculation method for updating the network weights is:

[0073]

[0074] In the formula, w t+1 represents the network weight w at the t+1th iteration, w t represents the network weight w at the tth iteration, α and β are the control parameters of the algorithm, represents the gradient of the network weight w at the tth iteration

[0075] The loss functions of the pre-trained model are root mean square error and mean absolute error;

[0076] Among them, the loss function root mean square error calculation formula is:

[0077]

[0078] In the formula, RMSE represents the difference between the predicted value and the actual value, n represents the total number of samples, and y i represents the actual value y of the i-th sample, Represents the predicted value of the i-th sample

[0079] Among them, the loss function mean absolute error calculation formula is:

[0080]

[0081] In the formula, MAE represents the absolute value of the difference between the predicted value and the actual value;

[0082] The loss function is used to measure model performance, guide model training and optimization, and monitor overfitting.

[0083] The low-rank decomposition needs to select a suitable intermediate layer dimension R;

[0084] Based on the pre-trained model, the query (Q) and key (K) layer weight matrices of the attention mechanism layer of the selection model are low-rank decomposed. The weight matrix occupies a large amount of storage space and computing resources. In order to optimize these models, the weight matrix of the key layer is low-rank decomposed. In addition to avoiding the model forgetting effect caused by training, the model adaptive learning is also affected by changes in the wind farm climate environment and equipment operating conditions.

[0085] Decompose a matrix into the product of two matrices, that is, a layer with D*D parameters, into a layer with D*R parameters and a layer with R*D parameters. Then the number of parameters to be trained changes from D*D to 2*R*D (R is much smaller than D).

[0086] The output of each low-rank decomposition layer is superimposed on the output of the corresponding layer in the original model, and the test set is used to evaluate the performance of the model with superimposed low-rank adaptive layers. The evaluation indicators include prediction accuracy, root mean square error, and mean absolute error. By comparing the model performance before and after replacement, it is determined whether the low-rank decomposition effectively reduces the training complexity of the model without significantly sacrificing prediction accuracy.

[0087] The method for adaptively adjusting the parameters after low-rank decomposition according to real-time data is:

[0088] In the process of model prediction, we first need to establish a system that continuously monitors the performance of real-time data, including prediction accuracy and loss function value, so as to quickly evaluate the performance of the model in the current data environment. These indicators should be updated regularly and stored in historical records for subsequent analysis and comparison;

[0089] Update the parameter weights after low-rank decomposition. For the newly arrived real-time data, do not directly retrain the entire model, because this is time-consuming and may cause the model to forget the old data. Instead, treat it as incremental data and use incremental learning to locally update the model.

[0090] Dynamically adjust the weight distribution of the attention mechanism according to the feature distribution and importance of real-time data. The attention mechanism plays a key role in the model, allowing the model to pay more attention to important input features when predicting;

[0091] The transformer model is used to process the normalized features of the input. Through the position encoding information and attention mechanism, the transformer can capture the dependencies of these features in the time series, and the input features can be weighted by combining the attention mechanism.

[0092] During the adaptive adjustment process, the performance of the model is continuously monitored, which includes regularly evaluating key indicators such as the model's prediction accuracy and loss function value, comparing them with historical records, and adjusting the strategy based on the feedback results.

[0093] The method for low-rank decomposition of the matrix is:

[0094] Select the query (Q) and key (K) layer weight matrices of the model's attention mechanism layer for low-rank decomposition;

[0095] For the D*D parameter layer to be decomposed, select an appropriate intermediate layer dimension R and decompose it into a D*R parameter layer and an R*D parameter layer;

[0096] Initialize the weight parameter value of the D*R layer through the Gaussian function, and initialize the weight parameter value of the R*D layer to zero;

[0097] During training, the weight parameters of the pre-trained model are frozen, and only the weight parameters of the low-rank decomposition layer are adjusted.

[0098] Deploying wind power prediction methods on a cloud platform is a key step to improve their scalability, accessibility, and efficiency. This deployment model not only enables remote monitoring and real-time data processing of prediction services, but also provides users with a flexible and convenient access method. No matter where they are, they can enjoy professional wind power prediction services through the Internet. By remotely accessing and using wind power prediction services through the Internet, users can enjoy great convenience. Whether in the office, at home, or on a mobile device, they only need to connect to the Internet and log in to the cloud platform to view wind power prediction results, monitor wind farm operation conditions, adjust prediction parameters, etc. anytime and anywhere. This flexible access method not only improves work efficiency, but also enhances user experience and satisfaction.

[0099] In the field of wind power forecasting, using newly collected wind power data to dynamically adjust model parameters, update the prediction model in real time and realize online learning is a key strategy to improve prediction accuracy and adapt to the dynamic changes of wind farms. This process not only requires the model to be able to respond quickly to the arrival of new data, but also requires an efficient learning mechanism to optimize its own performance.

[0100] During the data preprocessing phase, the model’s predictive capabilities are enhanced by combining multi-source data including nearby wind farms, weather stations, and satellite images.

[0101] The use process of the present invention is as follows: collect historical and real-time meteorological and operating data of wind farms, preprocess the data to clean noise and normalize it, select the transformer network with attention mechanism as the pre-trained model and train it, optimize the model parameters through low-rank decomposition, use real-time data to adaptively adjust the model, evaluate the model and deploy it to the wind farm, realize remote monitoring and online learning, and integrate multi-source data to improve prediction capabilities.

[0102] In summary, the advantages of the present invention are: improving prediction accuracy, reducing computational complexity, and enhancing the adaptability of the model. Through the transformer model with attention mechanism, this method effectively captures the complex influencing factors of wind power, and at the same time uses a low-rank adaptive mechanism to optimize parameters, thereby reducing the number of parameters for model training and improving prediction speed and accuracy. In addition, it supports remote monitoring, online learning, and real-time updates, and uses multi-source data to enhance prediction capabilities, which is of great significance to the operating efficiency of wind farms and the stability of power systems.

[0103] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A wind power prediction method based on a pre-trained model and low-rank adaptation, characterized in that: include: Obtain historical meteorological data and values ​​of wind farms, obtain weather forecast data, obtain historical operation data of wind turbines and numerical weather forecast data; Clean and normalize the acquired data, and divide the historical meteorological data and historical operation data into training sets and test sets; Select a pre-trained model based on the wind power forecast demand, and train the pre-trained model using the training set data; The parameter matrix in the pre-trained model is decomposed into a low-rank matrix. During the model adaptive training process, the parameters after the low-rank decomposition are adaptively adjusted according to the real-time data. Input numerical weather forecast data and operation data into the model, and the model calculates the predicted value of wind power based on the input data and outputs the predicted result; Use the test set data to evaluate the model prediction results and adjust the model based on the evaluation results; Deploy the trained model to the wind farm to achieve real-time prediction of wind power.

2. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 1 is characterized in that: The method for obtaining the historical meteorological data and numerical weather forecast data of the wind farm and the historical operation data and numerical weather forecast data of the wind turbine is as follows: Historical meteorological data of wind farms: through the meteorological data platform, select the geographical location of the wind farm, enter the start and end time of the query, select the required meteorological data, and download and export; Numerical weather forecast data: through the meteorological data platform, select the geographical location of the wind farm and obtain the required numerical weather forecast data in real time; Historical operation data of wind turbines: select the wind turbines to be queried through the data management system of the wind farm, enter the start and end time of the query, obtain the historical operation data within the time period and export it. The historical operation data includes power generation, power output and operation status; Real-time operation data of wind turbines: the operation status and data of wind turbines are obtained in real time through the wind farm monitoring system.

3. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 1 is characterized in that: According to the wind power forecast demand, the pre-trained model is selected based on the transformer network with attention mechanism, where the mathematical representation of the attention mechanism is: Where Q, K and V represent query, key and value matrices respectively, softmax() is the function, T represents the scaling factor, and d k The dimension of the vector representing the key K; Use the deep learning framework to build a pre-trained model and use the training set data to train the pre-trained model; During the training process, an algorithm is used to update the network weights, where the calculation method for updating the network weights is: In the formula, w t+1 represents the network weight w at the t+1th iteration, w t represents the network weight w at the tth iteration, α and β are the control parameters of the algorithm, represents the gradient of the network weight w at the tth iteration 4. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 3 is characterized in that: The loss functions of the pre-trained model are root mean square error and mean absolute error; Among them, the loss function root mean square error calculation formula is: In the formula, RMSE represents the difference between the predicted value and the actual value, n represents the total number of samples, and y i represents the actual value y of the i-th sample, Represents the predicted value of the i-th sample Among them, the loss function mean absolute error calculation formula is: In the formula, MAE represents the absolute value of the difference between the predicted value and the actual value; The loss function is used to measure model performance, guide model training and optimization, and monitor overfitting.

5. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 1 is characterized in that: The low-rank decomposition selects an appropriate intermediate layer dimension R: Based on the pre-trained model, the query (Q) and key (K) layer weight matrices of the model's attention mechanism layer are selected for low-rank decomposition; Decompose a matrix into the product of two matrices, that is, a layer with D*D parameters, into a layer with D*R parameters and a layer with R*D parameters. Then the number of parameters to be trained changes from D*D to 2*R*D (R is much smaller than D). The output of each low-rank decomposition layer is superimposed on the output of the corresponding layer in the original model, and the performance of the model superimposed with the low-rank adaptive layer is evaluated using the test set.

6. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 1 is characterized in that: The method for adaptively adjusting the parameters after low-rank decomposition according to real-time data is: During the model prediction process, continuously monitor the performance of real-time data, including prediction accuracy and loss function value; Update the parameter weights after low-rank decomposition. For the newly arrived real-time data, treat it as incremental data and only update the model locally. Dynamically adjust the weight distribution of the corresponding layers in the attention mechanism according to the feature distribution and importance of real-time data; Use the transformer model to process the normalized features of the input; During the adaptive adjustment process, the performance of the model is continuously monitored and the strategy is adjusted based on the feedback results.

7. The method for wind power prediction based on pre-training model and low-rank adaptation according to claim 5, characterized in that: The method for low-rank decomposition of the matrix is: Select the query (Q) and key (K) layer weight matrices of the model's attention mechanism layer for low-rank decomposition; For the D*D parameter layer to be decomposed, select an appropriate intermediate layer dimension R and decompose it into a D*R parameter layer and an R*D parameter layer; Initialize the weight parameter value of the D*R layer through the Gaussian function, and initialize the weight parameter value of the R*D layer to zero; During training, the weight parameters of the pre-trained model are frozen, and only the weight parameters of the low-rank decomposition layer are adjusted.

8. The method for wind power prediction based on pre-training model and low-rank adaptation according to claim 1 is characterized in that: The wind power prediction method is deployed on a cloud platform to realize the rights of remote monitoring and prediction services, and users can remotely access and use the prediction services through the Internet.

9. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 1 is characterized in that: Dynamically adjust model parameters through newly collected wind power data, update the prediction model in real time and learn online.

10. The wind power prediction method based on pre-training model and low-rank adaptation according to claim 1, characterized in that: During the data preprocessing phase, the model’s predictive capabilities are enhanced by combining multi-source data including nearby wind farms, weather stations, and satellite images.

Images