Photovoltaic power prediction method based on pre-training model

Abstract

The invention relates to a photovoltaic power prediction method based on a pre-training model, and the method comprises the steps: obtaining the historical power data and regional historical meteorological data of a photovoltaic power station in a certain region, screening related data, and carrying out the normalization of the data; dividing a pre-training data set and a training data set, and extracting corresponding feature vectors; pre-training a GPT photovoltaic power prediction model by using historical non-prediction cycle data; finely tuning the pre-trained GPT photovoltaic power prediction model by using the training set data; and predicting a power value on the test data set by using the pre-trained GPT photovoltaic power prediction model after fine tuning. According to the technical scheme provided by the invention, the model is pre-trained on a large amount of historical power related data to predict the photovoltaic power value in the prediction period, so that the accuracy of photovoltaic power prediction is improved, a scheduling department is helped to make a scheduling plan and avoid risks in advance, and the safety of a power system and the competitiveness of photovoltaic power generation are improved.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic power prediction, and in particular to a photovoltaic power prediction method based on a pre-training model. Background Art

[0002] In recent years, photovoltaic power generation has been continuously developing and innovating, and has become a viable renewable energy source. Photovoltaic power generation is closely related to weather factors, and there will be seasonal and diurnal fluctuations, especially changes in light intensity have a direct impact on photovoltaic power generation; therefore, accurate prediction of photovoltaic power generation is of great significance.

[0003] After years of research and development, there are many works on photovoltaic power prediction, which can be roughly divided into four types: physical methods, statistical methods, deep learning methods, and hybrid methods. The physical method is to combine numerical weather forecasts with geographical meteorological information to predict photovoltaic power; the statistical method is to predict photovoltaic power by establishing a mapping relationship between meteorological and other related information and photovoltaic power; deep learning methods are commonly used in convolutional neural networks (CNN) and long short-term neural networks (LSTM); hybrid methods mainly include weather classification hybrid prediction models based on clustering algorithms and hybrid prediction models based on signal decomposition.

[0004] At present, most PV power forecasting work adopts statistical methods, but there are great defects. On the one hand, the prediction algorithm does not give full play to the role of historical data, ignores the reference value of historical data, lacks pre-training, and does not make the algorithm model parameters more suitable for the field of PV power forecasting; on the other hand, the ordinary statistical method is not accurate enough for simple power forecasting, and does not fully consider the nonlinear time series characteristics, so the prediction accuracy is not high enough. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the problems and defects existing in the prior art, including: failure to adjust the model parameters so that the model is applicable to the field of photovoltaic power prediction; failure to fully mine the information of historical data and failure to adopt a pre-training method; and failure of the algorithm model to fully consider the nonlinear time series characteristics of photovoltaic power. A photovoltaic power prediction method based on a pre-training model is provided, the method comprising:

[0006] Obtain a large amount of historical power data of photovoltaic power stations in a certain area and historical regional meteorological data;

[0007] Extract feature vectors corresponding to a large amount of historical power data and meteorological data in non-forecast periods;

[0008] The feature vector is input into the initial GPT photovoltaic power prediction model, first passes through the Masked Multi-HeadAttention layer, then passes through the residual connection and Layer Normalization layer, then passes through the FeedForward layer, and then passes through the second residual connection and Layer Normalization layer, repeats the above hierarchical steps 12 times, and finally passes through the fully connected layer for prediction;

[0009] It is determined whether the Huber loss value of the model is less than the initial threshold. If not, the GPT model parameters are adjusted through the back propagation algorithm and the gradient descent algorithm until the Huber loss value is less than or equal to the preset threshold, and finally the pre-trained GPT photovoltaic power prediction model is obtained.

[0010] Divide the data in the historical forecast period into training sets and test sets, and extract the feature vectors of each type of data;

[0011] The feature vector of the power-related data of the training set is used as the input of the pre-training model;

[0012] After the simplified Decoder layer and the fully connected layer, the output vector is obtained;

[0013] Map the output vector to a specific value, compare the predicted value with the actual value, and calculate the Huber loss;

[0014] Determine whether the Huber loss value of the model is less than the initially set fine-tuning threshold. If not, adjust the GPT model parameters through the back propagation algorithm and the gradient descent algorithm until the Huber loss value is less than or equal to the preset threshold, and finally obtain the fine-tuned GPT photovoltaic power prediction model.

[0015] The feature vector of the power-related data of the test set is used as the input of the fine-tuned pre-trained GPT photovoltaic power prediction model;

[0016] After the simplified Decoder layer and the fully connected layer, the output vector is obtained;

[0017] The output vector is then mapped to a specific numerical value, which is the predicted value of the fine-tuned pre-trained GPT photovoltaic power prediction model.

[0018] The method comprises obtaining historical power data of photovoltaic power stations in a certain area and historical meteorological data of the area, selecting relevant data with a correlation coefficient greater than 0.6, and performing data normalization processing;

[0019] The selected relevant data types include at least the following: photovoltaic power generation, total radiation, direct radiation, scattered radiation, photovoltaic module temperature, solar irradiance, ambient temperature, air pressure, and relative humidity.

[0020] The method for dividing the training set and the test set of the historical data includes: dividing the training set and the test set according to the minimum division module of the day, that is, from 0:00 to 24:00 is defined as the minimum division module, and randomly dividing the training set and the test set according to the ratio of 70% and 30%.

[0021] The method for extracting feature vectors of various types of data includes: converting the acquired training set and test set data into vectors of various types of power-related data, and mapping them to a feature vector of a fixed length using a representation function as the feature vector corresponding to various types of photovoltaic power-related data.

[0022] Compared with the prior art, the method of the present invention has obvious beneficial effects:

[0023] The pre-training method is used to fully train historical power-related data, making the model more applicable to the field of photovoltaic power prediction;

[0024] Use the training set to fine-tune the GPT model to fully mine historical data information;

[0025] The use of advanced GPT models improves the accuracy of photovoltaic power prediction, helps the dispatching department to make dispatch plans and avoid risks in advance, and thus improves the safety of the power system and the competitiveness of photovoltaic power generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a photovoltaic power prediction method based on a pre-training model provided by the present invention;

[0027] Figure 2 It is a schematic diagram of the GPT photovoltaic power prediction model provided by the present invention;

[0028] Figure 3 It is a schematic diagram of the Decoder structure of the GPT photovoltaic power prediction model provided by the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The present invention relates to a photovoltaic power prediction method based on a pre-training model, the method comprising the following steps:

[0031] Step 1: Obtain a large amount of historical power data of photovoltaic power stations in a certain area and historical regional meteorological data; the selected relevant data types include at least the following: photovoltaic power generation, total radiation, direct radiation, scattered radiation, photovoltaic module temperature, solar irradiance, ambient temperature, air pressure, relative humidity. Divide the historical power-related data into pre-training data and training data according to the data in the non-prediction period and the prediction period; for the divided training data, select the relevant data with a correlation coefficient greater than 0.6, and perform data normalization.

[0032] The correlation coefficient calculation formula is as follows:

[0033]

[0034] The normalization formula is as follows:

[0035]

[0036] Step 2: According to the division method of the smallest division module as day, that is, from 0:00 to 24:00 is defined as the smallest division module, the training set and the test set are randomly divided into 70% and 30% ratios; and a large amount of historical power data and meteorological data corresponding to pre-training and training are extracted. Feature vectors.

[0037] The method for extracting feature vectors of various types of data includes: converting the acquired training set and test set data into vectors of various types of power-related data, and mapping them to a feature vector of a fixed length using a representation function as the feature vector corresponding to various types of photovoltaic power-related data.

[0038] Step 3: Input the feature vector of the pre-trained data into the initial GPT photovoltaic power prediction model, first pass through the Masked Multi-Head Attention layer, then pass through the residual connection and Layer Normalization layer, then pass through the FeedForward layer, and then pass through the second residual connection and Layer Normalization layer. Repeat the above hierarchical steps 12 times, and finally pass through the fully connected layer for prediction;

[0039] It is determined whether the Huber loss value of the model is less than the initial threshold. If not, the GPT model parameters are adjusted through the back propagation algorithm and the gradient descent algorithm until the Huber loss value is less than or equal to the preset threshold, and finally the pre-trained GPT photovoltaic power prediction model is obtained.

[0040] Among them, the GPT photovoltaic power prediction model structure is as follows Figure 2 As shown;

[0041] The GPT model builds the following prediction model:

[0042]

[0043] Among them, the Masked Multi-Head Attention layer uses a sequence of query vectors, a sequence of key vectors, and a sequence of value vectors:

[0044] Q=W Q X

[0045] K=W K X

[0046] V=W V X

[0047] The self-attention mechanism used is shown in the following formula:

[0048] X attention =selfAttention(Q,K,V)

[0049] Among them, the residual connection connects the original vector and the weighted vector to save the information of the original vector. The formula is as follows:

[0050] X nidden =X embedding +selfAttention(Q,K,V)

[0051] The Layer Normalization layer normalizes the vector output by the Masked Multi-Head Attention layer to a standard normal distribution, that is, independent and identically distributed, to speed up training and convergence. The formula for calculating the mean is as follows:

[0052]

[0053] The formula for finding the variance is as follows:

[0054]

[0055] The normalization formula is as follows:

[0056]

[0057] The Feed Forward layer projects the vector obtained by Multi-Head Attention into a larger space, then uses the Relu activation function to extract the required information, and finally projects it back to the original space of the token vector. Projecting the information in the token vector into a higher-dimensional space makes it easier to distinguish the various types of information in the vector in the high-dimensional space. The corresponding formula is as follows:

[0058] FFN(x)=ReLU(W1x+b1)W2+b2

[0059] Among them, the fully connected layer maps the high-dimensional vector to a one-dimensional vector to calculate the power prediction value.

[0060] Among them, the Huber loss function combines the advantages of mean absolute error MAE and mean square error MSE, and the calculation formula is as follows:

[0061]

[0062] Among them, the GPT model structure is a linear neural network on top of the Decoder, and the Relu function is used to output the corresponding prediction value. The Decoder model structure is as follows Figure 3 The specific formula is as follows:

[0063] h0=UW e +W p

[0064]

[0065] Step 4: Use the feature vector of the power-related data of the training set as the input of the pre-training model;

[0066] After the simplified Decoder layer and the fully connected layer, the output vector is obtained;

[0067] Map the output vector to a specific value, compare the predicted value with the actual value, and calculate the Huber loss;

[0068] Determine whether the Huber loss value of the model is less than the initially set fine-tuning threshold. If not, adjust the GPT model parameters through the back propagation algorithm and the gradient descent algorithm until the Huber loss value is less than or equal to the preset threshold, and finally obtain the fine-tuned GPT photovoltaic power prediction model.

[0069] Among them, for fine-tuning on a specific task dataset, such as the supervised dataset C, the data is a sequence x 1 ,…,x m And a label y. After inputting the data into the pre-trained model, the output vector Then use the linear regression layer to predict the label value; at the same time, it is necessary to maximize the likelihood function. The specific formula is as follows:

[0070]

[0071] Finally, the objective function of the supervised task model fine-tuned on the basis of the pre-trained model is as follows:

[0072] L3(C)=L2(C)+λ*L1(C)

[0073] Step 5: Use the feature vector of the power-related data of the test set as the input of the fine-tuned pre-trained GPT photovoltaic power prediction model;

[0074] After the simplified Decoder layer and the fully connected layer, the output vector is obtained;

[0075] The output vector is then mapped to a specific numerical value, which is the predicted value of the fine-tuned pre-trained GPT photovoltaic power prediction model.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A photovoltaic power prediction method based on a pre-trained model, characterized in that: The method comprises the steps of: First, obtain the historical power data of photovoltaic power stations in a certain area and the historical meteorological data of the area; Then, the historical data is divided into training set and test set, and the feature vectors of each type of data are extracted; Next, the training set data feature vector is used as the input of the pre-trained GPT photovoltaic power prediction model to obtain the output of the pre-trained GPT photovoltaic power prediction model, that is, the predicted photovoltaic power station power data, and compared with the actual power data value to fine-tune the parameters of the pre-trained GPT photovoltaic power prediction model; Finally, the test set data feature vector is used as the input of the fine-tuned pre-trained GPT photovoltaic power prediction model to obtain the output of the fine-tuned pre-trained GPT photovoltaic power prediction model, that is, the photovoltaic power station power prediction value; Among them, the pre-trained GPT photovoltaic power prediction model is obtained based on various power-related historical data and historical power data of photovoltaic power stations.

2. The method according to claim 1, characterized in that The method comprises obtaining historical power data of photovoltaic power stations in a certain area and historical meteorological data of the area, selecting relevant data with a correlation coefficient greater than 0.6, and performing data normalization processing; The selected relevant data types include at least the following: photovoltaic power generation, total radiation, direct radiation, scattered radiation, photovoltaic module temperature, solar irradiance, ambient temperature, air pressure, and relative humidity.

3. The method according to claim 1, characterized in that The method for dividing the training set and the test set of the historical data includes: dividing the training set and the test set according to the minimum division module of the day, that is, from 0:00 to 24:00 is defined as the minimum division module, and randomly dividing the training set and the test set according to the ratio of 70% and 30%.

4. The method according to claim 1, characterized in that The method for extracting feature vectors of various types of data includes: converting the acquired training set and test set data into vectors of various types of power-related data, and mapping them to a feature vector of a fixed length using a representation function as the feature vector corresponding to various types of photovoltaic power-related data.

5. The method according to claim 1, characterized in that The process of obtaining the pre-trained GPT photovoltaic power prediction model includes: Obtain historical power data and weather data of photovoltaic power plants; Extracting feature vectors corresponding to various types of historical power-related data; The feature vector is input into the initial GPT photovoltaic power prediction model, first passes through the Mask Multi-HeadAttention layer, then passes through the residual connection and Layer Normalization layer, then passes through the FeedForward layer, and then passes through the second residual connection and Layer Normalization layer, repeats the above hierarchical steps 12 times, and finally passes through the fully connected layer for prediction; It is determined whether the Huber loss value of the model is less than the initial threshold. If not, the GPT model parameters are adjusted through the back propagation algorithm and the gradient descent algorithm until the Huber loss value is less than or equal to the preset threshold, and finally the pre-trained GPT photovoltaic power prediction model is obtained.

6. The method according to claim 1, characterized in that The fine-tuning method of the pre-trained GPT photovoltaic power prediction model comprises: The feature vector of the power-related data of the training set is used as the input of the pre-training model; After the simplified Decoder layer and the fully connected layer, the output vector is obtained; Map the output vector to a specific value, compare the predicted value with the actual value, and calculate the Huber loss; Determine whether the Huber loss value of the model is less than the initially set fine-tuning threshold. If not, adjust the GPT model parameters through the back propagation algorithm and the gradient descent algorithm until the Huber loss value is less than or equal to the preset threshold, and finally obtain the fine-tuned GPT photovoltaic power prediction model.

7. The method according to claim 1, characterized in that The prediction method after fine-tuning the pre-trained GPT photovoltaic power prediction model includes: The feature vector of the power-related data of the test set is used as the input of the fine-tuned pre-trained GPT photovoltaic power prediction model; After the simplified Decoder layer and the fully connected layer, the output vector is obtained; The output vector is then mapped to a specific numerical value, which is the predicted value of the fine-tuned pre-trained GPT photovoltaic power prediction model.

Images