Photovoltaic power station generation power prediction method based on Q learning combination model

Abstract

The invention relates to the technical field of power systems and intelligent power grids, and discloses a photovoltaic power station generation power prediction method based on a Q learning combination model, and the method comprises the steps: collecting meteorological data, power station operation data and time sequence data, carrying out the preprocessing of the collected original data, and screening out the stronger correlation characteristics; a photovoltaic power prediction model based on an XGBoost model is established according to the processed data, a photovoltaic power prediction model based on an LSTM model is established according to the processed data, and after preliminary prediction results of the XGBoost model and the LSTM model are obtained respectively, a Q learning algorithm is adopted to optimize the weight of the combined model to realize accurate prediction of photovoltaic power. And finally, selecting four indexes of a decision coefficient, a root-mean-square error, a mean absolute error and a mean absolute percentage error, evaluating the obtained prediction result, and integrating the steps to finally obtain the prediction method with dynamic optimization combination parameters, adaptive data distribution change and enhanced small sample generalization ability.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems and smart grids, and in particular to a method for predicting the generated power of a photovoltaic power station based on a Q-learning combination model. Background Art

[0002] As a core clean energy source, the accuracy of photovoltaic power forecasting directly affects the stability and absorption capacity of the power grid, and is of great significance for optimizing scheduling, maintenance plans, and cross-domain transmission. However, the randomness and intermittency of photovoltaic output pose challenges to traditional forecasting methods. In existing technologies, traditional physical modeling methods are gradually being eliminated due to their high complexity and insufficient accuracy. Although artificial intelligence algorithms have made progress, they still have obvious limitations: machine learning models rely on large amounts of data and have weak generalization capabilities for small samples. Deep learning models find it difficult to fully capture time-series dependency characteristics, and combined forecasting models mostly use static weight strategies and cannot dynamically adapt to changes in data distribution. The above problems make it difficult for existing methods to meet the needs of refined grid scheduling. To this end, combining big data and artificial intelligence technologies to explore dynamically optimized combined forecasting models has become an important direction for improving the accuracy of photovoltaic power forecasting. Summary of the Invention

[0003] (1) Technical problems solved

[0004] In response to the shortcomings of the existing technology, the present invention provides a photovoltaic power station power generation prediction method based on the Q-learning combination model, which has the advantages of dynamically optimizing combination parameters, adapting to changes in data distribution, and enhancing the generalization ability of small samples. It solves the problems of limited prediction accuracy due to fixed weights of traditional combination models, difficulty in balancing time series feature extraction and generalization performance of a single model, and insufficient robustness of prediction results in complex scenarios.

[0005] (2) Technical solution

[0006] To achieve the above object, the present invention provides the following technical solution: a method for predicting the power generation of a photovoltaic power station based on a Q-learning combination model, comprising the following steps:

[0007] Step 1: Collect meteorological data, power plant operation data, and time series data, and pre-process the collected raw data to screen out strong correlation features;

[0008] Step 2: Based on the data processed in step 1, a photovoltaic power prediction model based on the XGBoost model is established;

[0009] Step 3: Based on the data processed in step 1, a photovoltaic power prediction model based on the LSTM model is established;

[0010] Step 4: After obtaining the preliminary prediction results of the XGBoost model and LSTM model in steps 2 and 3 respectively, the Q-learning algorithm is used to optimize the weights of the combined model to achieve accurate prediction of photovoltaic power;

[0011] Step 5: Select the four indicators of determination coefficient, root mean square error, mean absolute error and mean absolute percentage error to evaluate the prediction results obtained in step 4.

[0012] Preferably, the raw data preprocessing process in step 1 is:

[0013] S1.1. Eliminate missing values and outliers from the original data;

[0014] S1.2. Scale the normal data to the range [0,1] using the maximum and minimum normalization formula. The maximum and minimum normalization formula is:

[0015]

[0016] In the formula, x norm is the normalized data, x is the original data, and x max is the maximum value in the original data, x min is the minimum value in the original data;

[0017] S1.3. Use the Pearson correlation coefficient method for feature extraction. The Pearson correlation coefficient in statistics is used to measure the linear correlation between two variables, which is expressed as:

[0018]

[0019] In the formula, N is the total number of samples; x i ——The characteristic vector of the i-th sample; y i ——The actual photovoltaic power generation power of the i-th sample; The average value of the eigenvectors; The average value of actual generated power.

[0020] Preferably, the photovoltaic power prediction model of the XGBoost model in step 2 is established:

[0021] S2.1. Establish the XGBoost model. XGBoost is an optimized distributed gradient boosting tree based on the boosting family of algorithms. It follows the forward distribution addition method and its objective function is:

[0022]

[0023] In the formula, t is the number of iterations of the model, that is, the total number of trees; k is the k-th tree model; loss function Indicates the deviation of the model, reflecting the actual value y of sample i i and predicted values The gap between the regular term Ω(f k ) represents the variance of the k-th tree model;

[0024] Minimize the objective function, perform Taylor second-order expansion on it and remove all constant terms to find the minimum value O * and the corresponding weight value Then, the optimal split point of the leaf node is generated through a greedy algorithm or an approximate algorithm to determine the structure of the tree, and finally the XGBoost model is obtained;

[0025] S2.2. Preliminary prediction based on XGBoost: The meteorological features with strong correlation and their corresponding photovoltaic power are selected from the training set through the Pearson correlation coefficient method as input to train the XGboost model parameters, and preliminary prediction is performed on the test set data.

[0026] Preferably, the photovoltaic power prediction model of the LSTM model in step 3 is established:

[0027] S3.1. Establish an LSTM model. The LSTM model introduces three control gates: input gate it, forget gate ft, and output gate ot.

[0028] S3.2. Preliminary prediction based on the LSTM model: The time series with strong correlation and their corresponding photovoltaic power are selected from the training set using the Pearson correlation coefficient method as input to train the LSTM model parameters, and preliminary prediction is performed on the test set data.

[0029] Preferably, the problem-solving process in S3.1 is: first, through f t To selectively forget the short-term memory information of the previous moment; secondly, through i t To selectively memorize the input information; finally, through O t Determine the final output;

[0030] 3 control doors t 、f t and O t The calculation formulas are shown in formulas (1, 2):

[0031]

[0032] In the formula, h t-1 ——Short-term memory information at time t-1; x t ——Input sample at time t; ω i 、ω f 、ω O——The weight coefficients corresponding to the input gate, forget gate, and output gate; b i 、b f 、b O ——Bias coefficients corresponding to the input gate, forget gate, and output gate; σ——Sigmoid activation function;

[0033] After obtaining the input of the three control gates, we can further obtain the long-term information C at time t. t 、Short-term memory information h t And the final output y of the LSTM model t for:

[0034]

[0035] In the formula, C t-1 ——Long-term memory information at time t-1; tanh——hyperbolic tangent activation function; ω c 、ω y ——weight coefficient of the model; b c 、b y ——bias coefficient of the model; product.

[0036] Preferably, the Q learning algorithm is used to optimize the weight of the combined model in step 4, including:

[0037] S4.1. Establish the target state matrix S, action matrix A and Q table;

[0038] S4.2. Establish the loss function L and the reward and punishment function R;

[0039] S4.3. Establishment and updating of Q table.

[0040] Preferably, the establishment of the target state matrix S, action matrix A and Q table in S4.1: During the model training process, the optimization goal is to obtain the optimal weight of the combined model, and the target state matrix S is:

[0041] S=[ω1,ω2]

[0042] In the formula, ω1 is the weight of the photovoltaic power prediction result under the XGBoost model; ω2 is the weight of the photovoltaic power prediction result under the LSTM model, where ω2 = 1-ω1, ω1, ω2∈[0,1];

[0043] In the optimization process, the agent needs to continuously explore weight combinations, so that the action matrix A of each exploration is:

[0044] A=[Δω,-Δω]

[0045] In the formula, Δω is the amplitude of each movement.

[0046] Preferably, the establishment of the loss function L and the reward and punishment function R in S4.2: the predicted value of the combined model The error with the true value y of the photovoltaic power is minimized, and the loss function L is set to:

[0047]

[0048] In the Q-learning algorithm, a reward and punishment function R is introduced to guide the learning strategy of the agent. The reward and punishment function R can be expressed as:

[0049]

[0050] In the formula, L m ——The corresponding loss function during the mth training; L m+1 ——The corresponding loss function during the m+1th training.

[0051] Preferably, the establishment and update of the Q table in S4.3: The Q table is a table that records the state-action value, wherein the rows represent the elements in the target state matrix S, and the columns represent the elements in the action matrix A. Each element Q(s,a) in the table represents the reward value that can be obtained in the future by taking action a in state s. Since S and A each contain 2 elements, the Q table is a 2*2 matrix, that is:

[0052] Q new (S m ,a m )=Q old (S m ,a m )

[0053] +α m [λ m maxQ old (S m+1 ,a m+1 )

[0054] +R(S m ,a m )-Q old (S m ,a m )]a m

[0055] In the formula, Q new (S m ,a m )——State S in the new Q table m When performing action a m The corresponding Q value when Q old (S m ,a m )——State S in the old Q tablem When performing action a m The corresponding Q value when R(S m ,a m )——State S m The R value corresponding to the execution of the action; maxQ old (S m+1 ,a m+1 )——next state S in the old Q table m+1 When performing the next action a m+1 The maximum Q value corresponding to α m —depreciation coefficient, reflecting the decay intensity of future rewards; λ m — learning rate;

[0056] By introducing the ε-greedy strategy to achieve action a m The choice of , ε-greedy strategy is expressed as:

[0057]

[0058] In the formula, a Qmax ——The action corresponding to the maximum value in the Q table; a random ——Randomly select an action from the action matrix A; δ——a random number between 0 and 1; ε——greedy coefficient, that is, when ε=0.8;

[0059] The Q learning algorithm is used to train samples online in real time and the Q table is continuously updated until the number of training times reaches the requirement, and finally the optimal weight of the combined model is obtained.

[0060] Preferably, in the evaluation process of step 5, the determination coefficient R is selected 2 , root mean square error X RMSE , mean absolute error X MAE and X MAPE Four indicators are used to evaluate the prediction results of power generation. The four evaluation indicators are expressed as follows:

[0061]

[0062] In the formula, The predicted value of photovoltaic power of the i-th sample; y i ——The actual value of the photovoltaic power of the i-th sample; The average value of the actual PV power.

[0063] Compared with the existing technology, the present invention provides a photovoltaic power station power generation prediction method based on the Q learning combination model, which has the following beneficial effects:

[0064] 1. The present invention optimizes and combines the prediction results of the XGBoost model and the LSTM model through the Q learning algorithm. In each prediction, the algorithm can dynamically adjust the weight according to the error situation, giving a greater weight to the prediction result with a smaller error, thereby effectively improving the prediction accuracy of the overall model; the XGBoost model can effectively handle nonlinear relationships and complex interactions between features, and can well explore the intrinsic relationship between the input data of meteorological characteristics and photovoltaic power, while the LSTM model is good at processing time series data and can capture the changing pattern of photovoltaic power in the time dimension. The present invention combines these two models to give full play to their respective advantages, so that the prediction model can more accurately consider the comprehensive impact of meteorological factors and time series factors on photovoltaic power when facing complex photovoltaic power prediction tasks, thereby improving the prediction accuracy.

[0065] 2. The present invention adopts the Q learning algorithm for online rolling learning. Each time the model makes a prediction, the previous test sample is added to the original data set for learning, so that the model can adapt to changes in meteorological conditions in real time. When meteorological conditions cause actual power changes more frequently, the algorithm can accurately predict and quickly follow the actual value, so that the combined weight of a single model is in real-time change. This dynamic adjustment mechanism enables the model to flexibly adjust the weights of the XGBoost model and the LSTM model according to the latest data and prediction error. It can be seen from the partial combination weight values that at different sampling points, ω1 and ω2 will be adjusted according to the actual situation. This dynamic adjustment can better adapt to the characteristics of dynamic changes in photovoltaic power over time, and improve the adaptability and prediction accuracy of the model.

[0066] 3. The present invention avoids taking all meteorological factors and the power under all time series into account in the prediction model by using the Pearson correlation coefficient method to screen out feature vectors with a strong correlation with the generated power as the input vector of the prediction model in the data preprocessing stage. This can reduce the calculation time and computing power consumption while effectively avoiding the problem of model overfitting. The four meteorological features with a strong correlation with photovoltaic power generation power, namely surface horizontal radiation, temperature, relative humidity, and scattered radiation, are screened out from the meteorological characteristics as the input features of XGBoost, and the relevant time series of the first 25 minutes are screened out from the time series as the input features of the LSTM prediction model, so that the model can focus more on important features and improve model performance. In the data preprocessing process, the normal data is scaled to the [0,1] interval through the maximum and minimum normalization formula, which helps to eliminate the dimensional differences between different features, not only improving the model training efficiency, but also improving the model performance, so that the model can run more stably in subsequent training and prediction processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is the structural diagram of the LSTM model of the present invention;

[0068] Figure 2 This is a structural diagram of the Q-learning combination model of the present invention;

[0069] Figure 3 Figure a is the BP-NN model of the present invention;

[0070] Figure 4 This is the LSTM model diagram b of the present invention;

[0071] Figure 5 Figure c is the XGBoost model of the present invention;

[0072] Figure 6 Figure d is the ER-XGBoost-LSTM model of the present invention;

[0073] Figure 7 This is the Q-learning-XGBoost-LSTM model diagram of the present invention. DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0075] See also Figure 1-Figure 7 A photovoltaic power station power generation prediction method based on a Q-learning combination model includes the following steps:

[0076] Step 1: Collect meteorological data, power plant operation data, and time series data, and pre-process the collected raw data to screen out strong correlation features;

[0077] Step 2: Based on the data processed in step 1, a photovoltaic power prediction model based on the XGBoost model is established;

[0078] Step 3: Based on the data processed in step 1, a photovoltaic power prediction model based on the LSTM model is established;

[0079] Step 4: After obtaining the preliminary prediction results of the XGBoost model and LSTM model in steps 2 and 3 respectively, the Q-learning algorithm is used to optimize the weights of the combined model to achieve accurate prediction of photovoltaic power;

[0080] Step 5: Select the four indicators of determination coefficient, root mean square error, mean absolute error and mean absolute percentage error to evaluate the prediction results obtained in step 4.

[0081] The raw data preprocessing process in step 1 is:

[0082] S1.1. Eliminate missing values and abnormal values (such as negative values, values exceeding the rated limit, etc.) in the original data;

[0083] S1.2. Use the maximum and minimum normalization formula to scale normal data to the range [0,1], eliminate the dimensional differences between different features, and improve model training efficiency and performance. The maximum and minimum normalization formula is:

[0084]

[0085] In the formula, x norm is the normalized data, x is the original data, and x max is the maximum value in the original data, x min is the minimum value in the original data;

[0086] S1.3. The power generation of a photovoltaic power station is not only affected by meteorological conditions, but is also closely related to its own timing. However, if all meteorological factors and the power under all time series are taken into account in the prediction model, it will not only significantly increase the calculation time and computing power consumption, but also cause the model to overfit and reduce the overall prediction accuracy. Therefore, feature vectors with a strong correlation with the power generation are extracted from the data set as the input vector of the prediction model. Therefore, the Pearson correlation coefficient method is used for feature extraction. The Pearson correlation coefficient in statistics is used to measure the linear correlation between two variables, which is expressed as:

[0087]

[0088] In the formula, N is the total number of samples; x i ——The characteristic vector of the i-th sample; y i ——The actual photovoltaic power generation power of the i-th sample; The average value of the eigenvectors; The average value of actual power generation;

[0089] The value of the Pearson correlation coefficient r is between -1 and +1. When the absolute value of r is greater than 0.5, it can be considered that there is a strong correlation between the feature vector x and the actual generated power y. In the present invention, x with a strong correlation with y (i.e., the absolute value of r is greater than 0.5) is selected as the input feature of the prediction model, and x with a weaker correlation (i.e., the absolute value of r is less than 0.5) is ignored.

[0090] The advantages are: by using the Pearson correlation coefficient method to select feature vectors with a strong correlation with the generated power as the input vectors of the prediction model in the data preprocessing stage, all meteorological factors and the power in all time series are avoided from being taken into account in the prediction model. This can reduce the calculation time and computing power consumption while effectively avoiding the problem of model overfitting. Four meteorological features with a strong correlation with photovoltaic power generation power, namely surface horizontal radiation, temperature, relative humidity, and scattered radiation, are selected from the meteorological characteristics as the input features of XGBoost, and the relevant time series of the first 25 minutes are selected from the time series as the input features of the LSTM prediction model, so that the model can focus more on important features and improve model performance. In the data preprocessing process, the maximum and minimum normalization formula is used to scale the normal data to the [0,1] range, which helps to eliminate the dimensional differences between different features. This can not only improve the model training efficiency, but also improve the model performance, so that the model can run more stably in subsequent training and prediction processes.

[0091] In step 2, the photovoltaic power prediction model of the XGBoost model is established:

[0092] S2.1. Establish the XGBoost model. XGBoost is an optimized distributed gradient boosting tree based on the boosting family of algorithms. It follows the forward distribution addition method and its objective function is:

[0093]

[0094] In the formula, t is the number of iterations of the model, that is, the total number of trees; k is the k-th tree model; loss function Indicates the deviation of the model, reflecting the actual value y of sample i i and predicted values The gap between the regular term Ω(f k ) represents the variance of the k-th tree model to prevent the model from overfitting;

[0095] In order to minimize the objective function, Taylor second-order expansion is performed and all constant terms are removed to find the minimum value O * and the corresponding weight value Then, the optimal split point of the leaf node is generated through a greedy algorithm or an approximate algorithm to determine the structure of the tree, and finally the XGBoost model is obtained;

[0096] S2.2. Preliminary prediction based on XGBoost: The meteorological features with strong correlation and their corresponding photovoltaic power are selected from the training set through the Pearson correlation coefficient method as input to train the XGboost model parameters, and preliminary prediction is performed on the test set data.

[0097] In step 3, the photovoltaic power prediction model of the LSTM model is established:

[0098] S3.1. Build an LSTM model. The LSTM model introduces three control gates: input gate it, forget gate ft, and output gate ot. This solves the gradient vanishing and gradient exploding problems encountered by traditional RNNs when processing long sequence data.

[0099] S3.2. Preliminary prediction based on the LSTM model: The time series with strong correlation and their corresponding photovoltaic power are selected from the training set using the Pearson correlation coefficient method as input to train the LSTM model parameters, and preliminary prediction is performed on the test set data.

[0100] Specifically, the problem-solving process in S3.1 is: first, through f t To selectively forget the short-term memory information of the previous moment; secondly, through i t To selectively memorize the input information; finally, through O t Determines the final output, which makes the LSTM model have the function of long-term memory based on the RNN model. Its prediction model structure is as follows Figure 1 As shown;

[0101] 3 control doors t 、f t and O t The calculation formulas are shown in formulas (1, 2):

[0102]

[0103] In the formula, h t-1 ——Short-term memory information at time t-1; x t ——Input sample at time t; ω i 、ω f 、ω O ——The weight coefficients corresponding to the input gate, forget gate, and output gate; b i 、b f 、b O ——Bias coefficients corresponding to the input gate, forget gate, and output gate; σ——Sigmoid activation function;

[0104] After obtaining the input of the three control gates, the long-term information C at time t can be further obtained. t 、Short-term memory information h t And the final output y of the LSTM model t for:

[0105]

[0106] In the formula, C t-1——Long-term memory information at time t-1; tanh——hyperbolic tangent activation function; ω c 、ω y ——weight coefficient of the model; b c 、b y ——bias coefficient of the model; product.

[0107] In step 4, the Q-learning algorithm is used to optimize the weights of the combined model, including:

[0108] S4.1. Establish the target state matrix S, action matrix A and Q table;

[0109] S4.2. Establish the loss function L and the reward and punishment function R;

[0110] S4.3. Establishment and updating of Q table.

[0111] Specifically, the establishment of the target state matrix S, action matrix A and Q table in S4.1: During the model training process, the optimization goal is to obtain the optimal weight of the combined model, and the target state matrix S is:

[0112] S=[ω1,ω2]

[0113] In the formula, ω1 is the weight of the photovoltaic power prediction result under the XGBoost model; ω2 is the weight of the photovoltaic power prediction result under the LSTM model, where ω2 = 1-ω1, ω1, ω2∈[0,1];

[0114] In the optimization process, the agent needs to continuously explore weight combinations, so that the action matrix A of each exploration is:

[0115] A=[Δω,-Δω]

[0116] In the formula, Δω is the amplitude of each movement.

[0117] Specifically, the establishment of the loss function L and the reward and punishment function R in S4.2:

[0118] To make the predicted value of the combined model The error with the true value y of the photovoltaic power is minimized, and the loss function L is set to:

[0119]

[0120] In the Q-learning algorithm, a reward and punishment function R is introduced to guide the learning strategy of the agent. When the action performed by the agent is biased towards the target state, it will receive positive rewards to take corresponding actions to approach the target state; if the action causes the agent to deviate from the target state, it will receive negative penalties to remind the agent to choose other actions next time. Through continuous feedback on the agent's actions, the agent gradually reaches the target state. The reward and punishment function R can be expressed as:

[0121]

[0122] In the formula, L m ——The corresponding loss function during the mth training; L m+1 ——The corresponding loss function during the m+1th training.

[0123] Specifically, the creation and updating of the Q-table in S4.3: The Q-table is a table that records the state-action values. Its rows represent the elements in the target state matrix S, and its columns represent the elements in the action matrix A. Each element Q(s,a) in the table represents the future reward value obtained by taking action a in state s. In this invention, S and A each contain 2 elements, so the Q-table is a 2*2 matrix.

[0124] The update of the Q table is the core part of the Q learning algorithm. The Q table is continuously optimized through iteration to guide the intelligent agent to make better decisions. The update of the Q table is achieved through the Bellman equation, that is:

[0125] Q new (S m ,a m )=Q old (S m ,a m )

[0126] +α m [λ m maxQ old (S m+1 ,a m+1 )

[0127] +R(S m ,a m )-Q old (S m ,a m )]a m

[0128] In the formula, Q new (S m ,a m )——State S in the new Q table m When performing action a m The corresponding Q value when Qold (S m ,a m )——State S in the old Q table m When performing action a m The corresponding Q value when R(S m ,a m )——State S m The R value corresponding to the execution of the action; maxQ old (S m+1 ,a m+1 )——next state S in the old Q table m+1 When performing the next action a m+1 The maximum Q value corresponding to α m —depreciation coefficient, reflecting the decay intensity of future rewards; λ m — learning rate;

[0129] When selecting an action, the agent can choose the currently known best action or randomly try non-optimal actions, so as to balance the exploration and utilization of the agent and improve learning efficiency. By introducing the ε-greedy strategy, action a m The choice of , ε-greedy strategy is expressed as:

[0130]

[0131] In the formula, a Qmax ——The action corresponding to the maximum value in the Q table; a random ——Randomly select an action from the action matrix A; δ——A random number between 0 and 1; ε——Greedy coefficient, that is, when ε=0.8, it means that when the agent chooses an action, there is an 80% chance that it will choose a Qmax , there is a 20% chance of choosing a random .

[0132] The Q learning algorithm is used to train samples online in real time and continuously update the Q table until the number of training times reaches the required number, and finally the optimal weight of the combined model is obtained. The algorithm structure is as follows Figure 2 shown.

[0133] The advantages are: the prediction results of the XGBoost model and the LSTM model are optimized and combined through the Q learning algorithm. In each prediction, the algorithm can dynamically adjust the weight according to the error situation, giving a larger weight to the prediction result with a smaller error, thereby effectively improving the prediction accuracy of the overall model; the XGBoost model can effectively handle nonlinear relationships and complex interactions between features, and can well explore the intrinsic relationship between the input data of meteorological characteristics and photovoltaic power, while the LSTM model is good at processing time series data and can capture the changing law of photovoltaic power in the time dimension. The present invention combines these two models to give full play to their respective advantages, so that the prediction model can more accurately consider the comprehensive impact of meteorological factors and time series factors on photovoltaic power when facing complex photovoltaic power prediction tasks, thereby improving the prediction accuracy.

[0134] Evaluation process in step 5: Select the coefficient of determination R 2 , root mean square error X RMSE , mean absolute error X MAE and X MAPE Four indicators are used to evaluate the prediction results of power generation, among which R 2 To measure the degree of fit of the model, the value range is 0 to 1. The closer it is to 1, the better the prediction effect of the model, and the closer it is to 0, the worse the prediction effect. The other three indicators are mainly used to evaluate the accuracy of the prediction value. MAE and X RMSE It is used to measure the absolute size of the deviation of the predicted value from the true value; and X MAPE Used to measure the relative size of the deviation value, the four evaluation indicators are expressed as:

[0135]

[0136] In the formula, The predicted value of photovoltaic power of the i-th sample; y i ——The actual value of the photovoltaic power of the i-th sample; The average value of the actual photovoltaic power;

[0137] When the actual photovoltaic power value is 0, X MAPE The value of X is infinite. To ensure that the evaluation index is meaningful, MAPE When conducting the evaluation, only the data from the time period of 8:00 to 17:30 were selected for evaluation.

[0138] The advantages are: by adopting the Q learning algorithm for online rolling learning, the previous test sample will be added to the original data set for learning each time the model makes a prediction, so that the model can adapt to changes in meteorological conditions in real time. When meteorological conditions cause actual power to change frequently, the algorithm can accurately predict and quickly follow the actual value, so that the combined weight of a single model is in real-time change. This dynamic adjustment mechanism enables the model to flexibly adjust the weights of the XGBoost model and the LSTM model according to the latest data and prediction error. It can be seen from some of the combined weight values that at different sampling points, ω1 and ω2 will be adjusted according to the actual situation. This dynamic adjustment can better adapt to the characteristics of dynamic changes in photovoltaic power over time, and improve the adaptability and prediction accuracy of the model.

[0139] Example

[0140] This paper conducts a case analysis based on the meteorological data and power generation of photovoltaic power stations in a certain area in May 2018. The analysis process is as follows:

[0141] T1. Experimental data: The PV power station samples data every 5 minutes, and a total of 8,928 data points were collected throughout May. Data points from May 1 to 30 were selected as training samples, and data points from May 31 were selected as test samples.

[0142] T2. Meteorological feature extraction: The photovoltaic power station collects a total of 6 meteorological features, namely: temperature, relative humidity, surface horizontal radiation, scattered radiation, wind direction, and rainfall;

[0143] Through Pearson correlation coefficient analysis, the correlation coefficients between various meteorological characteristics and photovoltaic power generation are shown in Table 1:

[0144] Table 1 Pearson correlation coefficients of meteorological characteristics

[0145]

[0146]

[0147] As can be seen from Table 1, photovoltaic power generation has the highest correlation with surface horizontal radiation, with a coefficient close to 1, which is the most important factor affecting photovoltaic power generation. The correlation coefficients of temperature, relative humidity, and diffuse radiation are 0.537, -0.501, and 0.565, respectively, which are significantly correlated with photovoltaic power generation. The correlation coefficients of wind direction and rainfall are both less than 0.2, which are weakly correlated with photovoltaic power generation. Therefore, the present invention selects four meteorological features with a strong correlation with photovoltaic power generation (the absolute value of the correlation coefficient is greater than 0.5), namely surface horizontal radiation, temperature, relative humidity, and diffuse radiation, as input features of XGBoost.

[0148] T3. Time step selection: In the LSTM model, the selection of the time step will directly affect the length of the time series and also directly affect the prediction effect of the model. To determine the optimal time step, the Pearson correlation coefficient method is also used to perform correlation analysis on the photovoltaic power generation corresponding to a total of 12 time series within 1 hour before the prediction point. The results are shown in Table 2:

[0149] Table 2 Pearson correlation coefficient of time step

[0150] Time Series Correlation coefficient Time Series Correlation coefficient 1 (5 minutes) 0.917 7 (35 minutes) 0.349 2 (10 minutes) 0.834 8 (40 minutes) 0.253 3 (15 minutes) 0.742 9 (45 minutes) 0.186 4 (20 minutes) 0.638 10 (50 minutes) 0.101 5 (25 minutes) 0.519 11 (55 minutes) 0.078 6 (30 minutes) 0.411 12 (60 minutes) 0.021

[0151] As can be seen from Table 2, the closer the time series is to the prediction point, the greater the correlation coefficient of the corresponding photovoltaic power generation, and vice versa. The correlation coefficients of time series 1 to 5 within 25 minutes of the prediction point are all greater than 0.5, indicating that the photovoltaic power of this time series is strongly correlated with the photovoltaic power at the prediction point. Therefore, in this invention, the photovoltaic power corresponding to time series 1 to 5 is used as the input feature of the LSTM prediction model, that is, the time step of the LSTM model is 5;

[0152] T4. Verification of the effectiveness of the Q-learning combined model (Q-learning-XGBoost-LSTM): To verify the effectiveness of the Q-learning combined model (Q-learning-XGBoost-LSTM) proposed in this invention, it is compared with the widely used Back Propagation Neural Network (BP-NN), the single XGBoost algorithm, the LSTM algorithm, and the combined algorithm based on the inverse error method (ER-XGBoost-LSTM). The model parameters are shown in Table 3:

[0153] Table 3 Model parameters

[0154]

[0155] The prediction results obtained using the five models are as follows: Figure 3-7 (a, b, c, d and e, the predicted and actual values of photovoltaic power generation under the five models) are shown in Table 4. The evaluation indicators are shown in Table 4:

[0156] Table 4 (Evaluation indicators of photovoltaic power generation prediction results under five models)

[0157]

[0158]

[0159] from Figure 3-7 As can be seen from Table 4:

[0160] (1) The prediction accuracy of a single model is poor. When the meteorological conditions in different periods of time change frequently in the morning, afternoon and evening, the predicted values of the BP-NN model and the LSTM model cannot follow the actual values well and show strong hysteresis and error. Among them, the BP-NN model has the highest R 2 The lowest value, X RMSE The highest value indicates that the prediction effect of this model is the worst and the error is the largest, followed by the LSTM model. Compared with the BP-NN model and the LSTM model, the XGBoost model has a higher R 2 The value is slightly higher, but X MAE Value, X MAPE The value is the highest among the four prediction models. During the period from 11:00 to 15:00, its predicted value still has a large error compared with the true value;

[0161] (2) The prediction accuracy of the combined prediction model based on the inverse error method has been improved. After the combination of the inverse error method, the combined model can take into account the prediction advantages of each single model. Compared with the first three single models, the four evaluation indicators under the combined prediction model have been significantly improved, and X RMSE Value, X MAE Value and X MAPE The values were reduced by about 50%, indicating that the prediction accuracy of the model has improved;

[0162] (3) The prediction accuracy of the Q-learning combination model is the highest. The Q-learning algorithm optimizes the prediction results of the two single prediction models, LSTM and XGBoost, so that in each prediction, the results with smaller errors are given a larger weight and the results with smaller errors are given a smaller weight, thereby improving the overall prediction accuracy of the model. It can be seen that compared with the combination model of the inverse error method, except for the X MAPE Except for a slightly higher value, the other three evaluation indicators R 2 Value, X RMSE Value, X MAE The values have all improved. It can also be seen from the forecast graph that even when meteorological conditions cause the actual power to change frequently, the algorithm can accurately predict and quickly follow the actual value, showing excellent forecast performance.

[0163] T5. Optimizing weights: Since the photovoltaic power prediction based on the Q-learning combination model is an online rolling learning, that is, each time the model makes a prediction, the previous test sample is added to the original data set for learning to determine the weight of the next single model and perform a combined prediction. Therefore, the combined weights of the single models are in real-time change during each prediction. Some weights are shown in Table 5:

[0164] Table 5. Some combination weights of the learning combination model

[0165] Sampling point <![CDATA[ω1]]> <![CDATA[ω2]]> 1 0.981 0.019 2 0.973 0.027 3 0.922 0.078 … … … 141 0.472 0.528 142 0.829 0.171 143 0.674 0.326 … … … 281 0.912 0.088 282 0.891 0.109 283 0.938 0.062

[0166] Summary: The Q-learning-XGBoost-LSTM combination model of the present invention can flexibly adjust the weights according to the latest data and prediction error through a dynamic weight adjustment mechanism, thereby further improving the adaptability and prediction accuracy of the model. This makes the model perform well in photovoltaic power forecasting, with high prediction accuracy and good dynamic adaptability, and can effectively cope with photovoltaic power changes under complex meteorological conditions.

[0167] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A photovoltaic power station power generation prediction method based on a Q-learning combination model, characterized in that: The following steps are involved: Step 1: Collect meteorological data, power plant operation data, and time series data, and pre-process the collected raw data to screen out strong correlation features; Step 2: Based on the data processed in step 1, a photovoltaic power prediction model based on the XGBoost model is established; Step 3: Based on the data processed in step 1, a photovoltaic power prediction model based on the LSTM model is established; Step 4: After obtaining the preliminary prediction results of the XGBoost model and LSTM model in steps 2 and 3 respectively, the Q-learning algorithm is used to optimize the weights of the combined model to achieve accurate prediction of photovoltaic power; Step 5: Select the four indicators of determination coefficient, root mean square error, mean absolute error and mean absolute percentage error to evaluate the prediction results obtained in step 4.

2. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 1 is characterized in that: The raw data preprocessing process in step 1 is as follows: S1.

1. Eliminate missing values and outliers from the original data; S1.

2. Scale the normal data to the range [0,1] using the maximum and minimum normalization formula. The maximum and minimum normalization formula is: In the formula, x norm is the normalized data, x is the original data, and x max is the maximum value in the original data, x min is the minimum value in the original data; S1.

3. Use the Pearson correlation coefficient method for feature extraction. The Pearson correlation coefficient in statistics is used to measure the linear correlation between two variables, which is expressed as: In the formula, N is the total number of samples; x i ——The characteristic vector of the i-th sample; y i ——The actual photovoltaic power generation power of the i-th sample; ——The average value of N eigenvectors; ——The average value of N actual power generation.

3. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 1 is characterized in that: Establishment of the photovoltaic power prediction model of the XGBoost model in step 2: S2.

1. Establish the XGBoost model. XGBoost is an optimized distributed gradient boosting tree based on the boosting family of algorithms. It follows the forward distribution addition method and its objective function is: In the formula, t is the number of iterations of the model, that is, the total number of trees; k is the k-th tree model; loss function Indicates the deviation of the model, reflecting the actual value y of sample i i and predicted values The gap between the regular term Ω(f k ) represents the variance of the k-th tree model; Minimize the objective function, perform Taylor second-order expansion on it and remove all constant terms to find the minimum value O * and the corresponding weight value Then, the optimal split point of the leaf node is generated through a greedy algorithm or an approximate algorithm to determine the structure of the tree, and finally the XGBoost model is obtained; S2.

2. Preliminary prediction based on XGBoost: The meteorological features with strong correlation and their corresponding photovoltaic power are selected from the training set through the Pearson correlation coefficient method as input to train the XGboost model parameters, and preliminary prediction is performed on the test set data.

4. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 1 is characterized in that: Establishment of the photovoltaic power prediction model of the LSTM model in step 3: S3.

1. Establish an LSTM model. The LSTM model introduces three control gates: input gate it, forget gate ft, and output gate ot. S3.

2. Preliminary prediction based on the LSTM model: The time series with strong correlation and their corresponding photovoltaic power are selected from the training set using the Pearson correlation coefficient method as input to train the LSTM model parameters, and preliminary prediction is performed on the test set data.

5. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 4 is characterized in that: The problem-solving process in S3.1: First, through f t To selectively forget the short-term memory information of the previous moment; secondly, through i t To selectively memorize the input information; finally, through O t Determine the final output; 3 control doors t 、f t and O t The calculation formulas are shown in formulas (1, 2): In the formula, h t-1 ——Short-term memory information at time t-1; x t ——Input sample at time t; ω i 、ω f 、ω O ——The weight coefficients corresponding to the input gate, forget gate, and output gate; b i 、b f 、b O ——Bias coefficients corresponding to the input gate, forget gate, and output gate; σ——Sigmoid activation function; After obtaining the input of the three control gates, we can further obtain the long-term information C at time t. t 、Short-term memory information h t And the final output y of the LSTM model t for: In the formula, C t-1 ——Long-term memory information at time t-1; tanh——hyperbolic tangent activation function; ω c 、ω y ——weight coefficient of the model; b c 、b y ——bias coefficient of the model; ——Hadamard product.

6. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 1 is characterized in that: In step 4, the Q learning algorithm is used to optimize the weight of the combined model, including: S4.

1. Establish the target state matrix S, action matrix A and Q table; S4.

2. Establish the loss function L and the reward and punishment function R; S4.

3. Establishment and updating of Q table.

7. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 6 is characterized in that: Establishment of the target state matrix S, action matrix A and Q table in S4.1: During the model training process, the optimization goal is to obtain the optimal weight of the combined model. Let the target state matrix S be: S=[ω1,ω2] In the formula, ω1 is the weight of the photovoltaic power prediction result under the XGBoost model; ω2 is the weight of the photovoltaic power prediction result under the LSTM model, where ω2 = 1-ω1, ω1, ω2∈[0,1]; In the optimization process, the agent needs to continuously explore weight combinations, so that the action matrix A of each exploration is: A=[Δω,-Δω] In the formula, Δω is the amplitude of each movement.

8. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 6 is characterized in that: The establishment of the loss function L and the reward and punishment function R in S4.2: The predicted value of the combined model The error with the true value y of the photovoltaic power is minimized, and the loss function L is set to: In the Q-learning algorithm, a reward and punishment function R is introduced to guide the learning strategy of the agent. The reward and punishment function R can be expressed as: In the formula, L m ——The corresponding loss function during the mth training; L m+1 ——The corresponding loss function during the m+1th training.

9. The photovoltaic power generation prediction method based on the Q-learning combination model according to claim 6, characterized in that: The establishment and update of the Q table in S4.3: The Q table is a table that records the state-action value. Its rows represent the elements in the target state matrix S, and its columns represent the elements in the action matrix A. Each element Q(s,a) in the table represents the reward value that can be obtained in the future by taking action a in state s. Since S and A contain 2 elements each, the Q table is a 2*2 matrix, that is: Q new (S m ,a m )=Q old (S m ,a m ) +a m [l m maxQ old (S m+1 ,a m+1 ) +R(S m ,a m )-Q old (S m ,a m )]a m In the formula, Q new (S m ,a m )——State S in the new Q table m When performing action a m The corresponding Q value when Q old (S m ,a m )——State S in the old Q table m When performing action a m The corresponding Q value when R(S m ,a m )——State S m The R value corresponding to the execution of the action; maxQ old (S m+1 ,a m+1 )——next state S in the old Q table m+1 When performing the next action a m+1 The maximum Q value corresponding to α m —depreciation coefficient, reflecting the decay intensity of future rewards; λ m — learning rate; By introducing the ε-greedy strategy to achieve action a m The choice of , ε-greedy strategy is expressed as: In the formula, a Qmax ——The action corresponding to the maximum value in the Q table; a random ——Randomly select an action from the action matrix A; δ——a random number between 0 and 1; ε——greedy coefficient, that is, when ε=0.8; The Q learning algorithm is used to train samples online in real time and the Q table is continuously updated until the number of training times reaches the requirement, and finally the optimal weight of the combined model is obtained.

10. The photovoltaic power generation prediction method based on the Q learning combination model according to claim 1, characterized in that: Evaluation process in step 5: Select the determination coefficient R 2 , root mean square error X RMSE , mean absolute error X MAE and X MAPE Four indicators are used to evaluate the prediction results of power generation. The four evaluation indicators are expressed as follows: In the formula, ——the predicted value of photovoltaic power of the i-th sample; y i ——The actual value of the photovoltaic power of the i-th sample; ——The average value of the actual photovoltaic power.

Images