Electricity price prediction method based on similar days and data filling

Abstract

The invention belongs to the technical field of electricity price prediction, and particularly relates to an electricity price prediction method based on similar days and data filling. Power grid supply and demand data are obtained, data preprocessing is carried out, and features with high correlation with the electricity price are screened out to form an electricity price prediction data set; based on the screened features with high correlation with the electricity price, calculating the similarity of the feature vectors of the to-be-predicted day and the historical day, calculating the random forest feature weight of each feature by using the random forest, and obtaining the final similarity of each feature by using the weighted average of the similarity and the random forest feature weight so as to screen the similar day; obtaining first feature data; filling the missing features to obtain second feature data; performing principal component analysis on the first feature data and the second feature data to obtain principal component features; and on the basis of the principal component features, constructing an electricity price prediction model, and training the electricity price prediction model to obtain a trained electricity price prediction model so as to realize the prediction of the electricity price.

Description

Technical Field

[0001] The present invention belongs to the technical field of electricity price prediction, and in particular relates to an electricity price prediction method based on similar days and data filling. Background Art

[0002] Driven by the dual forces of power market reform and the high-proportion integration of renewable energy, the accuracy and real-time nature of electricity price forecasts have become key technical support for ensuring profitability for market participants and resolving supply-demand imbalances. However, three key challenges in electricity price forecasting urgently require breakthroughs: extreme scenario modeling, full-chain data integration, and dynamic modeling for specific regions. Against this backdrop, overcoming the bottleneck in electricity price forecast accuracy has become a core technological imperative for ensuring the stable operation of the spot market.

[0003] At present, there are two main research methods for electricity price forecasting: traditional methods and new intelligent methods. In addition to the impact of model selection on prediction accuracy, feature selection is also particularly important. However, there are barriers to obtaining the features required for electricity price forecasting, scarcity of extreme scenario data, and difficulty in obtaining differentiated features. Summary of the Invention

[0004] In order to overcome the problems in the prior art, the present invention proposes an electricity price prediction method based on similar days and data filling.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: The present invention provides an electricity price prediction method based on similar days and data filling, comprising the following steps: Obtain power grid supply and demand data to form an initial electricity price prediction dataset; perform data preprocessing on the initial electricity price prediction dataset and select features with high correlation with electricity prices to form an electricity price prediction dataset; Based on the selected features with high correlation with electricity prices, the similarity of the feature vectors of the predicted day and the historical day is calculated. The random forest feature weight of each feature is calculated using random forest. The final similarity of each feature is obtained by weighted average of the similarity and the random forest feature weight. This is used to screen similar days and obtain the first feature data. Fill in the missing features to obtain the second feature data; Performing principal component analysis on the first feature data and the second feature data to obtain principal component features; Based on the principal component characteristics, an electricity price prediction model is constructed, and the electricity price prediction model is trained to obtain a trained electricity price prediction model.

[0006] Furthermore, the preprocessing includes random missing value processing, outlier detection and correction, and duplicate data processing.

[0007] Furthermore, features with high correlation with electricity prices are selected to form the electricity price prediction dataset, including: Extract calendar information data from timestamp data and generate time features; For the preprocessed electricity price prediction data set, the Pearson correlation coefficient between each feature and the electricity price is calculated, and the features with high correlation with the electricity price are screened out to form the electricity price prediction data set, where each feature includes the original features in the preprocessed electricity price prediction data set and the generated time features.

[0008] Furthermore, the missing features are filled to obtain the second feature data, including: Fill in missing features based on KNN, global statistics and XGBoost prediction models to obtain KNN filling values, global statistics values ​​and XGBoost prediction values; The KNN filling value, global statistical value and XGBoost prediction value are weighted to obtain the second feature data.

[0009] Furthermore, principal component analysis is performed on the first feature data and the second feature data to obtain principal component features, including: performing standardization processing on the first characteristic data and the second characteristic data; Calculate the covariance matrix based on the standardized data; Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors; Calculate the cumulative contribution of the principal components, determine the number of principal components to be retained, and then obtain the principal component characteristics.

[0010] Furthermore, the LGBM algorithm is used to construct an electricity price prediction model.

[0011] Compared with the prior art, the present invention has the following technical effects: By screening features highly correlated with electricity prices and utilizing similar day screening and data filling techniques, this method can more accurately capture patterns in electricity price fluctuations, thereby improving forecast accuracy. Filling in missing features reduces the impact of missing data on model performance and enhances model robustness. Principal component analysis fuses multiple features into a single principal component feature, reducing data dimensionality, computational complexity, and improving model training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 This is a comparison chart of the predicted effect of electricity prices before and after the improvement on the first day of the present invention; Figure 3 This is a comparison chart of the predicted effect of electricity prices before and after the improvement on the second day of the present invention; Figure 4 This is a comparison chart of the predicted effects of electricity prices before and after the improvement on the third day of the present invention; Figure 5 This is a comparison chart of the predicted effects of electricity prices before and after the improvement on the fourth day of the present invention; Figure 6 This is a comparison chart of the predicted effects of electricity prices before and after the improvement on the fifth day of the present invention; Figure 7 This is a comparison chart of the predicted effects of electricity prices before and after the improvement on the sixth day of the present invention; Figure 8 This is a comparison chart of the predicted effects of electricity prices before and after the improvement on the seventh day according to the present invention. DETAILED DESCRIPTION

[0014] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation methods, structures, features, and effects of the technical solutions proposed by the present invention. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0015] In one embodiment of the present invention, referring to Figure 1 , provides an electricity price prediction method based on similar days and data filling, including the following steps: Obtain power grid supply and demand data to form an initial electricity price prediction data set; perform data preprocessing on the initial electricity price prediction data set; and select features with high correlation with electricity prices to form an electricity price prediction data set; Based on the selected features with high correlation with electricity prices, the similarity of the feature vectors of the predicted day and the historical day is calculated. The random forest feature weight of each feature is calculated using random forest. The final similarity of each feature is obtained by weighted average of the similarity and the random forest feature weight. This is used to screen similar days and obtain the first feature data. By integrating local similarity matching, global statistical filling and prediction models, missing features are filled to obtain the second feature data; Performing principal component analysis on the first feature data and the second feature data to obtain principal component features; Based on the principal component characteristics, an electricity price prediction model is constructed, and the electricity price prediction model is trained to obtain a trained electricity price prediction model.

[0016] The following is a detailed explanation of each of the above steps: Step 100: Obtain power grid supply and demand data to form an initial electricity price prediction data set.

[0017] Obtain historical electricity price data, meteorological data such as temperature and air pressure, grid supply and demand data such as unified load and total output of new energy sources to form the initial electricity price prediction data set.

[0018] Step 200: Preprocess the initial electricity price prediction data set.

[0019] The preprocessing includes random missing value processing, outlier detection and correction, and duplicate data processing.

[0020] Random missing value processing involves interpolating randomly missing individual data points based on related variables. Outlier detection and correction involves using statistical methods, such as boxplot removal or correction of outlier values. Duplicate data processing ensures unique timestamps.

[0021] Step 300: Screen out features that are highly correlated with electricity prices to form an electricity price prediction data set.

[0022] Calendar information data is extracted from timestamp data to generate time features. For the preprocessed electricity price prediction data set, the Pearson correlation coefficient between each feature (including original features and generated time features) and electricity price is calculated, and features with high correlation with electricity price are selected to form the electricity price prediction data set.

[0023] Step 400: Based on the screened features with high correlation with electricity prices, calculate the similarity of the feature vectors of the predicted day and the historical day, use random forest to calculate the random forest feature weight of each feature, and use the similarity and random forest feature weight to weighted average to obtain the final similarity of each feature, so as to screen similar days and obtain the first feature data.

[0024] As an example, this step 400 includes the following sub-steps: Step 410: Calculate the similarity coefficient between the features of the predicted day and the historical days: ; In the above formula, represents the similarity coefficient; Indicates the forecast date i Features Indicates historical day i Features Indicates the number of feature variables.

[0025] Step 420: Calculate the weight of each feature using random forest.

[0026] Random forest is used to calculate the mean square error (MSE) to measure the importance of each feature to the electricity price. The importance is mainly judged based on the degree to which the data purity is improved when the decision tree splits the tree node. The splitting criterion is to select the feature and threshold that maximizes the difference between the parent node MSE and the weighted MSE of the child node.

[0027] The specific calculation process is as follows: Step 4201: Calculate the parent node MSE: ; In the above formula, Represents the parent node MSE; Indicates the i The value of the local data subset that has not been split in the current parent node; Represents the mean of the parent node target values.

[0028] Step 4202: Calculate the MSE of the child nodes after splitting.

[0029] Assume by feature f Threshold t Split into left and right child nodes: ; ; In the above formula, Represents the MSE value of the left child node; Indicates the number of left child nodes; Represents the left child node; Represents the mean of the left child node; Represents the MSE value of the right child node; Indicates the number of right child nodes; Represents the right child node; Represents the mean of the right child node.

[0030] The weighted MSE after splitting is: ; In the above formula, is the MSE weighted value after splitting.

[0031] Step 4203: Calculate the MSE importance score of the feature based on the MSE of the parent node and the MSE of the child node after splitting.

[0032] Pair Features f , its importance score is the sum of the MSE reductions when the feature is split in all trees, that is: ; In the above formula, Representation characteristics f Importance score; express t Features in a tree f Parent node MSE; express t Features in a tree f The weighted MSE value after splitting.

[0033] Step 4204: Based on the MSE importance score of the feature, calculate the random forest feature weight.

[0034] Normalize the importance scores of all features to get the feature weights : ; In the above formula, M represents the total number of features; J represents the Jth feature; represents the importance score of the J-th feature.

[0035] Step 430: The similarity coefficient and the random forest feature weight are weighted averaged to obtain a comprehensive feature weight.

[0036] ; In the above formula, represents the comprehensive feature weight; Represents the weight of the similarity coefficient.

[0037] Step 440: Based on the weight of each feature and the comprehensive feature, the top 10 similar day combinations are screened to obtain feature data of the screened similar days.

[0038] Step 500: Fusing local similarity matching, global statistical filling and prediction model, filling in missing features to obtain second feature data.

[0039] In response to the missing data in the feature data set caused by the untimely update of some disclosed data of the power trading center, the dynamically weighted data intelligent filling strategy of the present invention achieves high-fidelity filling of delayed update features by integrating the advantages of local similarity matching, global statistical laws and high-precision prediction models.

[0040] Step 510: Perform missing value distribution analysis on the dataset and calculate the missing rate of each feature; construct a missing marker matrix with the feature sample dataset as the row vector and the feature as the column vector, and use 0 / 1 identifiers to record the missing status of each delayed update feature, so as to determine the data missing time period of each delayed update feature.

[0041] Step 520: Construct a multi-fill method coordination method.

[0042] As an example, this step 520 may include the following sub-steps: Step 5201: Local similarity matching (KNN filling).

[0043] Measuring the similarity of non-missing features in a dataset based on Euclidean distance , adaptively determine the optimal number of neighbors through the elbow rule k ; Then, k The corresponding eigenvalues ​​of the nearest neighbors are Gaussian kernel weighted, and the weight formula is as follows: ; ; In the above formula, p is the number of features, Indicates the i The data point in s The value of a feature; Indicates the j The data point in s The value of a feature; represents the Gaussian kernel weight; Represents data points The distance to the target data point; Represents data points The distance to the target data point; Indicates bandwidth parameter; Indicates the number of neighbors.

[0044] KNN padding value: ; in, is the result of filling missing values, It is i The values ​​of the nearest neighbors, is its corresponding weight.

[0045] Step 5202: Global statistics filling.

[0046] Calculate the global statistics of the data set and select the mean based on the distribution characteristics of the data or majority This method is simple and intuitive, can quickly handle missing values, and is especially suitable for situations where the data distribution is relatively stable.

[0047] ; In the above formula, Indicates the global statistics filling result.

[0048] Step 5203: XGBoost predictive modeling.

[0049] The missing features are used as labels and the holiday features are used as inputs. The XGBoost model is trained using the non-missing data. The accuracy of the model validation set is used to measure the reliability of the model, which is denoted as . Input the holiday feature data corresponding to the missing feature to get the predicted value : ; In the above formula, Represents the regression tree data training set space, Represents the leaf node weight of the tree; is the number of independent trees, Indicates the l An independent tree.

[0050] Step 530: Fill the KNN value , global statistics and XGBoost prediction values A weighted calculation is performed to obtain the second characteristic data, thereby obtaining a filling result of the delayed update characteristic data.

[0051] The specific formula is as follows: ; ; ; Normalized weights: ; ; ; ; In the above formula, Fill value weights for KNN; is the average distance of the nearest neighbors, reflecting local consistency; is the global statistical value weight; is the characteristic variance. The smaller the variance, the more reliable the global statistics. The weight of the XGBoost prediction value; is the model confidence, normalized to [0, 1]; represents the second feature data; Represents the normalized KNN filling value weight; Represents the normalized global statistical value weight; Represents the normalized XGBoost prediction value weight.

[0052] Step 600: Perform data fusion on the first feature data and the second feature data using principal component analysis.

[0053] The feature data obtained through the data intelligent filling method is fused with the feature data obtained through similar day screening to reduce the data dimension while retaining the main information and improving the prediction performance and efficiency of the model.

[0054] Step 610: The second feature data obtained by the data intelligent filling method and the first feature data obtained by screening similar days are normalized.

[0055] Eliminate the dimensional effects between different features so that each feature can be compared and analyzed on the same scale. Center the feature data by subtracting the mean of each feature: ; in, It is the data after centralized processing. is the feature data, is the mean of the feature data.

[0056] Step 620: Calculate the covariance matrix based on the standardized data.

[0057] Measure the correlation between features to understand which features have a strong linear relationship.

[0058] ; in, z is the sample size; express The transpose of .

[0059] Step 640: Perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors.

[0060] Solve for the eigenvalues ​​and eigenvectors of the covariance matrix to find the main directions of variation in the data.

[0061] Solution , and get the eigenvalue and eigenvectors .

[0062] The eigenvalue represents the variance in the direction of the corresponding eigenvector, and the eigenvector represents the change pattern of the data in that direction.

[0063] Step 650: Calculate the cumulative contributions of the principal components and determine the number of principal components to be retained.

[0064] Calculate the cumulative contribution rate of each principal component, that is, the ratio of the sum of the eigenvalues ​​of the first a principal components to the sum of all eigenvalues. Determine the number of principal components to be retained based on the cumulative contribution rate. You can set a threshold and retain the minimum number of principal components whose cumulative contribution rate reaches the threshold.

[0065] Step 700: constructing an electricity price prediction model based on the principal component characteristics, and training the electricity price prediction model to obtain a trained electricity price prediction model.

[0066] The present invention uses the LGBM (Light Gradient Boosting Machine) algorithm to build an electricity price prediction model, and uses the principal component features as the input features of the LGBM model.

[0067] Specifically, the principal component features are used as training data, and the corresponding real electricity price data are used as labels.

[0068] The initialized LGBM model is trained using the training set data. After training, the electricity price prediction model can be evaluated using the validation set. Common regression metrics, such as mean squared error (MSE), root mean squared error (RMSE), and mean absolute error (MAE), are calculated to measure the model's predictive performance. The mean squared error is the average of the squares of the differences between the predicted and true values, reflecting the overall degree of deviation between the predicted and true values. The root mean squared error is the square root of the mean squared error, which has the same units as the original data and more intuitively represents the magnitude of the prediction error. The mean absolute error is the average of the absolute values ​​of the differences between the predicted and true values, reflecting the average magnitude of the prediction error.

[0069] To improve the performance of the LGBM electricity price forecasting model, we used the Optuna framework to optimize its parameters. Optuna is a hyperparameter optimization tool based on Bayesian optimization. It uses intelligent search strategies to find the optimal hyperparameter combination with the fewest possible experiments. Its core advantage lies in its ability to efficiently explore the hyperparameter space and dynamically adjust the search direction based on early results, thereby accelerating the optimization process.

[0070] First, clarify the model parameters that need to be tuned and their value ranges, select appropriate hyperparameters, such as learning rate, regularization coefficient, number of layers, etc., and define a reasonable range of hyperparameters.

[0071] Then, Optuna uses its intelligent search algorithms, such as tree-structured parameter optimization, to perform multiple iterations of optimization. During each iteration, Optuna dynamically adjusts the search strategy based on the current best results to improve efficiency and effectiveness.

[0072] Finally, Optuna will output the optimal hyperparameter combination and its corresponding model performance indicators after multiple iterations.

[0073] according to Figure 2-Figure 8 As shown in the comparison chart of electricity price forecasting, the electricity price forecasting model based on the collaborative optimization of similar day extraction and intelligent data filling shows significant performance improvements compared to the baseline model constructed using a single LGBM algorithm. The improved model has achieved breakthroughs in curve fitting and the accuracy of electricity price peak and valley forecasts. This enhanced forecasting capability provides electricity market participants with a more accurate price signal analysis tool, which not only helps power generation companies optimize their unit combination strategies but also assists power retail companies in developing risk hedging plans, thereby comprehensively improving the operational efficiency and decision-making reliability of the electricity spot market.

[0074] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for predicting electricity prices based on similar days and data filling, characterized in that: The following steps are involved: Obtain power grid supply and demand data to form an initial electricity price prediction dataset; perform data preprocessing on the initial electricity price prediction dataset and select features with high correlation with electricity prices to form an electricity price prediction dataset; Based on the selected features with high correlation with electricity prices, the similarity of the feature vectors of the predicted day and the historical days is calculated; the random forest feature weight of each feature is calculated using random forest, and the final similarity of each feature is obtained by weighted average of the similarity and the random forest feature weight. This is used to screen similar days and obtain the first feature data; Fill in the missing features to obtain the second feature data; Performing principal component analysis on the first feature data and the second feature data to obtain principal component features; Based on the principal component characteristics, an electricity price prediction model is constructed, and the electricity price prediction model is trained to obtain a trained electricity price prediction model.

2. The electricity price prediction method based on similar days and data filling according to claim 1, characterized in that: The preprocessing includes random missing value processing, outlier detection and correction, and duplicate data processing.

3. The electricity price prediction method based on similar days and data filling according to claim 1 is characterized in that: The features with high correlation with electricity prices are selected to form the electricity price prediction dataset, including: Extract calendar information data from timestamp data and generate time features; For the preprocessed electricity price prediction data set, the Pearson correlation coefficient between each feature and the electricity price is calculated, and the features with high correlation with the electricity price are screened out to form the electricity price prediction data set, where each feature includes the original features in the preprocessed electricity price prediction data set and the generated time features.

4. The electricity price prediction method based on similar days and data filling according to claim 1 is characterized in that: Fill in the missing features to obtain the second feature data, including: Fill in missing features based on KNN, global statistics and XGBoost prediction models to obtain KNN filling values, global statistics values ​​and XGBoost prediction values; The KNN filling value, global statistical value and XGBoost prediction value are weighted to obtain the second feature data.

5. The electricity price prediction method based on similar days and data filling according to claim 1 is characterized in that: Perform principal component analysis on the first feature data and the second feature data to obtain principal component features, including: performing standardization processing on the first characteristic data and the second characteristic data; Calculate the covariance matrix based on the standardized data; Perform eigenvalue decomposition on the covariance matrix to obtain the eigenvalues ​​and corresponding eigenvectors; Calculate the cumulative contribution of the principal components, determine the number of principal components to be retained, and then obtain the principal component characteristics.

6. The electricity price prediction method based on similar days and data filling according to claim 1, characterized in that: The LGBM algorithm is used to construct an electricity price prediction model.

Images