Day-ahead electricity price prediction method and system based on similar day adaptive screening and SHAP compensation

Abstract

The invention discloses a day-ahead electricity price prediction method and system based on similar day adaptive screening and SHAP compensation. The method mainly comprises the following steps: preprocessing historical electricity price, load, new energy output and meteorological data; calculating the maximum information coefficient of the features and the electricity price based on a sliding window, and carrying out the self-adaptive screening of similar days through combining the clustering and grey correlation degree after dynamic weighting; the method comprises the following steps: constructing a prediction model by fusing a conditional variation auto-encoder with a multi-head attention mechanism, capturing historical sequence multi-scale time sequence characteristics and future condition information through a double-path encoder, and generating a day-ahead electricity price point prediction result; and performing feature contribution decomposition on the prediction error by using an SHAP tool, and training an error compensation model to correct an initial prediction value. The method can dynamically adapt to market changes, the prediction precision and interpretability are improved, closed-loop self-optimization is achieved, and reliable decision support is provided for electricity market transactions.

Description

Technical Field

[0001] This invention belongs to the field of power market data analysis and forecasting technology, and specifically relates to a method and system for forecasting day-ahead electricity prices in the power market that combines dynamic sample screening, deep learning, and interpretable artificial intelligence (XAI). Background Technology

[0002] With the deepening of power market reforms and the increasing proportion of renewable energy, the volatility, nonlinearity, and uncertainty of day-ahead electricity prices have significantly increased. Accurate day-ahead electricity price forecasts are crucial for the trading decisions and risk management of market participants such as power generation companies, electricity retailers, and large users.

[0003] Existing day-ahead electricity price forecasting methods suffer from the following shortcomings: Traditional methods often employ a "similar day" selection strategy based on fixed indicators or static distance metrics, which struggles to adapt to dynamic changes in market conditions, resulting in insufficient representativeness of training samples and poor model generalization ability. While deep learning models perform well in time series forecasting, their ability to fuse multi-source information and capture multi-modal time series features in parallel still needs improvement. Traditional models struggle to adaptively weigh the impact of different historical moments on future predictions. Deep learning models are often considered "black boxes," lacking transparency in prediction results and making it difficult to trace the source of prediction bias. Existing interpretability methods are mostly used for "post-hoc explanations," failing to effectively integrate with model optimization and error correction loops, resulting in a lack of effective self-correction mechanisms when systematic biases occur in practical applications.

[0004] Therefore, there is an urgent need for a new day-ahead electricity price forecasting method that can dynamically adapt to market changes, deeply integrate multi-source information, and possess interpretability and self-optimization capabilities. Summary of the Invention

[0005] (I) Purpose of the Invention

[0006] To address the problems in the prior art, this invention proposes a CVAE-MHA day-ahead electricity price forecasting method and system that integrates similar-day adaptive screening and SHAP compensation. This invention aims to improve forecast accuracy, model generalization ability, and result interpretability, and to achieve automatic identification and compensation of forecast errors.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A day-ahead electricity price forecasting method based on similar day adaptive screening and SHAP compensation includes the following steps:

[0010] (1) Data preprocessing and feature engineering: Historical data, including day-ahead electricity prices, system load, renewable energy output, conventional power output, and meteorological data, were obtained from the power trading center database. The raw data was cleaned, including missing value imputation and outlier handling, and time-series features were constructed.

[0011] (2) Adaptive Selection of Similar Days: This step aims to dynamically select the sample set most similar to the predicted day from historical data as training data for the model. Based on data within a sliding time window (e.g., the most recent 3 months), the maximum information coefficient (MIC) between each feature and the target electricity price is recalculated periodically to generate a dynamic feature weight vector to capture the temporal changes of the dominant factors in electricity prices. An unsupervised clustering algorithm is used to cluster the feature vectors of historical days, dividing them into different clusters. Within the same cluster, the weighted grey correlation degree between the predicted day and historical days is calculated by combining the dynamic MIC weights obtained in step S2.1, and finally, the historical days with the highest correlation degree are selected as similar days.

[0012] (3) Electricity Price Forecasting Based on the CVAE-MHA Model: A forecasting model is constructed using a Conditional Variational Autoencoder (CVAE) fused with a Multi-Head Attention (MHA) mechanism. This model comprises a dual-path encoder and a decoder. The historical sequence encoder processes historical sequence data obtained through similar day filtering. This encoder integrates a multi-head attention mechanism, designing different attention heads to capture various key time-series patterns such as short-term fluctuations, intraday cycles, and weekly cycles in parallel. The conditional information encoder processes known future conditional information for the forecast day. Information Fusion and Forecasting: The outputs of the historical sequence encoder and the conditional information encoder are fused to generate the distribution parameters of the latent space. The latent variable z is obtained through reparameterization techniques and input into the decoder to generate the predicted electricity price for the next 24 hours.

[0013] (4) Interpretability Analysis and SHAP Error Compensation: Compare the model's prediction results with the actual values ​​to calculate the point prediction error. Use the SHAP interpretability tool to decompose the error at each prediction point, quantify the contribution of each input feature to the error, and identify the key features causing the prediction bias. Train an error compensation model (such as linear regression, gradient boosting tree, etc.) using the prediction results of the base model, the original features, and their SHAP values ​​as input. This model outputs the error compensation amount for each prediction time. Apply the error compensation amount to the initial prediction value of the base model to obtain the corrected final day-ahead electricity price prediction value.

[0014] (III) Beneficial Effects

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] (1) Dynamic adaptive capability: The similar day adaptive screening mechanism implemented by sliding window MIC and clustering algorithm can dynamically respond to changes in market status, significantly improve the quality and representativeness of training samples, and enhance the generalization ability of the model in complex and ever-changing market environments.

[0017] (2) Precise time series modeling: The innovative CVAE-MHA model structure effectively integrates multi-source future information through a conditional encoder and uses a multi-head attention mechanism to capture multi-scale time series patterns in parallel, thereby more accurately depicting the electricity price formation mechanism and improving the accuracy of point prediction.

[0018] (3) Interpretability and self-optimization: The interpretability analysis of SHAP is upgraded from "post-expectation interpretation" to "in-expectation intervention". By constructing an error compensation model, the automatic identification, attribution and correction of systematic biases in prediction are realized, forming a closed loop of "prediction-interpretation-compensation-optimization", continuously improving the accuracy and robustness of the model, while enhancing the transparency and credibility of the prediction results.

[0019] (4) Systematic Engineering: This invention provides a complete end-to-end prediction system framework. The modules work together and have good practicality and engineering application value, which can provide more reliable and transparent decision support for electricity market participants. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of adaptive filtering based on similar dates.

[0021] Figure 2 This is a schematic diagram of the CVAE-MHA electricity price prediction model.

[0022] Figure 3 This is a schematic diagram of the SHAP error compensation model. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Example 1:

[0025] This embodiment uses the Shaanxi power market as an example to illustrate the implementation process of this method.

[0026] (1) Data preparation: Collect data on day-ahead electricity prices, load, wind and solar power output forecasts, and temperature in the market over the past year.

[0027] (2) Similar day filtering: Set the sliding window to 90 days and calculate the MIC weight. Use DBSCAN to cluster the historical data, and then calculate the weighted cosine similarity to filter out the 30 most similar historical days for the prediction date.

[0028] (3) Model training and prediction: Construct the CVAE-MHA model with a historical sequence length of 168 hours (7 days) and a future condition information of 24 hours. Train the model using selected similar daily data and output the initial predicted electricity price for the next 24 hours.

[0029] (4) Error Compensation: After the model is put into online operation, new actual electricity price data is continuously collected. The prediction error of the previous day is analyzed daily using SHAP, and a lightweight gradient boosting tree (GBDT) model is trained as a compensator to correct the prediction results of the day in real time.

[0030] Experimental comparisons show that the prediction results using the method of this invention have a mean absolute error (MAE) and root mean square error (RMSE) that are reduced by approximately 15% and 18% respectively compared to the traditional LSTM model and static similarity day screening method. Furthermore, the prediction interval is more reliable, and the model decision-making process can be clearly interpreted through the SHAP value.

[0031] This invention is not limited to the specific embodiments described above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of this invention should also fall within the protection scope of this invention.

Claims

1. A CVAE-MHA day-ahead electricity price forecast integrating similar day adaptive screening and SHAP compensation, characterized in that, Includes the following steps: (1) Obtain the day-ahead data disclosed by a provincial power trading center from the database, specifically including day-ahead electricity price, system load value, hydropower output value, nuclear power output value, new energy output value, etc. (2) Preprocessing of raw data includes filling missing values ​​and handling outliers, followed by feature engineering to provide high-quality input for subsequent modeling; (3) Construct a similar day adaptive filtering module, wherein: - Based on a sliding time window, the MIC coefficients of each feature and electricity price are periodically recalculated to generate a dynamic weight vector; - An unsupervised clustering algorithm is introduced to cluster historical days, calculate the cluster to which the predicted day belongs, calculate the weighted cosine similarity of historical days within the cluster, obtain the ranking of similar days, and select the top-k most similar historical days as the training set. (4) SHAP-compensated CVAE-MHA day-ahead electricity price forecasting module, wherein: - A dual-path encoder design is adopted, in which the historical sequence encoder processes data such as electricity prices, load, and renewable energy output from past days, while the conditional information encoder processes load forecasts, renewable energy generation, and meteorological factors for the next 24 hours. To enhance the ability of different time points in the historical sequence to distinguish future forecasts, a multi-head attention mechanism is introduced into the encoder. Different attention heads capture short-term fluctuations, intraday cycles, intraweek cycles, and anomalous events respectively, and generate a multi-scale comprehensive sequence representation through weighted averaging. - After fusing historical and conditional information, the model generates the distribution parameters μ and σ of the latent space. 2 The latent variable z is obtained through reparameterized sampling and input into the decoder to generate the electricity price forecast for the next 24 hours. (5) Interpretability Feedback Module: This module compares the model's prediction results with the actual values, calculates the point prediction error, and decomposes the point prediction error using XAI tools such as SHAP during the interpretability analysis phase, quantifying the contribution of each input feature to the error. A point prediction error compensation module is constructed, whose inputs include the basic model prediction results, original features, and SHAP values, and outputs the error compensation amount at each time step to correct systematic biases. Applying the compensation amount to the basic prediction yields the corrected point prediction values. (6) Integrate the above modules into a unified end-to-end system to realize the complete process from data input, similar day screening, electricity price forecasting, interpretation and analysis to feedback optimization.

2. The method according to claim 1, characterized in that, The sliding time window in step (3) is 3 months in length, and the unsupervised clustering algorithm is the DBSCAN algorithm.

3. The method according to claim 1, characterized in that, The multi-head attention mechanism described in step (4) includes four attention heads.

4. The method according to claim 1, characterized in that, The SHAP error analysis in step (5) uses the SHAP algorithm to perform feature attribution analysis on the error at each prediction time.

5. The method according to claim 1, characterized in that, The error compensation module in step (5) uses a gradient boosting decision tree model, taking the SHAP value and the original features as inputs, to predict the amount of error compensation.

Images