Day-ahead market electricity price prediction method and device based on causal reinforcement learning

Abstract

The invention discloses a day-ahead market electricity price prediction method and device based on causal reinforcement learning, and relates to the technical field of electricity market analysis and prediction.The method comprises the steps that a causal graph between variable relations related to the electricity price is constructed, and causal variables having a direct causal relation with the electricity price are obtained through screening according to the causal graph; for each causal variable, determining a dynamic weight corresponding to the causal variable in a reinforcement learning state space according to a causal contribution degree and a recent fluctuation rate of the causal variable; the strategy network is called to determine an LSTM hidden layer weight adjustment amount based on a preset action range constraint, and the LSTM hidden layer weight is finely adjusted based on the adjustment amount; superposing a causal attention layer on the top of the LSTM, and setting a causal contribution degree calculation mechanism input by each time step on the causal attention layer; performing disturbance processing on key variables in the causal variables, and simulating an electricity price change interval; and performing day-ahead market electricity price prediction based on LSTM.

Description

Technical Field

[0001] This invention relates to the field of electricity market analysis and forecasting technology, and in particular to a day-ahead market electricity price forecasting method and apparatus based on causal reinforcement learning. Background Technology

[0002] Current market electricity price forecasting is a core component of electricity market operations, directly impacting generation plans, trading strategies, and grid stability. Traditional methods primarily rely on the following technologies: Technique 1: Day-ahead market electricity price forecasting based on statistical models. This approach relies on the linear assumption of time series data and is difficult to capture non-linear fluctuations in electricity prices.

[0003] Technique 2: Day-ahead electricity price forecasting based on machine learning models, such as Support Vector Machine (SVM), Random Forest, and Neural Network (LSTM), can handle nonlinear relationships, but they are mostly black-box models and lack causal interpretability.

[0004] Technique 3: Day-ahead market electricity price forecasting based on reinforcement learning (RL). This technique attempts to use RL for dynamic electricity price forecasting, but its reward function design does not consider the causal mechanism behind electricity price fluctuations, resulting in insufficient robustness of the strategy under sudden events such as extreme weather and policy adjustments.

[0005] It is evident that existing technologies suffer from the following main shortcomings: 1. Neglecting causal relationships: Traditional data-driven models rely solely on statistical correlation, failing to model the causal mechanisms between electricity prices and external factors such as fuel prices, weather, and policies, leading to prediction bias. 2. Poor dynamic adaptability: Reinforcement learning strategies do not incorporate causal structures, making it difficult to distinguish between short-term noise and long-term trends, and are prone to failure in non-steady-state market environments. 3. Insufficient interpretability: Black-box models cannot provide decision-making support, making it difficult to support the trust and intervention of electricity traders. Summary of the Invention

[0006] The purpose of this invention is to provide a day-ahead market electricity price prediction method and apparatus based on causal reinforcement learning, which can solve at least one of the above-mentioned problems in the prior art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: This invention provides a day-ahead electricity price forecasting method based on causal reinforcement learning, wherein the method includes: Construct a causal graph of the relationships between various variables related to electricity prices, and filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph; For each causal variable, the dynamic weight corresponding to the causal variable is determined in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable. The policy network determines the adjustment amount of the LSTM hidden layer weights based on the preset action range constraints, and fine-tunes the LSTM hidden layer weights based on the adjustment amount. A causal attention layer is superimposed on top of the LSTM, and a causal contribution calculation mechanism for each time step input is set in the causal attention layer; Perturbation processing is applied to the key variables in the causal variables to simulate the range of electricity price fluctuations; Day-ahead electricity price forecasting is performed based on the LSTM.

[0008] Optionally, the step of constructing a causal graph of the relationships between various variables related to electricity prices, and filtering out causal variables with a direct causal relationship to electricity prices based on the causal graph, includes: A causal graph is constructed using a hybrid causal discovery algorithm; Based on the causal graph, the first variable that has a direct causal relationship with electricity price was obtained; By eliminating spurious variables from the first variable using a pre-defined causal effect testing algorithm, causal variables with a direct causal relationship to electricity prices are obtained.

[0009] Optionally, the causal contribution of the causal variable is calculated as follows: Determine the path coefficients and SHAP values ​​of the causal variables in the causal graph; Based on the path coefficients and SHAP values, the causal contribution of the causal variables is calculated.

[0010] Optionally, the causal contribution calculation mechanism for the time step input includes: Determine the target of interest at the current time step, and the historical time periods in the historical time steps that match the target; Determine the importance of each of the aforementioned historical periods; Based on the aforementioned historical time periods and their corresponding importance, a query / key vector for causal variables is established; Based on the query / key vector and the length of the vector, the causal contribution of the input at the current time step is calculated.

[0011] Optionally, the method further includes: Receive the user's first action and display the user interface; Receive a second operation from the user on the user interface and determine the combination of causal variables set by the user; When performing day-ahead market electricity price forecasting based on the LSTM, a forecast curve and risk probability are generated.

[0012] Optionally, the method further includes: Based on the path coefficients of each causal variable in the causal graph, calculate the confidence level of the causal effect corresponding to each causal variable; When the confidence level of the causal effect between variables corresponding to any causal variable exceeds the historical confidence interval, causal graph relearning is triggered to update the causal graph. When a change is detected in the causal graph in the LSTM, a matching target prediction strategy is selected from the pre-stored prediction strategies in the system using a gradient matching algorithm, and the system quickly migrates to the target prediction strategy. The system employs a meta-learning framework to pre-train and store the prediction strategies of the LSTM under multiple sets of causal graphs. A causal invariance penalty term is introduced into the reward function of the LSTM to generate an updated LSTM.

[0013] Optionally, the method further includes: The first causal variable whose causal contribution in the causal graph meets the set conditions is monitored. When the causal contribution of any first causal variable exceeds the preset dynamic threshold, the corresponding level of warning is triggered. Based on the causal graph, Bayesian network inference is performed to locate the source of the anomaly. Generate risk hedging strategies.

[0014] Optionally, the step of generating a risk hedging strategy includes: When a fuel price surge warning is triggered, the integrated financial derivatives pricing model calculates the optimal amount of call options to be purchased for the fuel price and generates dynamic hedging recommendations. A risk constraint term is added to the reinforcement learning framework. This constraint term is used to limit the maximum drawdown of the policy network in extreme scenarios to ensure a balance between returns and risks.

[0015] This invention also provides a day-ahead market electricity price forecasting device based on causal reinforcement learning, wherein the device includes: The module is used to construct a causal graph of the relationships between various variables related to electricity prices, and to filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph. The determination module is used to determine the dynamic weight corresponding to each causal variable in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable. The fine-tuning module is used to call the policy network to determine the adjustment amount of the LSTM hidden layer weights based on the preset action range constraints, and to fine-tune the LSTM hidden layer weights based on the adjustment amount. The overlay module is used to overlay a causal attention layer on top of the LSTM and set the causal contribution calculation mechanism of the input at each time step in the causal attention layer. The perturbation module is used to perturb the key variables in the causal variables to simulate the range of electricity price fluctuations. The forecasting module is used to forecast day-ahead market electricity prices based on the LSTM.

[0016] Optionally, the building module is specifically used for: A causal graph is constructed using a hybrid causal discovery algorithm; Based on the causal graph, the first variable that has a direct causal relationship with electricity price was obtained; By eliminating spurious variables from the first variable using a pre-defined causal effect testing algorithm, causal variables with a direct causal relationship to electricity prices are obtained.

[0017] Optionally, the causal contribution of the causal variable is calculated as follows: Determine the path coefficients and SHAP values ​​of the causal variables in the causal graph; Based on the path coefficients and SHAP values, the causal contribution of the causal variables is calculated.

[0018] Optionally, the causal contribution calculation mechanism for the time step input includes: Determine the target of interest at the current time step, and the historical time periods in the historical time steps that match the target; Determine the importance of each of the aforementioned historical periods; Based on the aforementioned historical time periods and their corresponding importance, a query / key vector for causal variables is established; Based on the query / key vector and the length of the vector, the causal contribution of the input at the current time step is calculated.

[0019] Optionally, the device further includes: The first receiving module is used to receive the user's first operation and display the user interface. The second receiving module is used to receive the user's second operation on the user interface and determine the combination of causal variables set by the user; In the generation module, when performing day-ahead market electricity price forecasting based on the LSTM, a forecast curve and risk probability are generated.

[0020] Optionally, the device further includes: The confidence calculation module is used to calculate the confidence level of the causal effect corresponding to each causal variable based on the path coefficient of each causal variable in the causal graph. The causal graph update module is used to trigger causal graph relearning and update the causal graph when the confidence level of the causal effect between variables corresponding to any causal variable exceeds the historical confidence interval. The transfer module is used to select a matching target prediction strategy from the pre-stored prediction strategies in the system through a gradient matching algorithm when a change in the causal graph in the LSTM is detected, and quickly transfer to the target prediction strategy; wherein, the system adopts a meta-learning framework to pre-train and store the prediction strategies of the LSTM under multiple sets of causal graphs. An introduction module is used to introduce a causal invariance penalty term into the reward function of the LSTM to generate an updated LSTM.

[0021] Optionally, the device further includes: The monitoring module is used to monitor the first causal variable whose causal contribution in the causal graph meets the set conditions. When the causal contribution of any first causal variable exceeds the preset dynamic threshold, the corresponding level of warning is triggered. The anomaly localization module is used to perform Bayesian network inference based on the causal graph to locate the source of the anomaly. The strategy generation module is used to generate risk hedging strategies.

[0022] Optionally, the strategy generation module is specifically used for: When a fuel price surge warning is triggered, the integrated financial derivatives pricing model calculates the optimal amount of call options to be purchased for the fuel price and generates dynamic hedging recommendations. A risk constraint term is added to the reinforcement learning framework. This constraint term is used to limit the maximum drawdown of the policy network in extreme scenarios to ensure a balance between returns and risks.

[0023] This invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor, when executing the program stored in the memory, implements any of the above-described day-ahead market electricity price forecasting methods based on causal reinforcement learning.

[0024] This invention discloses a day-ahead market electricity price forecasting scheme based on causal reinforcement learning. The scheme constructs a causal graph relating various electricity price-related variables and uses this graph to identify causal variables directly causally related to electricity prices. For each causal variable, dynamic weights are determined in the reinforcement learning state space based on its causal contribution and recent volatility. A policy network is invoked to determine the adjustment amount of the LSTM hidden layer weights based on preset action range constraints, and the LSTM hidden layer weights are fine-tuned based on this adjustment. A causal attention layer is superimposed on top of the LSTM, and a causal contribution calculation mechanism is set in the causal attention layer for each time step. Perturbations are applied to key variables among the causal variables to simulate electricity price fluctuation ranges. Day-ahead market electricity price forecasting is then performed based on the LSTM. This scheme, by identifying key drivers of electricity price fluctuations through a causal graph and designing a causal-aware reinforcement learning strategy to dynamically optimize the forecasting mechanism, effectively solves the problems of neglecting causal relationships and poor dynamic adaptability in existing technologies. This scheme can significantly improve the prediction accuracy and robustness of LSTM for day-ahead market electricity price forecasting. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the steps of a day-ahead market electricity price forecasting method based on causal reinforcement learning, according to an embodiment of this application. Figure 2 This is a structural block diagram illustrating a day-ahead market electricity price forecasting device based on causal reinforcement learning, according to an embodiment of this application. Detailed Implementation

[0026] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0027] This invention provides a day-ahead market electricity price forecasting method combining causal inference and reinforcement learning (RL), applicable to electricity market operation, energy trading decisions, and smart grid optimization. The day-ahead market electricity price forecasting method based on causal reinforcement learning provided in this application mainly includes the following parts: 1. Causal modeling: Identifying key drivers of electricity price fluctuations, such as fuel costs and renewable energy output, through a causal graph, and eliminating spurious correlation interference. 2. Dynamic optimization through reinforcement learning: Embedding the causal structure into a reinforcement learning framework, designing a causal-aware reward function and state space to improve the model's adaptability to sudden market events. 3. Interpretability and robustness: Providing causal contribution analysis to assist trading decisions and ensuring the stability of the forecast results in complex market environments.

[0028] The day-ahead market electricity price forecasting method based on causal reinforcement learning provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0029] As attached Figure 1 As shown, the day-ahead market electricity price forecasting method based on causal reinforcement learning in this application includes the following steps: Step 101: Construct a causal graph of the relationships between various variables related to electricity prices, and filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph.

[0030] The day-ahead market electricity price forecasting method based on causal reinforcement learning provided in this application can be executed by an electronic device. The electronic device is equipped with an integrated circuit power supply power fusion self-healing control computer program. When the computer program is executed by the processor, it implements the day-ahead market electricity price forecasting method based on causal reinforcement learning of this application.

[0031] In an optional embodiment, the following method can be used to construct a causal graph of the relationships between various variables related to electricity prices, and to filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph: First, a causal graph is constructed using a hybrid causal discovery algorithm; A combination of hybrid causal discovery algorithms can be a combination of PC algorithms and domain knowledge verification methods.

[0032] Secondly, the first variable that has a direct causal relationship with electricity price was obtained by screening based on the causal graph; The first variable may include, but is not limited to: historical electricity prices, fuel prices, weather data, unit output, policy events, etc.

[0033] Finally, by using a pre-defined causal effect testing algorithm to eliminate spurious variables in the first variable, causal variables with a direct causal relationship to electricity prices are obtained.

[0034] The preset causal effect test algorithm can be the backdoor criterion, the instrumental variable method, etc., and no specific restrictions are imposed on it in the embodiments of this application.

[0035] Step 102: For each causal variable, determine the dynamic weight of the causal variable in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable.

[0036] In practical implementation, dynamic weights can be assigned to each causal variable in the reinforcement learning state space using the following formula:

[0037] in The causal contribution of causal variables. To enhance the real-time impact of key variables by reflecting recent volatility of causal variables.

[0038] In an optional embodiment, the causal contribution of a causal variable can be calculated as follows: First, determine the path coefficients and SHAP values ​​of the causal variables in the causal graph; second, calculate the causal contribution of the causal variables based on the path coefficients and SHAP values.

[0039]

[0040] Among them, the path coefficients of the cause-effect graph can be used Shapley Additive Explanations (SHAP) is a widely used method for interpreting predictions from machine learning models. Because current models are often very complex and difficult to interpret, SHAP was proposed as a predictive explanation framework. The SHAP value is the numerical value obtained by applying the SHAP framework to the model's predictions.

[0041] Steps 101-102 are the causal feature selection and noise filtering process. In order to improve the accuracy of LSTM (Long Short-Term Memory) in predicting day-ahead market electricity prices, dynamic optimization of LSTM model parameters was also carried out on the basis of causal feature selection and noise filtering. LSTM is a recurrent neural network structure.

[0042] Step 103: Invoke the policy network to determine the adjustment amount of the LSTM hidden layer weights based on the preset action range constraints, and fine-tune the LSTM hidden layer weights based on the adjustment amount.

[0043] This step involves the adaptive adjustment of the LSTM hidden layer and the causal focusing process of the attention mechanism.

[0044] In practical implementation, the reinforcement learning action space can be designed as an array of LSTM hidden layer weights. Fine-tuning, with the range of motion constrained. By using the policy network Actor to output weight adjustments, the model's ability to capture nonlinear patterns can be optimized in real time.

[0045] Step 104: Overlay a causal attention layer on top of the LSTM and set the causal contribution calculation mechanism for the input at each time step in the causal attention layer.

[0046] In one alternative embodiment, the causal contribution calculation mechanism for the time step input can be as follows: Identify the target of interest at the current time step, and the historical time periods in the historical time steps that match the target; Determine the importance of each historical period; based on each historical period and its corresponding importance, establish query / key vectors for causal variables; based on the query / key vectors and their lengths, calculate the causal contribution of the input at the current time step.

[0047] Specifically, the causal contribution of a causal variable can be calculated using the following formula:

[0048] Here, Q / K is the query / key vector of the causal variables, which allows the model to focus on key periods such as electricity price peaks.

[0049] Q (Query): Represents the target to focus on at the current time step, such as predicting tomorrow's electricity price.

[0050] K (Key): Represents the feature identifier of a historical time step, used to match with Q to determine which time steps' information is most important for the current forecast. For example, to predict tomorrow's electricity price, we need to find which days have the greatest impact from the electricity consumption data of the past week. Q represents "which historical periods need to be referenced for tomorrow's electricity price," while K represents "the feature importance of each historical period."

[0051] If d is too small: the vector's expressive power is insufficient, and it may be unable to distinguish between critical and non-critical periods. If d is moderate: it can effectively capture complex time dependencies, such as the correlation between electricity price peaks and weather and load. If d is too large: it increases computational cost and may lead to overfitting. In practical implementation, d is an empirical value. In time series tasks, d is usually consistent with the hidden layer dimension of LSTM, such as 64 or 128, to maintain the continuity of information transmission.

[0052] In this embodiment, by filtering causal features and adjusting dynamic parameters, noise interference can be reduced, and the mean absolute error (MAE) can be reduced by 15%-25%, thereby improving the model's prediction accuracy. MAE is an indicator used to measure the predictive ability of a regression model; it measures the model's predictive ability by calculating the difference between the true value and the predicted value.

[0053] Step 105: Perturb the key variables in the causal variables to simulate the range of electricity price fluctuations.

[0054] This step is a method to enhance the interpretability of LSTM. This method mainly involves two aspects: firstly, quantifying the causal contribution of causal variables; and secondly, adding a sensitivity analysis module to LSTM. This sensitivity analysis module can perform counterfactual reasoning and provide scenario simulation tools.

[0055] In its implementation, counterfactual reasoning primarily simulates electricity price fluctuations by perturbing key variables and outputting prediction results. For example, assuming a 10% increase in natural gas prices, the simulated electricity price fluctuation range would output "Expected electricity price increase range: 5.8%~7.3%".

[0056] It should be noted that the simulated electricity price fluctuation range can be set at certain intervals, such as every 5%, for example, when the natural gas price increases by 5%, 10%, or 15%. This embodiment of the application does not impose specific limitations on this. When simulating the electricity price fluctuation range, a mapping model can be constructed based on historical data on natural gas prices and electricity prices, thereby enabling electricity price prediction during natural gas price fluctuations.

[0057] The scenario simulation tool provides a graphical user interface (GUI) for users to customize variable combinations. For example, if a user defines the variable combination as "extreme cold wave + 20% reduction in wind power output" through the CUI, the scenario simulation tool will generate a prediction curve and risk probability in real time.

[0058] In one optional implementation, the prediction based on a user-defined variable combination can be achieved by: receiving the user's first operation and displaying the user interface; receiving the user's second operation on the user interface and determining the user-defined causal variable combination; and generating a prediction curve and risk probability when performing day-ahead market electricity price prediction based on LSTM.

[0059] The counterfactual reasoning provided in this embodiment of the invention can further verify the predictive ability of the optimized model, so as to identify the shortcomings of the optimized model and make timely adjustments.

[0060] The interpretability enhancement method provided in this application embodiment can provide causal contribution ranking and sensitivity analysis, helping traders understand the prediction logic and improve the credibility of decision-making.

[0061] Step 106: Forecast day-ahead market electricity prices based on LSTM.

[0062] When using LSTM to forecast day-ahead market electricity prices, users can also define custom combinations of variables to focus on the impact of certain variables on day-ahead market electricity prices.

[0063] In an alternative embodiment, after optimizing the LSTM or performing day-ahead market electricity price forecasting based on the LSTM, a dynamic adaptive method can also be executed, which can update the causal state and transfer the strategy in real time.

[0064] Deploy a causal drift detection module. When the causal effect between variables exceeds the historical confidence interval, trigger causal graph relearning. The update cycle of the causal graph can be flexibly configured, such as 24 hours, 36 hours, 48 ​​hours, etc.

[0065] In one optional embodiment, the method for causal state updating and policy transition may include the following sub-steps: Sub-step 1: Based on the path coefficients of each causal variable in the causal diagram, calculate the confidence level of the causal effect corresponding to each causal variable; Sub-step 2: When the confidence level of the causal effect between variables corresponding to any causal variable exceeds the historical confidence interval, trigger the causal graph relearning and update the causal graph; The confidence interval range of a causal effect refers to the confidence range of the path coefficient values. In this case, it is the confidence interval of the path coefficient between fuel price and electricity price, calculated based on the aforementioned causal graph constructed from historical data. The historical confidence interval is the confidence interval range of the causal effect.

[0066] If the value exceeds the historical confidence interval, it means that the previously calculated confidence interval range no longer conforms to the current reality and needs to be recalculated. The calculation process is the same as the aforementioned causal graph construction process, including the aforementioned prediction accuracy improvement and interpretability enhancement parts, that is, return to execute steps 101 to 105.

[0067] Sub-step 3: When a change is detected in the causal graph in the LSTM, a matching target prediction strategy is selected from the pre-stored prediction strategies in the system using the gradient matching algorithm, and the system is quickly migrated to the target prediction strategy. The system employs a meta-learning framework to pre-train and store the prediction strategies of LSTM under multiple causal graphs.

[0068] In this optional embodiment, for the case of relearning the causal graph, directly matching the optimal prediction strategy to the historically learned causal graph can improve prediction efficiency. Rapid model parameter transfer can reduce the cost of model retraining. The LSTM dynamic adaptation method provided in this application embodiment can shorten the adaptation time from several hours in traditional methods to within 10 minutes.

[0069] Sub-step 4: Introduce a causal invariance penalty term into the reward function of the LSTM to generate the updated LSTM.

[0070] This sub-step involves adding a robust reward design to the causal-robust reward function, introducing a causal invariance penalty term into the reward function:

[0071] in, Indicates the relationship between non-causal variables Intervention forces the LSTM model to ignore irrelevant perturbations.

[0072] The dynamic adaptive method provided in this optional embodiment, which optimizes the reinforcement learning strategy in real time based on causal state, can reduce the fluctuation of prediction error by 30%-40% under sudden events such as extreme weather and policy changes.

[0073] In an optional embodiment, a risk control method is also provided, comprising two parts: causal anomaly detection and early warning, and risk hedging strategy generation. Specifically, the method may include the following sub-steps: Sub-step 1: Monitor the first causal variable whose causal contribution in the causal graph meets the set conditions. When the causal contribution of any first causal variable exceeds the preset dynamic threshold, trigger the corresponding level of warning.

[0074] The setting conditions can be set as the top 20% of causal contribution, the top 15% of causal contribution, or the top 10% of causal contribution, etc. In this embodiment of the application, no specific restrictions are placed on the setting conditions.

[0075] Sub-step 1 is the key variable monitoring step included in the causal anomaly detection and early warning section. In actual implementation, the top 20% of causal variables in terms of causal contribution, such as fuel prices and wind power output, can be monitored in real time. Dynamic thresholds are set, and when the causal contribution of a causal variable exceeds the dynamic threshold, a three-level early warning is triggered. The three-level alarm can include three levels: mild, moderate, and severe.

[0076] Sub-step 2: Perform Bayesian network inference based on the causal graph to locate the source of the anomaly.

[0077] Sub-step 2 is the root cause analysis step included in the causal anomaly detection and early warning section. In this sub-step, Bayesian network inference is performed based on the causal graph to locate the source of the anomaly. For example, if the electricity price is abnormal while the fuel price is normal, the cause can be traced back to a unit failure or transmission constraint.

[0078] Sub-step 3: Generate risk hedging strategies.

[0079] The risk hedging strategy generation process includes two parts: generating dynamic hedging recommendations and multi-objective optimization.

[0080] A feasible way to generate a risk hedging strategy is as follows: First, when a fuel price surge warning is triggered, the integrated financial derivatives pricing model calculates the optimal amount of call options to buy fuel prices and generates dynamic hedging recommendations. Integrated financial derivatives pricing models (such as Black-Scholes) automatically calculate the optimal amount of call options to purchase when a sudden change in fuel prices is detected. The hedging formula is as follows:

[0081] Secondly, risk constraints are added to the reinforcement learning framework.

[0082] The risk constraint term is used to limit the maximum drawdown of the strategy network in extreme scenarios, i.e., Max Drawdown, to ensure a balance between returns and risks.

[0083] The risk control method provided in this optional embodiment can identify potential market risks in advance through causal anomaly detection, such as early warning of sudden changes in fuel prices, and can formulate hedging strategies.

[0084] The steps provided in this application embodiment, except for step 106, are all part of the model training process and have a progressive relationship. First, the prediction accuracy is improved through causal graph learning. Second, the interpretability is improved through causal contribution quantification and sensitivity analysis to make it more consistent with actual influencing factors. Then, a dynamic adaptive mechanism is used to update the model and ensure the stability of the prediction accuracy. Finally, a risk control method is used to ensure the prediction accuracy of key variables and extreme cases.

[0085] The day-ahead market electricity price forecasting method based on causal reinforcement learning provided in this application constructs a causal graph of the relationships between various variables related to electricity prices, and selects causal variables with a direct causal relationship to electricity prices based on the causal graph. For each causal variable, the dynamic weight corresponding to the causal variable is determined in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable. The policy network is invoked to determine the adjustment amount of the LSTM hidden layer weights based on preset action range constraints, and the LSTM hidden layer weights are fine-tuned based on the adjustment amount. A causal attention layer is superimposed on top of the LSTM, and a causal contribution calculation mechanism is set in the causal attention layer for each time step input. The key variables in the causal variables are perturbed to simulate the range of electricity price fluctuations. Day-ahead market electricity prices are predicted based on LSTM. The method provided in this invention identifies key driving factors of electricity price fluctuations through a causal graph and designs a causal-aware reinforcement learning strategy to dynamically optimize the prediction mechanism, which can effectively solve the problems of ignoring causal relationships and poor dynamic adaptability in the prior art. This scheme can significantly improve the prediction accuracy and robustness of LSTM for day-ahead market electricity price forecasting.

[0086] Figure 2 The structural block diagram of a day-ahead market electricity price forecasting device based on causal reinforcement learning is shown in the embodiment of this application.

[0087] The device provided in this application includes the following functional modules: The construction module 201 is used to construct a causal graph of the relationships between various variables related to electricity prices, and to filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph; The determination module 202 is used to determine the dynamic weight corresponding to each causal variable in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable. The fine-tuning module 203 is used to call the policy network to determine the adjustment amount of the LSTM hidden layer weights based on the preset action range constraints, and to fine-tune the LSTM hidden layer weights based on the adjustment amount. The overlay module 204 is used to overlay a causal attention layer on top of the LSTM and set the causal contribution calculation mechanism of the input at each time step in the causal attention layer; The disturbance module 205 is used to perform disturbance processing on the key variables in the causal variables to simulate the range of electricity price fluctuations; The prediction module 206 is used to predict day-ahead market electricity prices based on the LSTM.

[0088] Optionally, the building module is specifically used for: A causal graph is constructed using a hybrid causal discovery algorithm; Based on the causal graph, the first variable that has a direct causal relationship with electricity price was obtained; By eliminating spurious variables from the first variable using a pre-defined causal effect testing algorithm, causal variables with a direct causal relationship to electricity prices are obtained.

[0089] Optionally, the causal contribution of the causal variable is calculated as follows: Determine the path coefficients and SHAP values ​​of the causal variables in the causal graph; Based on the path coefficients and SHAP values, the causal contribution of the causal variables is calculated.

[0090] Optionally, the causal contribution calculation mechanism for the time step input includes: Determine the target of interest at the current time step, and the historical time periods in the historical time steps that match the target; Determine the importance of each of the aforementioned historical periods; Based on the aforementioned historical time periods and their corresponding importance, a query / key vector for causal variables is established; Based on the query / key vector and the length of the vector, the causal contribution of the input at the current time step is calculated.

[0091] Optionally, the device further includes: The first receiving module is used to receive the user's first operation and display the user interface. The second receiving module is used to receive the user's second operation on the user interface and determine the combination of causal variables set by the user; In the generation module, when performing day-ahead market electricity price forecasting based on the LSTM, a forecast curve and risk probability are generated.

[0092] Optionally, the device further includes: The confidence calculation module is used to calculate the confidence level of the causal effect corresponding to each causal variable based on the path coefficient of each causal variable in the causal graph. The causal graph update module is used to trigger causal graph relearning and update the causal graph when the confidence level of the causal effect between variables corresponding to any causal variable exceeds the historical confidence interval. The transfer module is used to select a matching target prediction strategy from the pre-stored prediction strategies in the system through a gradient matching algorithm when a change in the causal graph in the LSTM is detected, and quickly transfer to the target prediction strategy; wherein, the system adopts a meta-learning framework to pre-train and store the prediction strategies of the LSTM under multiple sets of causal graphs. An introduction module is used to introduce a causal invariance penalty term into the reward function of the LSTM to generate an updated LSTM.

[0093] Optionally, the device further includes: The monitoring module is used to monitor the first causal variable whose causal contribution in the causal graph meets the set conditions. When the causal contribution of any first causal variable exceeds the preset dynamic threshold, the corresponding level of warning is triggered. The anomaly localization module is used to perform Bayesian network inference based on the causal graph to locate the source of the anomaly. The strategy generation module is used to generate risk hedging strategies.

[0094] Optionally, the strategy generation module is specifically used for: When a fuel price surge warning is triggered, the integrated financial derivatives pricing model calculates the optimal amount of call options to be purchased for the fuel price and generates dynamic hedging recommendations. A risk constraint term is added to the reinforcement learning framework. This constraint term is used to limit the maximum drawdown of the policy network in extreme scenarios to ensure a balance between returns and risks.

[0095] The embodiments provided in this application Figure 2 The day-ahead market electricity price forecasting device based on causal reinforcement learning shown can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0096] The day-ahead market electricity price forecasting device based on causal reinforcement learning provided in this application identifies key driving factors of electricity price fluctuations through causal graphs and designs a causal-aware reinforcement learning strategy to dynamically optimize the forecasting mechanism. This effectively solves the problems of ignoring causal relationships and poor dynamic adaptability in existing technologies. This scheme can significantly improve the forecasting accuracy and robustness of LSTM for day-ahead market electricity price forecasting.

[0097] This invention also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0098] Memory, used to store computer programs; When the processor executes the program stored in the memory, it implements the day-ahead market electricity price prediction method based on causal reinforcement learning as shown in the above method embodiments.

[0099] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0100] The communication interface is used for communication between the aforementioned terminal and other devices.

[0101] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0102] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores instructions that, when executed on an electronic device, cause the electronic device to implement the day-ahead market electricity price prediction method based on causal reinforcement learning as described in any of the above embodiments.

[0103] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on an electronic device, causes the electronic device to implement the day-ahead market electricity price prediction method based on causal reinforcement learning as described in any of the above embodiments.

[0104] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0105] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A day-ahead market electricity price forecasting method based on causal reinforcement learning, characterized in that, The method includes: Construct a causal graph of the relationships between various variables related to electricity prices, and filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph; For each causal variable, the dynamic weight corresponding to the causal variable is determined in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable. The policy network determines the adjustment amount of the LSTM hidden layer weights based on the preset action range constraints, and fine-tunes the LSTM hidden layer weights based on the adjustment amount. A causal attention layer is superimposed on top of the LSTM, and a causal contribution calculation mechanism for each time step input is set in the causal attention layer; Perturbation processing is applied to the key variables in the causal variables to simulate the range of electricity price fluctuations; Day-ahead electricity price forecasting is performed based on the LSTM.

2. The method according to claim 1, characterized in that, The steps of constructing a causal graph of the relationships between various variables related to electricity prices, and filtering out causal variables with a direct causal relationship to electricity prices based on the causal graph, include: A causal graph is constructed using a hybrid causal discovery algorithm; Based on the causal graph, the first variable that has a direct causal relationship with electricity price was obtained; By eliminating spurious variables from the first variable using a pre-defined causal effect testing algorithm, causal variables with a direct causal relationship to electricity prices are obtained.

3. The method according to claim 1, characterized in that, The causal contribution of causal variables is calculated as follows: Determine the path coefficients and SHAP values ​​of the causal variables in the causal graph; Based on the path coefficients and SHAP values, the causal contribution of the causal variables is calculated.

4. The method according to claim 1, characterized in that, The causal contribution calculation mechanism for the time step input includes: Determine the target of interest at the current time step, and the historical time periods in the historical time steps that match the target; Determine the importance of each of the aforementioned historical periods; Based on the aforementioned historical time periods and their corresponding importance, a query / key vector for causal variables is established; Based on the query / key vector and the length of the vector, the causal contribution of the input at the current time step is calculated.

5. The method according to claim 1, characterized in that, The method further includes: Receive the user's first action and display the user interface; Receive a second operation from the user on the user interface and determine the combination of causal variables set by the user; When performing day-ahead market electricity price forecasting based on the LSTM, a forecast curve and risk probability are generated.

6. The method according to claim 1, characterized in that, The method further includes: Based on the path coefficients of each causal variable in the causal graph, calculate the confidence level of the causal effect corresponding to each causal variable; When the confidence level of the causal effect between variables corresponding to any causal variable exceeds the historical confidence interval, the causal graph relearning is triggered to update the causal graph. When a change is detected in the causal graph in the LSTM, a matching target prediction strategy is selected from the pre-stored prediction strategies in the system using a gradient matching algorithm, and the system quickly migrates to the target prediction strategy. The system employs a meta-learning framework to pre-train and store the prediction strategies of the LSTM under multiple sets of causal graphs. A causal invariance penalty term is introduced into the reward function of the LSTM to generate an updated LSTM.

7. The method according to claim 1, characterized in that, The method further includes: The first causal variable whose causal contribution in the causal graph meets the set conditions is monitored. When the causal contribution of any first causal variable exceeds the preset dynamic threshold, the corresponding level of warning is triggered. Based on the causal graph, Bayesian network inference is performed to locate the source of the anomaly. Generate risk hedging strategies.

8. The method according to claim 7, characterized in that, The steps for generating a risk hedging strategy include: When a fuel price surge warning is triggered, the integrated financial derivatives pricing model calculates the optimal amount of call options to be purchased for the fuel price and generates dynamic hedging recommendations. A risk constraint term is added to the reinforcement learning framework. This constraint term is used to limit the maximum drawdown of the policy network in extreme scenarios to ensure a balance between returns and risks.

9. A day-ahead market electricity price forecasting device based on causal reinforcement learning, characterized in that, The device includes: The module is used to construct a causal graph of the relationships between various variables related to electricity prices, and to filter out causal variables that have a direct causal relationship with electricity prices based on the causal graph. The determination module is used to determine the dynamic weight corresponding to each causal variable in the reinforcement learning state space based on the causal contribution and recent volatility of the causal variable. The fine-tuning module is used to call the policy network to determine the adjustment amount of the LSTM hidden layer weights based on the preset action range constraints, and to fine-tune the LSTM hidden layer weights based on the adjustment amount. The overlay module is used to overlay a causal attention layer on top of the LSTM and set the causal contribution calculation mechanism of the input at each time step in the causal attention layer. The perturbation module is used to perturb the key variables in the causal variables to simulate the range of electricity price fluctuations. The forecasting module is used to forecast day-ahead market electricity prices based on the LSTM.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the day-ahead market electricity price forecasting method based on causal reinforcement learning as described in any one of claims 1-8.

Images