Wind power prediction method based on reinforcement learning and decision-oriented robust enhancement

Abstract

This invention discloses a wind power forecasting method based on reinforcement learning and decision-oriented robust enhancement. Addressing the shortcomings of existing methods that use prediction error as the model evaluation criterion, which fails to reflect the impact of prediction deviations on day-ahead operation scheduling, and the problems of non-differentiable optimization processes and difficulty in passing gradients to the prediction model when constructing decision evaluation indicators based on scheduling results, this invention constructs the additional operational resource consumption loss caused by prediction deviations as a reinforcement learning reward. The wind power forecasting model is updated using scheduling feedback through policy gradients. A differentiable operational resource consumption surrogate model is trained, generating worst-case adversarial samples within the range of constrained numerical weather forecast disturbances, and robustly training is performed by combining the original samples and adversarial samples. This invention can reduce prediction errors and additional operational resource consumption caused by adversarial disturbances, achieving adversarial robust enhancement of the wind power forecasting model for operation scheduling decisions.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of new energy power prediction in power systems, artificial intelligence security defense, and power system optimal dispatching, and particularly relates to a wind power prediction method based on reinforcement learning and decision-oriented robust enhancement. Background Technology

[0002] In power systems with a high proportion of renewable energy integration, wind power forecasts are typically used as crucial inputs for day-ahead dispatch optimization, reserve allocation, and operational risk assessment. With the increasing application of deep learning models in wind power forecasting, the reliance of these models on numerical weather prediction data is also growing. When input features such as wind speed and direction in numerical weather predictions are subjected to adversarial disturbances or anomalous shifts, wind power forecasts may become biased, further impacting downstream power system day-ahead operation and dispatch decisions.

[0003] Existing wind power prediction models and their robust training methods typically use prediction accuracy metrics such as mean squared error and mean absolute error as the main optimization objectives. While these metrics reflect the numerical difference between predicted and actual wind power, they fail to directly reflect the impact of prediction deviations on day-ahead dispatching results. Smaller prediction deviations may lead to significant power balance adjustments, reserve resource allocation, wind curtailment, or load shedding; larger prediction deviations, however, have a relatively limited impact on dispatching resource consumption. Therefore, simply constructing adversarial examples based on prediction errors and conducting robust training is insufficient to improve the robustness of wind power prediction models for day-ahead dispatching.

[0004] Existing adversarial training methods typically improve the robustness of the model's prediction accuracy when the input is disturbed by constructing adversarial examples that increase the prediction error. However, the adversarial example with the largest prediction error is not necessarily the one that causes the greatest additional loss of operational resources. Directly searching for adversarial examples using the day-ahead operation scheduling optimization problem of the power system is difficult because its solution process is usually non-differentiable, making it difficult to directly calculate the gradient corresponding to the input disturbance, and repeated solutions increase the computational burden. Therefore, a wind power prediction method is needed that can incorporate day-ahead operation scheduling feedback into the training of the wind power prediction model, while balancing the efficiency of adversarial example search and the accuracy of operation scheduling evaluation. Summary of the Invention

[0005] The purpose of this invention is to address the problems that existing wind power prediction adversarial training mainly uses prediction errors as the basis for generating adversarial samples and optimizing models, making it difficult to effectively defend against the additional operational resource consumption losses caused by adversarial disturbance inputs; and the problems that when constructing a decision-oriented wind power prediction model for adversarial training, the day-ahead operation scheduling optimization problem of the power system is not differentiable and it is difficult to directly pass gradients to the input disturbances and the wind power prediction model. The invention provides a wind power prediction method based on a reinforcement learning framework and a decision-oriented robust adversarial mechanism.

[0006] The objective of this invention is achieved through the following technical solution: According to a first aspect of this specification, a wind power prediction method based on reinforcement learning and decision-oriented robust enhancement is provided, the method comprising the following steps: S1: Construct a dataset that includes numerical weather forecast inputs, actual wind power, and power system load; S2: A wind power prediction model pre-trained based on numerical weather forecast inputs and real wind power in the dataset; S3: Establish a day-ahead operation and scheduling optimization problem for the power system. The optimization objective is to minimize the consumption of operating resources. Using day-ahead wind power forecast as input information, solve for the optimal decision variables of synchronous generation output and reserve resource scheduling under the condition of satisfying the system power balance constraints. S4: Input the predicted wind power value into the optimization problem to obtain the prediction-based operation scheduling strategy and the prediction-based operation resource consumption; input the actual wind power corresponding to the wind power prediction into the optimization problem to obtain the ideal operation scheduling strategy and the ideal operation resource consumption based on the actual value; take the difference between the prediction-based operation resource consumption and the ideal operation resource consumption based on the actual value as the additional operation resource consumption loss caused by the wind power prediction deviation. S5: Construct a continuous action reinforcement learning strategy, using the wind power prediction model as the strategy mean function and the numerical weather forecast input as the state. Sample the predicted actions according to the strategy distribution and use the predicted actions as the wind power prediction values ​​for the optimization problem. Calculate the additional operating resource consumption loss based on the predicted actions, the actual wind power, and the power system load. Write the predicted actions, the actual wind power, the power system load, and the additional operating resource consumption loss into the cache. S6: Train a differentiable operational resource consumption proxy model using buffer data. The input of the proxy model is the predicted action, the actual wind power, and the power system load. The output is the estimated additional operational resource consumption loss. S7: Based on the wind power prediction model and the surrogate model, calculate the gradient of the numerical weather forecast input with respect to the estimated additional operating resource consumption loss, use the projected gradient descent algorithm to construct adversarial perturbations on the numerical weather forecast input, maximize the estimated additional operating resource consumption loss, and generate multiple candidate adversarial samples. S8: Based on the proxy model, sort all candidate adversarial samples, select several adversarial samples with the highest estimated additional runtime resource consumption loss, call the optimization problem to calculate the additional runtime resource consumption loss caused by the selected adversarial samples, and select the adversarial sample with the largest loss as the worst adversarial sample. S9: The original numerical weather forecast input and the worst adversarial sample are respectively used as the input state of the reinforcement learning strategy. The prediction action is sampled and input into the optimization problem. The additional operating resource consumption loss is calculated as the reward. The reinforcement learning strategy is trained by simultaneously minimizing the additional operating resource consumption loss and the wind power prediction error. S10: Iterate through S7, S8, and S9 until the stopping condition is met, and output the optimal reinforcement learning strategy as a wind power prediction model with robust operational decision-making capabilities.

[0007] Furthermore, in S5, the policy distribution of the continuous action reinforcement learning policy is represented as follows: ,in For state, For wind power prediction models, the mean of the parameterized strategy distribution is used. To explore variance for the strategy, Represents a normal distribution; action a is obtained by sampling according to the strategy distribution, i.e. .

[0008] Furthermore, in S5, a clean branch-based policy interaction is performed. The original numerical weather forecast input is used as the state of the reinforcement learning policy. The clean wind power prediction value is obtained based on the initialized wind power prediction model. The clean prediction action is then sampled using the mean of the normal policy distribution. The clean prediction action is input into the day-ahead operation scheduling optimization problem of the power system to obtain additional operating resource consumption losses and rewards.

[0009] Furthermore, in S6, when the number of samples in the buffer reaches the set minimum threshold for activation, the proxy model is trained using the buffer samples. The buffer is divided into a subset of samples as independent validation samples, and proxy model gating validation is performed. On the independent validation subset, the Pearson correlation coefficient and Spearman rank correlation coefficient between the estimated additional runtime resource consumption loss of the proxy model and the actual additional runtime resource consumption loss are calculated. When both the Pearson correlation coefficient and the Spearman rank correlation coefficient meet the preset threshold, proceed to S7 and enable adversarial attacks oriented towards runtime resource consumption loss. Otherwise, continue to repeat S5 to collect samples and increase the sample size in the buffer.

[0010] Furthermore, in S7, the stationary wind power prediction model and proxy model The parameters are used to execute adversarial attacks oriented towards resource consumption losses; let δ be the adversarial perturbation added to the numerical weather forecast input X, then For the corresponding adversarial examples; define a feasible set of perturbations S, requiring the perturbations to satisfy the infinite norm constraint; For each randomly initialized perturbation, perform K steps of gradient ascent using the projected gradient descent algorithm; let... This represents the additional runtime resource consumption loss due to the current perturbation estimated under the surrogate model. The step size for the projected gradient attack. If the projection onto the feasible set S of the perturbation indicates that the perturbation is updated as follows: Where y represents the actual wind power output, and L represents the power system load. This indicates that the gradient of the input perturbation δ is taken, and sign indicates the sign of the gradient. This represents a candidate adversarial sample.

[0011] Furthermore, in S9, policy interaction based on clean branch and adversarial branch is executed simultaneously. For the clean branch, the original numerical weather forecast input is used as the reinforcement learning policy state. For the adversarial branch, the worst adversarial sample is used as the reinforcement learning policy state. The corresponding wind power prediction values ​​are obtained through the current wind power prediction model, and the prediction values ​​are used as the mean of the normal policy distribution to sample prediction actions. The clean prediction action and the adversarial prediction action are respectively input into the day-ahead operation scheduling optimization problem of the power system to obtain the corresponding additional operating resource consumption loss and reward, wherein the reward is set as the negative normalized additional operating resource consumption loss.

[0012] Furthermore, in S9, during the reinforcement learning policy training phase, both prediction accuracy constraints and additional runtime resource consumption loss constraints are retained; the final joint training objective is: in, and These represent the normalized prediction errors on the original numerical weather forecast input and the worst-case adversarial sample, respectively. and These represent the additional runtime resource consumption loss terms for reinforcement learning on the original numerical weather forecast input and the worst-case adversarial example, respectively. , , and These are non-negative weighting coefficients; gradient updates are performed on the wind power prediction model parameters based on the joint training loss.

[0013] According to a second aspect of this specification, a wind power prediction device based on reinforcement learning and decision-oriented robust enhancement is provided, comprising a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it is used to implement the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement as described in the first aspect.

[0014] According to a third aspect of this specification, a computer-readable storage medium is provided having a program stored thereon that, when executed by a processor, implements the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement as described in the first aspect.

[0015] According to a fourth aspect of this specification, a computer program product is provided, comprising a computer program, characterized in that, when the computer program is executed by a processor, it implements the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement as described in the first aspect.

[0016] The advantages and beneficial effects of this invention are: 1. This invention uses the additional operating resource consumption loss as the basis for model training and adversarial example evaluation, so that the wind power prediction model can take into account both prediction accuracy and robustness of operation and scheduling decisions.

[0017] 2. This invention utilizes a differentiable operational resource consumption proxy model to provide the gradient required for adversarial sample search, and sorts candidate adversarial samples according to the output of the proxy model. Then, it uses a power system day-ahead operation scheduling optimization model to calculate the actual additional operational resource consumption loss and perform verification, thereby balancing the efficiency of adversarial sample search and the accuracy of worst-case adversarial sample selection.

[0018] 3. This invention simultaneously constrains the prediction error and additional operating resource consumption loss of the wind power prediction model under both the original input and the worst-case adversarial input. While maintaining prediction accuracy, it effectively reduces the additional operating resource consumption caused by input disturbances and improves the robustness of the wind power prediction model for operation scheduling decisions.

[0019] 4. The functional modules of this invention are independent of each other and can be adapted to different wind power prediction models, day-ahead operation scheduling models and numerical weather forecast input scenarios, and have good versatility and scalability. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram illustrating the principle behind the additional operational resource consumption loss caused by prediction deviations; Figure 2 The flowchart shows a wind power prediction method based on reinforcement learning and decision-oriented robust enhancement. Figure 3 A schematic diagram illustrating the computational loss due to additional runtime resource consumption and the construction of training data for the proxy model; Figure 4 Generate a screening flowchart for the worst adversarial examples; Figure 5 This is a structural diagram of a wind power prediction device based on reinforcement learning and decision-oriented robust enhancement. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] This invention provides a wind power prediction method based on reinforcement learning and decision-oriented robust enhancement, such as... Figure 1 , Figure 2 As shown, the method includes the following steps: S1: Constructing a wind power prediction dataset The dataset includes numerical weather forecast input X, actual wind power y, and power system load L.

[0025] This embodiment uses day-ahead wind power forecasting as an example. The forecasting model uses numerical weather prediction as the input features, including four original wind speed components (U10, V10, U100, and V100), derived meteorological features calculated from the original wind speed components, and time features. The forecast output is the normalized wind power forecast value one day later. The power system load data uses the IEEE RTS summer weekday 24-hour standard load curve. The dataset contains 11,735 samples, divided into training, validation, and test sets in a 7:2:1 ratio.

[0026] S2: Based on the sample set in the dataset Pre-trained wind power prediction model The prediction model parameters are obtained with the goal of minimizing the mean square error of the predicted output. The predicted wind power value is denoted as .

[0027] Specifically, the wind power prediction model is pre-trained using a sample set (X,y) from the training set and a supervised learning approach. The initial wind power prediction model is obtained. .

[0028] S3: Establish a day-ahead operation and scheduling optimization problem for the power system. The optimization objective is to minimize the consumption of operating resources. Using day-ahead wind power forecasts as input information, solve for the optimal decision variables for synchronous generation output and reserve resource scheduling while satisfying system power balance constraints. .

[0029] S4: As Figure 3 As shown, the predicted wind power value Input an optimization problem and obtain a prediction-based runtime scheduling strategy. and prediction-based runtime resource consumption The optimization problem of the actual wind power input corresponding to the wind power prediction is transformed into an ideal operation and scheduling strategy based on the actual value. and ideal operating resource consumption The difference between the predicted operating resource consumption and the ideal operating resource consumption based on actual values ​​will be considered as the additional operating resource consumption loss caused by the wind power prediction bias. ,Right now .

[0030] The day-ahead operation and dispatch optimization problem of a power system is used to formulate a scheduling plan for generator output and reserve resources in the next operating cycle based on the day-ahead wind power forecast. Given the system load demand, generator operating constraints, and wind power forecast results, the wind power forecast is used as the wind power output input estimate. Under the conditions of satisfying system power balance and various operating constraints, the optimal decision variables for synchronous generation output and reserve resource scheduling are solved. Operating resource consumption includes one or more of the following: synchronous generation unit power generation resource consumption, reserve resource occupation, wind curtailment resource loss, external power purchase resource occupation, and load adjustment resource consumption. Since day-ahead operation and dispatch decisions directly depend on wind power forecast results, deviations in wind power forecasts will cause operation and dispatch decisions to deviate from the ideal optimal decision based on actual wind power, thus leading to increased power system operating resource consumption.

[0031] S5: Construct a continuous action reinforcement learning strategy, using a wind power prediction model as an example. As the strategy mean function, the numerical weather forecast input X is used as the state. Based on the strategy distribution sampling, a predicted action 'a' is obtained, and this predicted action 'a' is used as the wind power prediction value for the day-ahead operation scheduling optimization problem. Based on the predicted action 'a', the actual wind power 'y', and the power system load 'L', the additional operating resource consumption loss 'Regret' caused by the wind power prediction deviation is calculated according to the formula in S4. The predicted action 'a' is then used to calculate the wind power prediction deviation. Actual wind power output y, power system load L, and additional operating resource consumption losses Write to the scheduling and execution resource consumption sample cache area .

[0032] Specifically, the policy distribution of the continuous action reinforcement learning policy is represented as follows: ,in, For wind power prediction models, the mean of the parameterized strategy distribution is used. To explore variance for the strategy, Represents a normal distribution; action a is obtained by sampling according to the strategy distribution, i.e. , which is a continuous prediction action obtained by sampling near the mean of the wind power prediction model output.

[0033] First, a clean branch-based policy interaction is performed. The original numerical weather forecast input X is used as the state of the reinforcement learning policy. Based on the initialized wind power prediction model, the clean wind power prediction value clean_prediction is obtained. Then, the clean prediction action clean_action is sampled from the mean of the normal policy distribution using this prediction value. Inputting clean_action into the day-ahead operation scheduling optimization problem of the power system yields the additional operational resource consumption loss clean_regret and the reward clean_reward: in, As a scale factor for operational resource consumption loss, To prevent division by zero for positive numbers, write (clean_action, y, L, clean_regret) into... It is used for supervised learning of proxy models.

[0034] S6: Train a differentially operable resource consumption proxy model using cached data. The input to the proxy model is the predicted action. The actual wind power y and power system load L are used as the output, which is the estimated additional operating resource consumption loss.

[0035] Specifically, when The number of samples in the sample reaches the set minimum starting value N. min Training using cached samples .

[0036] Specifically, the proxy model The supervised learning objective is: in, Indicates the expectation. This represents the estimated additional runtime resource consumption loss. For the learning rate of the surrogate model, Indicates the parameters of the proxy model Find the gradient.

[0037] Will A subset of samples is divided into independent validation subsets, and surrogate model-gated validation is performed on these subsets. The Pearson correlation coefficient and Spearman rank correlation coefficient between the estimated additional runtime resource consumption loss and the actual additional runtime resource consumption loss are used to determine the target. If both the Pearson correlation coefficient and Spearman rank correlation coefficient meet the preset threshold, the process proceeds to S7 to enable adversarial attacks oriented towards runtime resource consumption loss; otherwise, the process continues to repeat S5 to collect samples and increase the sample size in the buffer.

[0038] S7: As Figure 4 As shown, based on the wind power prediction model and proxy model The gradient of the numerical weather prediction input X with respect to the estimated additional operational resource consumption loss is calculated. An adversarial perturbation is constructed on the numerical weather prediction input X using the projected gradient descent algorithm to maximize the estimated additional operational resource consumption loss, generating multiple candidate adversarial examples. .

[0039] Specifically, the model parameters θ and This involves executing an adversarial attack oriented towards resource consumption loss. Let δ be the adversarial perturbation added to the numerical weather forecast input X, then... Let S be the corresponding adversarial example. Define a feasible set S for the perturbation, requiring the perturbation to satisfy the infinite norm constraint, i.e.: in, To disrupt the budget.

[0040] For each randomly initialized perturbation, the projected gradient descent algorithm is applied to perform K-step gradient ascent. Let... This represents the additional runtime resource consumption loss due to the current perturbation estimated under the surrogate model. The step size for the projected gradient attack. If the projection onto the feasible set S of the perturbation indicates that the perturbation is updated as follows: in, This indicates that the gradient of the input perturbation δ is taken, and sign indicates the sign of the gradient. This represents a candidate adversarial sample.

[0041] S8: As Figure 4 As shown, based on the proxy model, all candidate adversarial samples are quickly sorted, and several adversarial samples with the highest estimated additional runtime resource consumption loss are selected. The selected adversarial samples are then used to apply the optimization problem to calculate the additional runtime resource consumption loss they cause, and the adversarial sample with the largest loss is selected as the worst adversarial sample. .

[0042] Specifically, perform real scheduling verification and worst-case selection. First, utilize... The candidate adversarial samples are sorted sample by sample, and the q samples with the highest estimated additional operating resource consumption loss are selected as the top-q candidate samples. The additional operating resource consumption loss Regrett is calculated by solving the day-ahead operation scheduling optimization problem of the power system, and the candidate adversarial sample with the largest Regrett is selected as the worst adversarial sample. .

[0043] Since the proxy model only provides fast sorting and differentiable directions, this invention further uses the actual additional runtime resource consumption loss obtained from solving the optimization problem to verify the adversarial examples selected by the proxy model. Through this mechanism, the worst adversarial example that finally enters the outer layer training is based on the actual runtime resource consumption loss, rather than solely determined by the proxy model score, thereby reducing the impact of estimation bias caused by the proxy model.

[0044] S9: Input the raw numerical weather forecast into X (as a clean input) and the worst-case adversarial sample respectively. (As adversarial input) The input state of the reinforcement learning policy is sampled and the predicted action is input into the optimization problem. The additional operating resource consumption loss is calculated as the reward. The reinforcement learning policy is trained by simultaneously minimizing the additional operating resource consumption loss and the wind power prediction error.

[0045] Specifically, policy interactions based on clean and adversarial branches are executed simultaneously. For the clean branch, the original numerical weather forecast input X is used as the reinforcement learning policy state; for the adversarial branch, the worst adversarial sample is used. As the state of the reinforcement learning policy, the corresponding wind power prediction values ​​are obtained through the current wind power prediction model, and the prediction values ​​are used as the mean of the normal policy distribution to sample and predict actions. Clean prediction actions respectively and anti-predictive action By inputting the day-ahead operation and scheduling optimization problem of the power system, we obtain the corresponding additional operational resource consumption losses and rewards: This includes setting the normalized additional runtime resource consumption loss to a negative reward. As a scale factor for operational resource consumption loss, To prevent division by zero for positive numbers, (clean_action, y, L, clean_regret) and (worse_action, y, L, worse_regret) are written to the runtime resource consumption sample cache for updating reinforcement learning rewards and training runtime resource consumption proxy models.

[0046] During the reinforcement learning strategy training phase, this invention simultaneously retains both prediction accuracy constraints and additional runtime resource consumption loss constraints; the final joint training objective is: in, and The original numerical weather forecast input X and the worst-case adversarial sample are respectively. Normalized prediction error on and The original numerical weather forecast input X and the worst-case adversarial sample are respectively. The additional runtime resource consumption loss term for reinforcement learning is mentioned. , , and As non-negative weighting coefficients, through the above objectives, this invention simultaneously constrains prediction error and additional runtime resource consumption loss on clean input and adversarial input.

[0047] For the additional runtime resource consumption loss term of reinforcement learning, the moving average reward baseline of the clean branch and the adversarial branch is updated based on the current batch reward, where E(⋅) represents the batch mean: baseline_clean = E(clean_reward) baseline_worse = E(worse_reward) Then, the policy gradient-based reinforcement learning loss (REINFORCE loss) is constructed for the clean branch and the adversarial branch respectively: StopGrad(⋅) represents the gradient blocking operator, which is used to block the reward advantage term from participating in gradient backpropagation and update the wind power prediction model parameters only through the policy log probability term.

[0048] Furthermore, the normalized prediction errors for the clean branch and the adversarial branch are calculated separately: Where mse_scale is a fixed prediction error scaling factor.

[0049] Finally, construct the joint training loss. Based on the joint training loss, the parameters of the wind power prediction model are... Perform gradient updates: in, To predict the learning rate of the model.

[0050] S10: Iterate through S7, S8, and S9 until the stopping condition is met, and output the optimal reinforcement learning strategy as a wind power prediction model with robust operational decision-making capabilities.

[0051] Specifically, the stopping condition is reaching the maximum number of iterations, or when the model performance does not improve for N_patience consecutive iterations. In this embodiment, N_patience is set to 10.

[0052] Model selection is performed. After training the wind power prediction model using the training set, models obtained at different iteration stages during training are used as candidate models. The validation set is then used to evaluate the mean squared error and average additional operating resource consumption loss of each candidate model under a preset adversarial attack. During model selection, candidate models with a mean squared error below a preset threshold are first selected. Then, from the selected candidate models, the one with the lowest average additional operating resource consumption loss is chosen as the target model. The validation set is only used for model selection and does not participate in model parameter updates.

[0053] Robustness evaluation is performed. After the target model is determined, a final evaluation is conducted using a test set under the same adversarial attack conditions. The test set is not involved in model training or model selection. Evaluation metrics include prediction accuracy and operational decision impact metrics. The prediction accuracy metric is Mean Squared Error (MSE), and the operational decision impact metrics include average additional runtime resource consumption loss and maximum additional runtime resource consumption loss.

[0054] In this embodiment, the following three models are compared and tested: a baseline model that uses only the original training samples without adversarial perturbations for supervised training, denoted as the Clean training model; a prediction error-oriented adversarial training model that uses adversarial samples that maximize the mean square error of prediction and focuses solely on prediction accuracy as the main adversarial training objective, denoted as the MSE adversarial training model (MSE AT); and a model obtained using the runtime resource consumption loss-oriented adversarial sample search, real scheduling verification, and joint training mechanism described in this invention, denoted as the runtime resource consumption loss-oriented adversarial training model (Worse-Regret AT).

[0055] Table 1 Performance of different models under adversarial attacks As shown in Table 1, under the adversarial attack scenario of the test set, the MSE, average additional runtime resource consumption loss, and maximum additional runtime resource consumption loss of the clean-trained model are 0.134808, 949.793, and 4759.210, respectively. This indicates that the model without adversarial training has a large prediction error and the resulting additional runtime resource consumption loss when the input is perturbed.

[0056] The MSE of the adversarial training model is 0.076494, which is lower than that of the clean training model, indicating that adversarial training aimed at prediction error can improve the model's robustness to prediction accuracy under input perturbations; its average additional runtime resource consumption loss and maximum additional runtime resource consumption loss are 586.899 and 3797.790, respectively.

[0057] The model of this invention has an MSE of 0.080574, with average and maximum additional operating resource consumption losses of 560.107 and 3147.450, respectively. Compared with the adversarial training model, the model of this invention further reduces the average and maximum additional operating resource consumption losses while maintaining similar prediction accuracy and robustness. These results demonstrate that this invention can reduce the additional operating resource consumption caused by adversarial disturbances and improve the robustness of the wind power prediction model for day-ahead operation and scheduling of the power system.

[0058] Corresponding to the aforementioned embodiments of the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement, the present invention also provides embodiments of a wind power prediction device based on reinforcement learning and decision-oriented robust enhancement.

[0059] See Figure 5 The wind power prediction device based on reinforcement learning and decision-oriented robust enhancement provided in this embodiment of the invention includes a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement in the above embodiment.

[0060] The embodiment of the wind power prediction device based on reinforcement learning and decision-oriented robust enhancement of this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 5The diagram shown is a hardware structure diagram of any device with data processing capabilities, including the wind power prediction device based on reinforcement learning and decision-oriented robust enhancement according to the present invention. (Except for...) Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0061] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0062] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0063] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement described in the above embodiments.

[0064] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0065] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.

Claims

1. A wind power prediction method based on reinforcement learning and decision-oriented robust enhancement, characterized in that, Includes the following steps: S1: Construct a dataset that includes numerical weather forecast inputs, actual wind power, and power system load; S2: A wind power prediction model pre-trained based on numerical weather forecast inputs and real wind power in the dataset; S3: Establish a day-ahead operation and scheduling optimization problem for the power system. The optimization objective is to minimize the consumption of operating resources. Using day-ahead wind power forecast as input information, solve for the optimal decision variables of synchronous generation output and reserve resource scheduling under the condition of satisfying the system power balance constraints. S4: Input the wind power forecast into the optimization problem to obtain the prediction-based operation scheduling strategy and the prediction-based operation resource consumption; The problem of optimizing the actual wind power input corresponding to the wind power prediction is solved to obtain the ideal operation scheduling strategy and ideal operation resource consumption based on the actual value; the difference between the operation resource consumption based on the prediction and the ideal operation resource consumption based on the actual value is taken as the additional operation resource consumption loss caused by the wind power prediction deviation. S5: Construct a continuous action reinforcement learning strategy, using the wind power prediction model as the strategy mean function and the numerical weather forecast input as the state. Sample the predicted actions according to the strategy distribution and use the predicted actions as the wind power prediction values ​​for the optimization problem. Calculate the additional operating resource consumption loss based on the predicted actions, the actual wind power, and the power system load. Write the predicted actions, the actual wind power, the power system load, and the additional operating resource consumption loss into the cache. S6: Train a differentiable operational resource consumption proxy model using buffer data. The input of the proxy model is the predicted action, the actual wind power, and the power system load. The output is the estimated additional operational resource consumption loss. S7: Based on the wind power prediction model and the surrogate model, calculate the gradient of the numerical weather forecast input with respect to the estimated additional operating resource consumption loss, use the projected gradient descent algorithm to construct adversarial perturbations on the numerical weather forecast input, maximize the estimated additional operating resource consumption loss, and generate multiple candidate adversarial samples. S8: Based on the proxy model, sort all candidate adversarial samples, select several adversarial samples with the highest estimated additional runtime resource consumption loss, call the optimization problem to calculate the additional runtime resource consumption loss caused by the selected adversarial samples, and select the adversarial sample with the largest loss as the worst adversarial sample. S9: The original numerical weather forecast input and the worst adversarial sample are respectively used as the input state of the reinforcement learning strategy. The prediction action is sampled and input into the optimization problem. The additional operating resource consumption loss is calculated as the reward. The reinforcement learning strategy is trained by simultaneously minimizing the additional operating resource consumption loss and the wind power prediction error. S10: Iterate through S7, S8, and S9 until the stopping condition is met, and output the optimal reinforcement learning strategy as a wind power prediction model with robust operational decision-making capabilities.

2. The wind power prediction method based on reinforcement learning and decision-oriented robust enhancement according to claim 1, characterized in that, In S5, the policy distribution of the continuous action reinforcement learning policy is represented as follows: ,in For state, For wind power prediction models, the mean of the parameterized strategy distribution is used. To explore variance for the strategy, Represents a normal distribution; action a is obtained by sampling according to the strategy distribution, i.e. .

3. The wind power prediction method based on reinforcement learning and decision-oriented robust enhancement according to claim 1, characterized in that, In S5, a clean branch-based policy interaction is performed. The original numerical weather forecast is input as the state of the reinforcement learning policy. The clean wind power prediction value is obtained based on the initialized wind power prediction model. The clean prediction action is then sampled using the mean of the normal policy distribution. The clean prediction action is input into the day-ahead operation scheduling optimization problem of the power system to obtain the additional operating resource consumption loss and reward.

4. The wind power prediction method based on reinforcement learning and decision-oriented robust enhancement according to claim 1, characterized in that, In S6, when the number of samples in the buffer reaches the set minimum threshold for activation, the proxy model is trained using the buffer samples. The buffer is divided into a subset of samples as independent validation samples, and gating validation of the proxy model is performed. The Pearson correlation coefficient and Spearman rank correlation coefficient between the estimated additional runtime resource consumption loss of the proxy model and the actual additional runtime resource consumption loss are calculated on the independent validation subset. When both the Pearson correlation coefficient and the Spearman rank correlation coefficient meet the preset threshold, the process proceeds to S7, enabling adversarial attacks oriented towards runtime resource consumption loss. Otherwise, S5 is repeated to collect samples and increase the sample size in the buffer.

5. The wind power prediction method based on reinforcement learning and decision-oriented robust enhancement according to claim 1, characterized in that, In S7, the fixed wind power prediction model and proxy model The parameters are used to execute adversarial attacks that prioritize resource consumption and loss. Let δ be the adversarial perturbation added to the numerical weather prediction input X, then For the corresponding adversarial examples; define a feasible set of perturbations S, requiring the perturbations to satisfy the infinite norm constraint; For each randomly initialized perturbation, perform K steps of gradient ascent using the projected gradient descent algorithm; let... This represents the additional runtime resource consumption loss due to the current perturbation estimated under the surrogate model. The step size for projected gradient attack. If the projection onto the feasible set S of the perturbation indicates that the perturbation is updated as follows: Where y represents the actual wind power output, and L represents the power system load. This indicates that the gradient of the input perturbation δ is taken, and sign indicates the sign of the gradient. This represents a candidate adversarial sample.

6. The wind power prediction method based on reinforcement learning and decision-oriented robust enhancement according to claim 1, characterized in that, In S9, policy interaction based on clean branch and adversarial branch is executed simultaneously; for clean branch, the original numerical weather forecast input is used as the reinforcement learning policy state; for adversarial branch, the worst adversarial sample is used as the reinforcement learning policy state. The corresponding wind power prediction values ​​are obtained through the current wind power prediction model, and the prediction values ​​are used as the mean of the normal strategy distribution to sample the prediction action. The clean prediction action and the counter-prediction action are respectively input into the day-ahead operation scheduling optimization problem of the power system to obtain the corresponding additional operating resource consumption loss and reward. Among them, the normalized additional operating resource consumption loss with negative reward is set.

7. The wind power prediction method based on reinforcement learning and decision-oriented robust enhancement according to claim 1, characterized in that, In S9, during the reinforcement learning policy training phase, both prediction accuracy constraints and additional runtime resource consumption loss constraints are retained; the final joint training objective is: in, and These represent the normalized prediction errors on the original numerical weather forecast input and the worst-case adversarial sample, respectively. and These represent the additional runtime resource consumption loss terms for reinforcement learning on the original numerical weather forecast input and the worst-case adversarial example, respectively. , , and These are non-negative weighting coefficients; gradient updates are performed on the wind power prediction model parameters based on the joint training loss.

8. A wind power prediction device based on reinforcement learning and decision-oriented robust enhancement, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it is used to implement the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement as described in any one of claims 1-7.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the wind power prediction method based on reinforcement learning and decision-oriented robust enhancement as described in any one of claims 1-7.

Images