Extreme weather wind power prediction method based on reinforcement learning adaptive sampling

Abstract

The application provides an extreme weather wind power prediction method based on reinforcement learning adaptive sampling, and relates to the field of wind power prediction. The method comprises the following steps: obtaining meteorological time series data and wind power data of a wind power station site, constructing a training set, a validation set and a test set, and respectively extracting a training subset, a validation subset and a test subset corresponding to extreme weather; designing a training framework based on reinforcement learning to obtain a Markov decision process component, build a parameterized sampling strategy network and a wind power prediction model; iteratively performing a cooperative optimization process of the sampling strategy network and the prediction model until the training converges, and outputting an optimized target prediction model and a target sampling strategy network. Through the reinforcement learning adaptive sampling method, the sampling strategy network and the prediction model form an optimized closed loop, effectively improving the accuracy of wind power prediction under extreme weather, and ensuring the prediction effect under normal weather.

Description

Technical Field

[0001] This application relates to the field of wind power prediction, specifically to a method for predicting wind power in extreme weather based on reinforcement learning adaptive sampling. Background Technology

[0002] With the continuous expansion of wind power capacity, large-scale wind energy is having an increasingly significant impact on the safe and stable operation of the power system. To access abundant wind resources, wind farms are typically located in areas with complex climates, such as plateaus and coastlines, frequently encountering extreme weather events such as cold waves, typhoons, and icing. A typical characteristic of such extreme weather is the drastic fluctuation of meteorological variables within a short period, leading to rapid changes in wind power output. In severe cases, this can cause turbine shutdowns or grid disconnection, posing a serious threat to the stability of the power system. Therefore, developing accurate wind power forecasting technology under extreme weather conditions is crucial for ensuring the stable operation of the power system.

[0003] Currently, deep learning-based wind power prediction technology has become a research focus in the industry, with most methods employing LSTM-based or Transformer-based networks for prediction. Addressing the problem of insufficient prediction accuracy due to the extremely low proportion of extreme weather samples and imbalanced training data, recent research has mainly fallen into two categories: one is based on generative models to expand extreme weather samples to improve prediction performance, and the other is prediction methods based on training with few samples. Existing patents propose using generative adversarial networks to generate similar data combined with CNN-LSTM networks to achieve power prediction in scenarios with insufficient data; utilizing Granger causality analysis algorithms to expand small samples in extreme weather; and solving the problem of insufficient samples in extreme scenarios through meta-training combined with fine-tuning models. Other research proposes reinforcement learning methods based on physical information to establish analytical physical expressions for wind power output under extreme events, and combines reinforcement learning replay technology to improve prediction performance under small batches of data.

[0004] However, existing technologies still have many shortcomings: First, data augmentation-based methods rely on generative models to synthesize extreme weather samples, and the quality of synthesized samples is difficult to guarantee, easily introducing noise into the dataset and causing a decline in the performance of the prediction model; Second, model adaptation techniques based on few-shot learning, while improving the prediction performance of extreme weather, often significantly reduce the prediction accuracy under normal weather conditions, making it impossible to achieve balanced predictions under the two scenarios, and the model adaptation process requires readjusting a large number of parameters, which is complex and has limited generalization ability; Third, although traditional static sampling methods can attempt to balance the sample distribution, oversampling can easily lead to model overfitting, undersampling may lose valuable information, and it is impossible to dynamically adjust the sampling weights based on the prediction performance feedback during the training process, making it difficult to accurately select high-value samples. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a method for predicting wind power in extreme weather based on reinforcement learning adaptive sampling. This method solves the problems of poor quality of synthesized samples and inability to balance the prediction accuracy of normal and extreme weather when dealing with imbalanced extreme weather samples.

[0006] To achieve the above objectives, this application provides the following technical solution: In a first aspect, embodiments of this application provide a method for predicting wind power in extreme weather based on reinforcement learning-based adaptive sampling. This method includes: acquiring meteorological time-series data and wind power data from wind farm sites; constructing training, validation, and test sets; and extracting training, validation, and test subsets corresponding to extreme weather conditions, respectively; the training, validation, and test sets all contain meteorological time-series data and wind power data; designing a reinforcement learning-based training framework to obtain Markov decision process components; and constructing a parameterized sampling strategy network. The wind power prediction model uses a sampling strategy network built on LSTM, and the prediction model is either an LSTM model or a Transformer model. Based on the Markov decision process component, the sampling strategy network and the prediction model are iteratively optimized until training converges, and the optimized target prediction model and target sampling strategy network are output. The target sampling strategy network is used to sample the acquired data samples in the wind power prediction stage, and the target prediction model is used to predict wind power under extreme and normal weather conditions.

[0007] According to a first aspect of the embodiments of this application, each iteration of the aforementioned collaborative optimization process of the sampling policy network and the prediction model includes: sampling from the union of the training set and the training subset to obtain a training batch; and inputting the training batch into the sampling policy network. The system outputs a mask batch, which is a binary vector matching the dimension of the training batch. A 1 in the binary vector indicates that the corresponding sample is included in the training, and a 0 in the binary vector indicates that the corresponding sample is discarded. The parameters of the prediction model are updated based on the mask batch. The updated prediction model is tested based on the test set and test subset to determine the optimized target prediction model. The sampling policy network is updated using the REINFORCE policy gradient method to obtain the target sampling policy network.

[0008] According to a first aspect of the embodiments of this application, the process of determining the components of a Markov decision process includes: state design and action design; the state design is used to provide meteorological context information to the reinforcement learning agent to support informed sampling decisions; the action design is used to perform binary decisions on the reinforcement learning sampling strategy, and the action... Indicates whether to include the current sequence sample in the training batch. Indicates inclusion, It indicates that it has been discarded.

[0009] According to a first aspect of the embodiments of this application, the state design includes: processing meteorological time-series data using a Long Short-Term Memory (LSTM) network through an agent to generate a meteorological data state representation to capture long-term dependencies in the meteorological data; and concatenating the meteorological data state representation with a uniquely thermally encoded label of the weather type to form a complete state. , ;in, This represents the state of meteorological data output by LSTM. A value of 0 indicates normal weather. A value of 1 indicates extreme weather.

[0010] According to a first aspect of the embodiments of this application, the action design includes: using the REINFORCE policy gradient method, outputting classification distribution parameters in the action space through a sampling policy and determining the corresponding classification distribution, and sampling actions from the classification distribution to achieve decoupling between differentiable policies and non-differentiable actions.

[0011] According to a first aspect of the embodiments of this application, the process of determining the components of a Markov decision process further includes: designing a fine-grained reward function; the fine-grained reward function includes an accuracy reward, a balance reward, and a shutdown event alignment reward; the fine-grained reward function satisfies the expression: In the formula, For accuracy rewards, To balance rewards, Align rewards for downtime events.

[0012] According to a first aspect of the embodiments of this application, the accuracy reward satisfies the expression: Balanced rewards satisfy the expression: In the formula, and These represent extreme weather losses and normal weather losses on the validation set, respectively. It is a natural exponential function.

[0013] According to a first aspect of the embodiments of this application, when designing a downtime event alignment reward, the following is defined: and , This is the set of actual shutdown start times. To predict the set of shutdown start times, Let t be the actual shutdown indicator variable at time t. The actual shutdown indicator variable at time t-1. Let be the predicted shutdown indicator variable at time t. Let be the predicted shutdown indicator variable at time t-1. The logical symbol AND represents AND; and , and These represent the actual wind power and the predicted wind power at time t, respectively. The threshold for determining shutdown; This is an indicator function used to determine whether the condition within the parentheses is true or false. If true, it outputs 1; otherwise, it outputs 0.

[0014] According to a first aspect of the embodiments of this application, the shutdown event alignment reward satisfies the expression: ; In the formula, A single element representing the actual start time of the shutdown. For a single element to predict the start time of shutdown, Given a time tolerance and .

[0015] According to a first aspect of the embodiments of this application, the aforementioned use of the REINFORCE policy gradient method to update the sampling policy network to obtain the target sampling policy network includes: determining the difference between the validation set and the validation subset corresponding to extreme weather as the normal weather validation set; calculating the extreme weather loss of the prediction model on the validation subset and the normal weather loss of the prediction model on the normal weather validation set, and calculating the reward by combining the fine-grained reward function. The policy parameters are updated by calculating the gradient of the expected cumulative reward relative to the policy parameters; a baseline is introduced. To reduce the variance of gradient estimation; where the baseline Let n be the moving average of the rewards over the most recent n training steps, where n is a positive integer; construct the loss function for policy optimization based on the reward. And loss function update sampling policy network The sampling policy network is obtained by minimizing the loss function through gradient descent and updating the parameters θ of the sampling policy network to achieve iterative optimization of the sampling policy.

[0016] According to a first aspect of the embodiments of this application, the loss function satisfies the expression: ; In the formula, The loss is the cost of strategy optimization. For batch size, The reward for the training step at time t. For the policy network in state Select action The probability, Let be the gradient operator with respect to the parameter θ.

[0017] Secondly, embodiments of this application provide an extreme weather wind power prediction system based on reinforcement learning adaptive sampling, which includes a data processing module, a design and construction module, and an iterative execution module.

[0018] Specifically, the data processing module is used to acquire meteorological time-series data and wind power data of wind farm sites, construct training sets, validation sets, and test sets, and extract training subsets, validation subsets, and test subsets corresponding to extreme weather events, respectively; the training set, validation set, and test set all contain meteorological time-series data and wind power data; the design and construction module is used to design a reinforcement learning-based training framework, obtain Markov decision process components, and construct a parameterized sampling policy network. The wind power prediction model uses a sampling strategy network built on LSTM, and the prediction model is either an LSTM model or a Transformer model. The iterative execution module is used to iteratively execute the co-optimization process of the sampling strategy network and the prediction model based on the Markov decision process component until the training converges, and outputs the optimized target prediction model and target sampling strategy network. The target sampling strategy network is used to sample the acquired data samples in the wind power prediction stage, and the target prediction model is used to predict wind power under extreme weather and normal weather conditions.

[0019] Thirdly, embodiments of this application provide an electronic device, which includes: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the extreme weather wind power prediction method based on reinforcement learning adaptive sampling described in the first aspect.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program or instructions that, when executed by a processor, implement the extreme weather wind power prediction method based on reinforcement learning adaptive sampling described in the first aspect above.

[0021] This application provides a method for predicting wind power in extreme weather based on reinforcement learning-based adaptive sampling. Compared with existing technologies, it has the following advantages: This application acquires meteorological time-series data and wind power data from wind farm sites, divides them into training, validation, and test sets, and extracts subsets corresponding to extreme weather conditions, providing accurate and hierarchical dataset support for subsequent prediction modeling. Based on a reinforcement learning-designed training framework, this application constructs a Markov decision process component, paired with an LSTM-based sampling policy network and an LSTM or Transformer-based prediction model, achieving targeted construction of the sampling policy and prediction model. Simultaneously, based on the Markov decision process component, iteratively executes the collaborative optimization of the sampling policy network and prediction model until convergence. The output target sampling policy network can efficiently sample data during the prediction phase, and the target prediction model can adapt to wind power prediction under both extreme and normal weather conditions. Overall, this application uses a reinforcement learning-based adaptive sampling method to form an optimization closed loop between the sampling policy network and the prediction model, effectively improving the accuracy of wind power prediction under extreme weather conditions and ensuring prediction performance under normal weather conditions, providing a reliable method for accurate prediction of wind power across all scenarios. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating an extreme weather wind power prediction method based on reinforcement learning adaptive sampling provided in an embodiment of this application. Figure 2 This is an exemplary architecture diagram of an extreme weather wind power prediction method based on reinforcement learning adaptive sampling provided in an embodiment of this application; Figure 3 This is an exemplary schematic diagram of a collaborative optimization algorithm for a sampling strategy network and a prediction model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an extreme weather wind power prediction system based on reinforcement learning adaptive sampling provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0026] This application provides an extreme weather wind power prediction method based on reinforcement learning adaptive sampling, which solves the problems of poor quality of synthesized samples and inability to balance the prediction accuracy of normal and extreme weather when dealing with imbalanced extreme weather samples in existing deep learning wind power prediction technologies.

[0027] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0028] The following section first introduces a method for predicting wind power in extreme weather based on reinforcement learning adaptive sampling, as provided in the embodiments of this application.

[0029] This application provides a flowchart illustrating an extreme weather wind power prediction method based on reinforcement learning adaptive sampling, which can be referred to in the embodiments below. Figure 1 and Figure 2 The extreme weather wind power prediction method based on reinforcement learning adaptive sampling may include the following steps S110-S130.

[0030] S110. Obtain meteorological time-series data and wind power data of wind farm sites, construct training sets, validation sets and test sets, and extract training subsets, validation subsets and test subsets corresponding to extreme weather conditions respectively; the training set, validation set and test set all contain meteorological time-series data and wind power data.

[0031] Understandably, extreme weather includes cold waves and typhoons. This application constructs a complete data system including training, validation, and test sets by acquiring meteorological time-series data and wind power data from wind farm sites. It also specifically extracts data subsets corresponding to extreme weather, providing basic data support that fits the actual wind farm scenario for the training, validation, and testing of subsequent sampling strategy networks and prediction models. Furthermore, it achieves a hierarchical division of normal and extreme weather data, which can adapt to the specific needs of wind power prediction under extreme weather conditions. This makes subsequent model training and optimization more targeted and effectively avoids the problem of insufficient learning of extreme weather features due to mixed data compilation.

[0032] S120. Design a reinforcement learning-based training framework to obtain Markov decision process components and build a parameterized sampling policy network. The prediction model for wind power is constructed using a sampling strategy network based on LSTM, and the prediction model is either an LSTM model or a Transformer model.

[0033] Understandably, this application designs a reinforcement learning-based training framework to obtain Markov decision process components, providing theoretical and rule support for the subsequent collaborative optimization of the sampling policy network and the prediction model. At the same time, it builds a sampling policy network based on LSTM, which can efficiently capture the temporal features of meteorological time series data and adapt to the time series data input requirements of sampling decision-making. By selecting LSTM or Transformer models as wind power prediction models, it can flexibly adapt to the different wind power prediction scenarios, taking into account the model's ability to capture temporal features and model long-distance correlations, providing a suitable model architecture foundation for subsequent collaborative optimization.

[0034] S130. Based on the Markov decision process component, iteratively execute the collaborative optimization process of the sampling strategy network and the prediction model until training converges, and output the optimized target prediction model and target sampling strategy network; the target sampling strategy network is used to sample the acquired data samples in the wind power prediction stage, and the target prediction model is used to predict wind power under extreme weather and normal weather conditions.

[0035] Understandably, please refer to Figure 3This application utilizes a Markov decision process component to iteratively optimize the sampling strategy network and the prediction model in a collaborative manner. This allows the optimization of the sampling strategy network and the performance feedback of the prediction model to form a closed loop. The sampling strategy network can be dynamically adjusted according to the training effect of the prediction model, achieving mutual adaptation and synergistic improvement between the two. The optimization method, which continues until training convergence, ensures the optimal performance of both the target sampling strategy network and the target prediction model. The output target sampling strategy network can accurately sample data during the wind power prediction stage, while the target prediction model can simultaneously achieve wind power prediction under both extreme and normal weather conditions. This approach caters to the wind power prediction needs of all scenarios, improves the prediction accuracy of the prediction model in extreme weather scenarios, and ensures the prediction effect in normal weather scenarios.

[0036] It should be noted that traditional static sampling methods have fixed sampling rules and cannot be optimized based on training feedback. The sampling strategy in this application receives performance feedback from the prediction model in real time through reinforcement learning, dynamically adjusts the sample selection weights, automatically reduces the sampling ratio of normal samples and focuses on optimizing extreme samples in the later stages of training, and its adaptability and flexibility far exceed those of static sampling methods.

[0037] In some embodiments, each iteration of the aforementioned collaborative optimization process of the sampling policy network and the prediction model includes: S210. Sample the training batch from the union of the training set and the training subset.

[0038] Understandably, in each iteration, this application samples the training batch from the union of the training set and the training subset corresponding to extreme weather to obtain the training batch. This can simultaneously include normal weather samples and extreme weather samples, providing a basic sample pool that takes into account all scenarios for subsequent sampling and screening, and ensuring the participation of extreme weather samples in training.

[0039] S220. Input the training batch into the sampling policy network. Output mask batch, which is a binary vector that matches the dimension of the training batch. 1 in the binary vector indicates that the corresponding sample is included in the training, and 0 in the binary vector indicates that the corresponding sample is discarded.

[0040] Understandably, this application inputs the training batch into the sampling strategy network and outputs a binary mask batch that matches the dimension. The sample selection is achieved by marking 0 / 1, allowing the training of the prediction model to select high-value samples based on the mask batch, thereby improving the effectiveness of the training.

[0041] S230. Update the parameters of the prediction model based on the mask batch, and test the updated prediction model based on the test set and test subset to determine the optimized target prediction model.

[0042] Understandably, this application updates the parameters of the prediction model based on mask batches and tests the updated model in combination with the test set and the test subset corresponding to extreme weather. This allows the prediction model to complete parameter optimization on the selected samples. At the same time, it relies on hierarchical testing to verify the prediction performance of the model in all scenarios and under extreme weather conditions, ensuring that the obtained target prediction model is suitable for actual prediction needs.

[0043] S240. Update the sampling policy network using the REINFORCE policy gradient method to obtain the target sampling policy network.

[0044] Understandably, this application employs the REINFORCE policy gradient method to update the sampling policy network, enabling efficient optimization of the network's parameters and continuously improving its decision-making capabilities. The resulting target sampling policy network can then more accurately make sample sampling decisions. Overall, this application achieves coordinated optimization of the sampling policy network and the prediction model in each iteration through the ordered execution of steps S210-S240. This allows sampling selection and model training to form a positive feedback loop, gradually improving the accuracy of the sampling policy and the performance of the prediction model, thus ensuring the effectiveness of collaborative optimization.

[0045] In some embodiments, the determination process of the Markov decision process components includes: state design and action design; the state design is used to provide meteorological context information to the reinforcement learning agent to support informed sampling decisions; the action design is used to make binary decisions on the reinforcement learning sampling strategy, and the actions... Indicates whether to include the current sequence sample in the training batch. Indicates inclusion, It indicates that it has been discarded.

[0046] The state design includes: processing meteorological time-series data using a Long Short-Term Memory (LSTM) network through an agent to generate a meteorological data state representation to capture long-term dependencies in the meteorological data; and concatenating the meteorological data state representation with uniquely encoded labels for weather types to form a complete state. , ;in, This represents the state of meteorological data output by LSTM. A value of 0 indicates normal weather. A value of 1 indicates extreme weather.

[0047] Action design includes: using the REINFORCE policy gradient method, outputting classification distribution parameters in the action space through sampling policy and determining the corresponding classification distribution, and sampling actions from the classification distribution to achieve decoupling between differentiable policies and non-differentiable actions.

[0048] In the embodiments of this application, it is understood that this application formalizes the sample selection process for extreme weather wind power prediction as a Markov decision process in reinforcement learning. Through the real-time interaction between the reinforcement learning agent and the prediction model, dynamic adaptive sampling is achieved, breaking through the limitations of traditional static sampling.

[0049] This Markov decision process component provides scientific and adaptable decision rule support for adaptive sampling in reinforcement learning through targeted state and action design. The state design first uses an LSTM network to process meteorological time-series data to generate state representations, capturing long-term dependencies in meteorological data. Then, it concatenates weather type uniquely encoded labels to form complete states. This provides comprehensive and accurate meteorological context information for reinforcement learning agents, ensuring that the agents' sampling decisions are based on sufficient meteorological data and guaranteeing the wisdom of the sampling decisions.

[0050] Furthermore, the action design employs the REINFORCE policy gradient method. It samples the policy output classification distribution parameters and constructs a classification distribution, then samples binary actions from this distribution. This successfully decouples differentiable policies from non-differentiable actions, effectively solving the problem that sampled binary decision actions cannot be directly differentiated to update the policy network. This allows the sampled policy network to perform parameter optimization based on gradient methods, ensuring the feasibility and effectiveness of the sampling policy binary decision. Based on this, this application, through reasonable state and action design, enables the Markov decision process component to accurately adapt to the sampling requirements of extreme weather wind power forecasting.

[0051] In some embodiments, the process of determining the components of a Markov decision process further includes: designing a fine-grained reward function; the fine-grained reward function includes an accuracy reward, a balance reward, and a shutdown event alignment reward.

[0052] The fine-grained reward function satisfies the expression: In the formula, For accuracy rewards, To balance rewards, Align rewards for downtime events; Accuracy reward satisfies the expression: ; Balanced rewards satisfy the expression: ; In the formula, and These represent extreme weather losses and normal weather losses on the validation set, respectively. It is a natural exponential function.

[0053] In the embodiments of this application, it is understood that by designing a fine-grained reward function that includes accuracy rewards, balance rewards, and shutdown event alignment rewards, this application can comprehensively guide and constrain the sampling strategy from multiple dimensions, including prediction accuracy, scene performance balance, and extreme weather shutdown event matching. The accuracy reward ensures the overall prediction accuracy of the prediction model, while the balance reward coordinates the model's prediction performance under extreme and normal weather conditions, avoiding performance bias in a single scene. The natural exponential function further stabilizes the reward output, making the reinforcement learning optimization process more stable and reliable. The overall fine-grained reward function provides precise and detailed optimization guidance for the sampling strategy network, effectively improving the rationality and relevance of adaptive sampling, thereby enhancing the comprehensive performance of the prediction model under extreme weather and all scenarios.

[0054] It should be noted that existing technologies (such as transfer learning and generative model data augmentation) often lead to a decrease in prediction accuracy or the introduction of noise in normal weather conditions when improving the performance of extreme weather predictions. This application improves the prediction performance of extreme scenarios and maintains high prediction accuracy in normal weather scenarios by using dynamic adaptive sampling and balanced rewards in a fine-grained reward function, thus achieving balanced optimization in both scenarios.

[0055] In some embodiments, when designing downtime event alignment rewards, the following is defined: and , This is the set of actual shutdown start times. To predict the set of shutdown start times, Let t be the actual shutdown indicator variable at time t. The actual shutdown indicator variable at time t-1. Let be the predicted shutdown indicator variable at time t. Let be the predicted shutdown indicator variable at time t-1. The logical symbol AND represents AND; and , and These represent the actual wind power and the predicted wind power at time t, respectively. The threshold for determining shutdown; This is an indicator function used to determine whether the condition within the parentheses is true or false. If true, it outputs 1; otherwise, it outputs 0. The shutdown event alignment reward satisfies the expression: ; In the formula, A single element representing the actual start time of the shutdown. For a single element to predict the start time of shutdown, Given a time tolerance and .

[0056] In the embodiments of this application, it is understood that in the design of the outage event alignment reward, this application defines actual outage indicator variables and predicted outage indicator variables, the set of actual outage start times and the set of predicted outage start times, and combines actual wind power, predicted wind power and outage judgment thresholds to accurately determine the outage status and outage start time at a single moment with the help of indicator functions, thus clarifying the judgment criteria for outage events. At the same time, by setting individual elements and time tolerances of the actual outage start time and the predicted outage start time, and combining expressions to quantify the matching degree between the predicted outage start time and the actual outage start time, the ability of the prediction model to capture the wind turbine outage start time caused by extreme weather is measured. The sampling strategy network is specifically guided to prioritize the selection of samples that can improve the prediction accuracy of the outage start time, thereby optimizing the prediction performance of the prediction model in extreme weather outage scenarios, meeting the accurate prediction requirements of key time nodes of wind turbine outage under extreme weather, and providing support for the power system to formulate reasonable scheduling strategies and safety control measures.

[0057] In some embodiments, the aforementioned use of the REINFORCE policy gradient method to update the sampling policy network to obtain the target sampling policy network includes: S310. Determine the difference between the validation set and the validation subset corresponding to extreme weather as the normal weather validation set.

[0058] Understandably, this application defines the validation set for normal weather by taking the difference between the validation set and the validation subset corresponding to extreme weather, thus dividing the validation data into normal weather and extreme weather data, providing a data foundation for the subsequent accurate calculation of model loss for the two types of scenarios.

[0059] S320. Calculate the extreme weather loss of the prediction model on the validation subset and the normal weather loss of the prediction model on the normal weather validation set, and calculate the reward using a fine-grained reward function. .

[0060] Understandably, this application calculates the loss of the prediction model on the validation subset corresponding to extreme weather and the validation set for normal weather, and obtains the reward by combining it with a fine-grained reward function. This can comprehensively quantify the merits of sampling decisions from multiple dimensions and provide accurate and reliable feedback signals for updating the sampling strategy network.

[0061] S330. Update the policy parameters by calculating the gradient of the expected cumulative reward relative to the policy parameters; introduce a baseline. To reduce the variance of gradient estimation; where the baseline This is the moving average of the rewards over the most recent n training steps, where n is a positive integer.

[0062] Understandably, this application updates the policy parameters by calculating the gradient of the expected cumulative reward relative to the policy parameters, and introduces the moving average of the rewards of the most recent n training steps as a baseline, which can effectively reduce the variance of gradient estimation and improve the stability and accuracy of policy parameter updates.

[0063] S340. Construct a loss function for policy optimization, based on reward. And loss function update sampling policy network .

[0064] Understandably, this application constructs a loss function for policy optimization and updates the sampling policy network based on the reward and loss function, providing an optimization objective and implementation path for the iterative optimization of the sampling policy.

[0065] S350. Minimize the loss function through gradient descent, update the parameters θ of the sampling policy network, and obtain the target sampling policy network to achieve iterative optimization of the sampling policy.

[0066] Understandably, this application uses gradient descent to minimize the loss function and update the parameters θ of the sampling policy network, which can steadily achieve iterative optimization of the sampling policy and ultimately obtain a target sampling policy network with stronger adaptability and more accurate sampling decisions.

[0067] It should be noted that after obtaining the target sampling strategy network, in the process of wind power prediction in extreme weather, compared with the data augmentation method based on generative models, this application can directly and dynamically select the required samples from the original dataset without synthesizing extreme weather samples, thus completely avoiding the problems of poor quality of synthesized samples and the introduction of noise that leads to a decrease in model performance, while reducing the dependence on computing resources.

[0068] In one example, the loss function satisfies the expression: ; In the formula, The loss is the cost of strategy optimization. For batch size, The reward for the training step at time t. For the policy network in state Select action The probability, Let be the gradient operator with respect to the parameter θ.

[0069] In the embodiments of this application, it is understood that the loss function normalizes the optimization objective by batch size, which can stabilize the gradient scale during training. By combining the reward, the probability of the policy network action, and the gradient operator, the optimization objective of maximizing the cumulative reward in reinforcement learning can be transformed into a loss minimization problem that can be directly solved. The introduction of gradient calculation on the parameter θ of the policy network enables the differentiable update of the sampling policy network parameters, ensuring that the sampling policy network can effectively iterate and optimize based on reward feedback, thereby improving the stability and reliability of the sampling policy update.

[0070] In some embodiments, this application provides an extreme weather wind power prediction system 400 based on reinforcement learning adaptive sampling, such as... Figure 4 As shown, the extreme weather wind power prediction system 400 based on reinforcement learning adaptive sampling may include the following modules: The data processing module 410 is used to acquire meteorological time-series data and wind power data of wind farm sites, construct training sets, validation sets and test sets, and extract training subsets, validation subsets and test subsets corresponding to extreme weather conditions respectively; the training set, validation set and test set all contain meteorological time-series data and wind power data; Module 420 is designed and built to design a reinforcement learning-based training framework, obtain Markov decision process components, and build a parameterized sampling policy network. The wind power prediction model uses a sampling strategy network built on LSTM, and the prediction model is either an LSTM model or a Transformer model. The iterative execution module 430 is used to iteratively execute the collaborative optimization process of the sampling strategy network and the prediction model based on the Markov decision process component until the training converges, and output the optimized target prediction model and target sampling strategy network. The target sampling strategy network is used to sample the acquired data samples in the wind power prediction stage, and the target prediction model is used to predict wind power under extreme weather and normal weather conditions.

[0071] According to embodiments of this application, any multiple modules among the data processing module 410, design and construction module 420, and iterative execution module 430 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module.

[0072] Figure 4 Each module in the system shown has the function of implementing each step in the aforementioned extreme weather wind power prediction method based on reinforcement learning adaptive sampling, and can achieve the corresponding technical effect. For the sake of brevity, it will not be elaborated here.

[0073] In some embodiments, this application provides an electronic device, the structural schematic of which is shown below. Figure 5 As shown.

[0074] The electronic device may include a processor 510 and a memory 520 storing computer program instructions.

[0075] Specifically, the processor 510 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0076] Memory 520 may include mass storage for data or instructions. For example, and not limitingly, memory 520 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 520 may include removable or non-removable (or fixed) media. Where appropriate, memory 520 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 520 is non-volatile solid-state memory.

[0077] Memory 520 may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory 520 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the reinforcement learning-based adaptive sampling extreme weather wind power prediction methods in the above embodiments.

[0078] The processor 510 reads and executes computer program instructions stored in the memory 520 to implement any of the extreme weather wind power prediction methods based on reinforcement learning adaptive sampling in the above embodiments.

[0079] In one example, the electronic device may also include a communication interface 530 and a bus 500. Wherein, such as Figure 5 As shown, the processor 510, memory 520, and communication interface 530 are connected via bus 500 and communicate with each other.

[0080] The communication interface 530 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0081] Bus 500 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 500 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0082] Furthermore, in conjunction with the reinforcement learning-based adaptive sampling method for predicting extreme weather wind power in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the reinforcement learning-based adaptive sampling methods for predicting extreme weather wind power in the above embodiments.

[0083] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0084] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0085] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0086] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting wind power in extreme weather based on reinforcement learning adaptive sampling, characterized in that, include: Meteorological time-series data and wind power data of wind farm sites are acquired, and training sets, validation sets and test sets are constructed. Training subsets, validation subsets and test subsets corresponding to extreme weather are extracted respectively. The training set, the validation set and the test set all contain meteorological time-series data and wind power data. We design a reinforcement learning-based training framework, obtain Markov decision process components, and build a parameterized sampling policy network. The prediction model for wind power is described, wherein the sampling strategy network is constructed based on LSTM, and the prediction model is either an LSTM model or a Transformer model. Based on the Markov decision process component, the collaborative optimization process of the sampling strategy network and the prediction model is iteratively executed until the training converges, and the optimized target prediction model and target sampling strategy network are output. The target sampling strategy network is used to sample the acquired data samples in the wind power prediction stage, and the target prediction model is used to predict wind power under extreme weather and normal weather conditions.

2. The extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in claim 1, characterized in that, Each iteration of the collaborative optimization process of the sampling strategy network and the prediction model includes: Training batches are obtained by sampling from the union of the training set and the training subset; The training batch is input into the sampling policy network. Output mask batch, where the mask batch is a binary vector that matches the dimension of the training batch. 1 in the binary vector indicates that the corresponding sample is included in the training, and 0 in the binary vector indicates that the corresponding sample is discarded. The parameters of the prediction model are updated based on the mask batch, and the updated prediction model is tested based on the test set and the test subset to determine the optimized target prediction model. The sampling policy network is updated using the REINFORCE policy gradient method to obtain the target sampling policy network.

3. The extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in claim 2, characterized in that, The process of determining the components of the Markov decision process includes: state design and action design; the state design is used to provide meteorological context information for the reinforcement learning agent to support informed sampling decisions; the action design is used to make binary decisions on the reinforcement learning sampling strategy, and the actions... Indicates whether to include the current sequence sample in the training batch. Indicates inclusion, Indicates discarding; The state design includes: processing meteorological time-series data using a Long Short-Term Memory (LSTM) network through an agent to generate a meteorological data state representation to capture long-term dependencies in the meteorological data; and concatenating the meteorological data state representation with a uniquely thermally encoded label for the weather type to form a complete state. , ;in, This represents the state of meteorological data output by LSTM. A value of 0 indicates normal weather. A value of 1 indicates extreme weather; The action design includes: using the REINFORCE policy gradient method, outputting classification distribution parameters in the action space through sampling policy and determining the corresponding classification distribution, and sampling actions from the classification distribution to achieve decoupling between differentiable policies and non-differentiable actions.

4. The extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in claim 2, characterized in that, The process of determining the components of the Markov decision process further includes: designing a fine-grained reward function; the fine-grained reward function includes an accuracy reward, a balance reward, and a shutdown event alignment reward; The fine-grained reward function satisfies the expression: In the formula, As a reward for the accuracy, For the balanced reward, Align rewards for the aforementioned downtime events; The accuracy reward satisfies the expression: ; The balanced reward satisfies the expression: ; In the formula, and These represent extreme weather losses and normal weather losses on the validation set, respectively. It is a natural exponential function.

5. The extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in claim 4, characterized in that, When designing the aforementioned downtime event alignment reward, the following is defined: and , This is the set of actual shutdown start times. To predict the set of shutdown start times, Let t be the actual shutdown indicator variable at time t. The actual shutdown indicator variable at time t-1. Let be the predicted shutdown indicator variable at time t. Let be the predicted shutdown indicator variable at time t-1. The logical symbol AND represents AND; and , and These represent the actual wind power and the predicted wind power at time t, respectively. The threshold for determining shutdown; This is an indicator function used to determine whether the condition within the parentheses is true or false. If true, it outputs 1; otherwise, it outputs 0. The alignment reward for the shutdown event satisfies the expression: ； In the formula, A single element representing the actual start time of the shutdown. For a single element to predict the start time of shutdown, Given a time tolerance and .

6. The extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in claim 4, characterized in that, The step of updating the sampling policy network using the REINFORCE policy gradient method to obtain the target sampling policy network includes: The difference between the validation set and the validation subset corresponding to extreme weather is determined as the normal weather validation set; The extreme weather loss of the prediction model on the validation subset and the normal weather loss of the prediction model on the normal weather validation set are calculated, and the reward is obtained by combining the results with a fine-grained reward function. ; The policy parameters are updated by calculating the gradient of the expected cumulative reward relative to the policy parameters; a baseline is introduced. To reduce the variance of gradient estimation; wherein, the baseline This is the moving average of the rewards over the most recent n training steps, where n is a positive integer; Construct a loss function for policy optimization based on reward. And loss function update sampling policy network ; The loss function is minimized by gradient descent, and the parameters θ of the sampling policy network are updated to obtain the target sampling policy network, thereby achieving iterative optimization of the sampling policy.

7. The extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in claim 6, characterized in that, The loss function satisfies the expression: ； In the formula, The loss is the cost of strategy optimization. For batch size, The reward for the training step at time t. For the policy network in state Select action The probability, Let be the gradient operator with respect to the parameter θ.

8. An extreme weather wind power prediction system based on reinforcement learning adaptive sampling, characterized in that, include: The data processing module is used to acquire meteorological time-series data and wind power data of wind farm sites, construct training sets, validation sets and test sets, and extract training subsets, validation subsets and test subsets corresponding to extreme weather conditions respectively; the training set, the validation set and the test set all contain meteorological time-series data and wind power data; The design and construction module is used to design a reinforcement learning-based training framework, obtain Markov decision process components, and build a parameterized sampling policy network. The prediction model for wind power is described, wherein the sampling strategy network is constructed based on LSTM, and the prediction model is either an LSTM model or a Transformer model. The iterative execution module is used to iteratively execute the collaborative optimization process of the sampling strategy network and the prediction model based on the Markov decision process component until the training converges, and output the optimized target prediction model and target sampling strategy network; the target sampling strategy network is used to sample the acquired data samples in the wind power prediction stage, and the target prediction model is used to predict wind power under extreme weather and normal weather conditions.

9. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed by a processor, implement the extreme weather wind power prediction method based on reinforcement learning adaptive sampling as described in any one of claims 1 to 7.

Images