Training method of wind power prediction data model based on hierarchical dual-model fusion

Abstract

The present invention discloses a training method for a hierarchical dual-model fusion wind power prediction data model, which relates to digital intelligent technology in wind power scenarios. The data model first constructs a data layer based on the Transformer architecture and convolutional neural network, and then pre-trains this data layer using a pre-built labeled data set to obtain a dual-model fusion wind power prediction data model, denoted as Model A. Furthermore, an independent data layer is constructed based on Model A, denoted as Model B, thereby forming a hierarchical data model architecture. The hierarchical model architecture proposed in the present invention can significantly improve the efficiency of wind power data intelligent modeling.

Description

Technical Field

[0001] The present invention relates to the field of digital intelligent technology in wind power scenarios, and in particular to a training method for a hierarchical dual-model fusion wind power prediction data model. Background Art

[0002] Renewable energy generation, particularly wind power and photovoltaic power generation, as the mainstay of renewable energy generation, has experienced rapid growth in recent years. However, due to the influence of numerous meteorological factors, such as wind speed, sunlight intensity, and temperature, renewable energy generation exhibits significant volatility and uncertainty. This intermittent and uncontrollable nature poses significant challenges to grid scheduling and stable operation. With the continuous increase in the capacity of renewable energy connected to the grid, effectively predicting renewable energy generation power has become a critical component of grid planning and scheduling. Accurate power forecasting not only ensures safe and stable grid operation but also optimizes grid scheduling, reduces backup capacity and scheduling costs, and thus enhances the economic benefits of renewable energy generation. Currently, renewable energy power forecasting methods are primarily divided into traditional statistical methods and AI-based intelligent forecasting methods. Traditional methods, such as time series analysis and regression models, rely on trend extrapolation from historical data and perform poorly when dealing with complex nonlinear relationships. While intelligent forecasting methods, such as neural networks and support vector machines, can model nonlinear relationships to a certain extent, their prediction results are limited by algorithmic complexity, data volume requirements, and their inability to capture long-term dependencies.

[0003] In recent years, with the rapid development of deep learning technology, deep learning models such as convolutional neural networks (CNNs) and Transformers have demonstrated powerful capabilities in time series data processing and feature extraction. CNNs excel at extracting short-term dependency features from local data and can effectively capture local fluctuation patterns in renewable energy power data. Transformers, on the other hand, can model dependencies over long timeframes through their self-attention mechanism, making them suitable for capturing global features in power forecasting. However, forecasting methods using only CNNs or Transformers often struggle to simultaneously extract both local and global features. Therefore, our technical team considered fusing the two to create a wind power data model with inherently multi-features.

[0004] In particular, further examination of existing power algorithms for various renewable energy sources, such as wind power, reveals that regardless of their internal complexity or the specific form of their data algorithms, they are essentially single-layer data architectures. The introduction and construction of a multi-layer data architecture—particularly considering the temporal nature of meteorological and geographical factors in wind power technology, and the resulting chronological order in which different geographic locations experience similar meteorological conditions—inspired our technical team to construct a new two-layer (or even multi-layer) data architecture, which will also bring beneficial performance improvements to the digital intelligent modeling of wind power. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the various shortcomings of the existing technology and provide a training method for a wind power prediction data model with a hierarchical dual-model fusion. This hierarchical data model not only considers the fusion of CNN or Transformer, but also especially considers the temporal characteristics of meteorological and geographical factors in wind power technology, and the temporal sequence of wind power scenes in different geographical locations experiencing similar meteorological conditions.

[0006] A training method for a hierarchical dual-model fusion wind power prediction data model is proposed. A data layer is constructed based on the Transformer architecture and convolutional neural network, and this data layer is pre-trained using a pre-constructed labeled data set to obtain a dual-model fusion wind power prediction data model, which is denoted as model A. Furthermore, an independent data layer is constructed based on model A, which is denoted as model B, thereby forming a hierarchical data model architecture. In this method, model A serves as the lower-layer data model, model B serves as the upper-layer data model, and model B optimizes all adjustable parameters in model A.

[0007] As a preferred technical solution of the present invention, the B model is constructed as a data model that integrates the following two data elements: element x: meteorological spatiotemporal dynamic characteristics; element y: wind power generation scenario.

[0008] As a preferred technical solution of the present invention, the correlation relationship between element x and element y is: element x considers the spatiotemporal order of the impact of different elements y due to the spatiotemporal dynamic characteristics of meteorological conditions; element x affects wind power generation scenarios in different geographical locations in sequence according to the spatiotemporal order, and the different wind power generation scenarios affected successively correspond to an ordered data set.

[0009] As an optimal technical solution of the present invention, model B is constructed as the upper structure of model A. First, for the pre-trained model A, model B pre-adjusts the adjustable parameters therein, and obtains the variant model set A′ after pre-adjustment; then, each element in the variant model set A′ acts on each element in the ordered data set corresponding to element y; since element x acts on each element of element y in sequence according to the time and space order, model B can perform data observation, that is, after observing each element in the variant model set A′ acting on the first few elements of y, identify one or several variant models whose prediction effect is closest to the actual wind power data, and obtain a new variant model set, denoted as A″, which includes the optimized model A with better prediction efficiency in the current cold wave weather; in this way, model A is adjusted in real time based on A″, thereby realizing real-time dynamic optimization of wind power prediction for the last few elements in the ordered data set corresponding to element y.

[0010] As an optimal technical solution of the present invention, the A model integrates the multi-level feature extraction of Transformer and convolutional neural network to construct a process-based data structure to obtain a dual-model fused wind power prediction data model; the process-based data structure of the A model includes: data process structure one, preprocessing historical wind power data and related meteorological data and generating an input feature matrix; data process structure two, using convolutional neural network CNN to extract local features of the preprocessed input data, capturing short-term dependencies in the wind power time series and generating a preliminary feature representation; data process structure three, inputting the preliminary feature representation in the data process structure two into the Transformer model, extracting long-term time dependency features through the self-attention mechanism and enhancing the global representation capability of the features; data process structure four, performing hierarchical feature fusion on the local features extracted by CNN and the global features extracted by Transformer to generate a comprehensive feature representation; data process structure five, performing multi-step prediction of wind power based on the fused features and using a fully connected layer to output the final prediction result.

[0011] As a preferred technical solution of the present invention, the convolutional neural network in the data process structure 2 adopts a multi-layer convolution structure to abstract the local features in the time series layer by layer and use the pooling layer to perform feature dimensionality reduction.

[0012] As a preferred technical solution of the present invention, the Transformer model in the data process structure three adopts a multi-layer encoder structure. Each layer of the encoder includes a self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism is used to capture the dependency between different time steps in the input sequence, and the global information of the sequence is captured by calculating the correlation weight between each time step and other time steps. At the same time, the position information in the time series is explicitly encoded into the input features in combination with position encoding.

[0013] As a preferred technical solution of the present invention, the multi-level feature fusion in the data process structure four effectively fuses the local features extracted by CNN and the global features extracted by Transformer through weighted summation or splicing; CNN extracts short-term dependency features from the local window and captures the local patterns and detail features in the wind power time series, and Transformer can extract global information and model dependencies over a long period of time through the self-attention mechanism; by fusing the features of the two and utilizing both short-term and long-term information to improve the accuracy of the prediction; in the process of feature fusion, the weighted summation method is used to dynamically adjust the weights according to the importance of local and global features, or the feature splicing method is used to retain more original information; in addition, an additional attention mechanism is introduced after the fusion to improve the model's sensitivity to wind power fluctuations and the prediction accuracy by learning the correlation between different features (focusing on features that are more important to the prediction task).

[0014] As a preferred technical solution of the present invention, the prediction part in the data process structure five adopts a multi-step prediction algorithm structure. By setting multiple output nodes, the wind power changes in multiple time steps in the future are predicted simultaneously, avoiding the problem of error accumulation in step-by-step prediction and directly giving the power prediction results for multiple time points in the future within one prediction cycle; each output node corresponds to the predicted value of a specific time step in the future. The model learns the dependency between the time steps in the time series through training and then makes a comprehensive prediction of the power trend in the future time period.

[0015] After adopting the above technical solution, the beneficial technical effects achieved by the present invention are as follows: In essence, the technical team of the present invention has carried out technical innovations on two levels. First, a multi-level feature extraction algorithm that integrates CNN and Transformer is proposed. Combining the advantages of both CNN and Transformer can more comprehensively model the timing characteristics of new energy power and improve prediction accuracy and reliability; second, the introduction and construction of a two-layer data architecture, which is the core of the technical development of the present invention. The timing characteristics of wind power generation under meteorological and geographical conditions have inspired our technical team to construct a new two-layer data architecture, which is expected to bring significant efficiency improvements to the digital intelligent modeling of wind power. Further technical effects of the present invention are also described in detail in Examples 1 and 3 below. DETAILED DESCRIPTION

[0016] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0017] In the following description of the embodiments, specific details such as specific system structures and techniques are provided for illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0018] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0019] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0020] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0021] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0023] Example 1: Wind power prediction data model based on dual-model fusion (Model A)

[0024] Data model A, the underlying (lower-layer) data model, is constructed as a dual-model data structure that integrates a Transformer and a convolutional neural network. This is what we call a dual-model fusion wind power prediction data model. In summary, the data structure of model A includes: a) preprocessing historical wind power data and related meteorological data, including data cleaning, normalization, and missing value processing; b) using a convolutional neural network (CNN) to extract local features from the preprocessed input data, capturing short-term dependencies in the wind power time series and generating a preliminary feature representation; c) inputting the preliminary feature representation from b into the Transformer model, extracting long-term temporal dependency features through a self-attention mechanism to enhance the global representation capability of the features; d) performing multi-level feature fusion of the local features extracted by the CNN and the global features extracted by the Transformer to generate a comprehensive feature representation; e) performing multi-step wind power forecasting based on the fused features, and outputting the final forecast using a fully connected layer.

[0025] Among them, the Transformer model in c adopts a multi-layer encoder structure. Each layer of the encoder includes a self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism is used to capture the dependencies between different time steps in the input sequence, which can effectively model the long-range dependency information in the wind power time series. The self-attention mechanism calculates the correlation weight between each time step and other time steps to capture the global information of the sequence, which can overcome the problem that traditional recurrent neural networks are difficult to handle long-distance dependencies. At the same time, combined with position encoding, the position information in the time series is explicitly encoded into the input features, which can make up for the lack of sequential information in the Transformer model itself, enabling the model to identify the changing trend in the time sequence and enhance the accuracy and stability of wind power time series prediction.

[0026] Among them, the multi-level feature fusion in d effectively fuses the local features extracted by CNN and the global features extracted by Transformer through weighted summation or splicing; CNN can extract short-term dependency features from local windows and capture local patterns and detailed features in wind power time series, while Transformer can extract global information and model dependencies over a long period of time through the self-attention mechanism; by fusing the features of the two, both short-term and long-term information can be utilized to improve the accuracy of prediction; in the process of feature fusion, a weighted summation method can be used to dynamically adjust the weights according to the importance of local and global features, or feature splicing can be used to retain more original information; in addition, an additional attention mechanism is introduced after fusion to further optimize the fusion effect. By learning the correlation between different features, it focuses on the features that are more important to the prediction task, thereby improving the model's sensitivity to wind power fluctuations and the accuracy of prediction.

[0027] Among them, the prediction part in e adopts a multi-step prediction strategy. By setting multiple output nodes, the model can simultaneously predict the changes in wind power in multiple time steps in the future. This multi-step prediction strategy can not only avoid the problem of error accumulation in step-by-step prediction, but also directly provide power prediction results for multiple time points in the future within one prediction cycle, greatly improving the timeliness and practicality of the model. Each output node corresponds to the predicted value of a specific time step in the future. Through training, the model can learn the complex dependencies between the time steps in the time series, and then make a comprehensive and accurate prediction of the power trend in the future period. While improving the prediction efficiency, this strategy also makes the model more adaptable, able to cope with fluctuations in wind power on different time scales, and enhance the feedback and guidance of the actual operation of the wind farm.

[0028] Example 2: Data execution process of model A in application scenario

[0029] Model A is a data model based on Transformer and convolutional neural networks (CNN) for predicting the power of renewable energy sources such as wind power. From the perspective of data execution, it mainly includes the following steps:

[0030] S1: Collect historical power data and related meteorological data of wind power renewable energy power stations. Meteorological data include but are not limited to wind speed, wind direction, temperature, humidity, light intensity, etc., which are obtained from on-site sensors or weather forecasts. Historical power data mainly refers to the power output of wind farms, in kilowatts (kW) or megawatts (MW). The correlation between meteorological data and power data forms the basis of power prediction, and the collected data will serve as the input features of the model. Among them, the renewable energy power output records the actual power generation of a wind farm in kW or MW. In meteorological data such as wind speed, wind direction and temperature, the wind speed unit is m / s, the wind direction is expressed in degrees, and the temperature unit is degrees Celsius.

[0031] S2: Preprocess the collected historical data. This includes removing outliers, processing missing values, and normalizing the data. By constructing a scatter plot of wind power renewable energy power output and meteorological factors, the distribution of abnormal data can be observed. Abnormal data includes but is not limited to records of low output power under high wind speed or high light conditions, or data with constant power and large fluctuations in meteorological conditions. These data are identified and eliminated. At the same time, all input features are normalized so that they are in the same numerical range to facilitate stable training of subsequent models. In specific implementation, a scatter plot is first used to visualize the relationship between meteorological data and power output to determine abnormal points and abnormal patterns, and then all input features are normalized to ensure that the ranges of different feature values are similar and to reduce the impact of numerical scale differences on model training.

[0032] S3: Construct a convolutional neural network model and extract short-term features from the data. The CNN model first extracts local features from meteorological data and historical power data through convolution operations, identifying short-term dependencies in renewable energy power fluctuations. The convolution layer further automatically captures local feature patterns between meteorological data and power output, extracting changing trends in time series data. Specifically, consider first constructing a multi-layer convolutional neural network, where each layer extracts short-term dependency features using convolution kernels. Then, a pooling layer is used to reduce feature dimensionality. This compresses the feature space while retaining important information and outputting local feature representations processed by the convolution layer as input for subsequent feature fusion.

[0033] S4: The local features extracted by the convolutional neural network are input into the Transformer model for global feature extraction. The Transformer uses a self-attention mechanism to capture the long-term dependencies between different time steps in the input sequence. Combined with positional encoding, the Transformer ensures the model's ability to recognize temporal order information, effectively modeling the long-term temporal dependencies involved in power prediction. In particular, the multi-head self-attention mechanism is used to calculate the attention weights between each time step to identify long-term dependencies. Positional encoding is used to incorporate positional information in the time series into the model input, enabling the model to capture temporal features. The final global feature representation is output and subsequently fused with local features.

[0034] S5: Perform multi-level feature fusion. Perform a weighted sum or concatenation of the local features extracted by the convolutional neural network and the global features extracted by the Transformer to generate a multi-level fused feature representation. This captures both local details and global trend information in renewable energy power forecasting and improves forecast accuracy. Specifically, consider using a weighted summation method to weight local and global features by importance and then add them together; or consider using feature concatenation to retain the complete information of both types of features, and then introducing a secondary attention mechanism to further optimize the correlation between different features, making the final feature fusion result more accurate.

[0035] S6: Perform multi-step predictions and output future power values. A fully connected layer is used to perform multi-step predictions of renewable energy power based on the fused multi-level features. By setting up multiple output nodes, power output predictions for multiple future time steps are achieved, ensuring that the prediction results cover a longer time range, thereby avoiding the accumulation of errors in step-by-step predictions. When constructing a fully connected layer with multiple output nodes, the fused features are input into the fully connected network, and the prediction results for each time step are output when the power for multiple future time steps is simultaneously predicted. In general, the prediction results need to be verified using actual data, and the accuracy of the predictions needs to be improved by iteratively optimizing the model parameters.

[0036] Example 3: Wind power prediction data model with dual-model fusion having a hierarchical architecture (Model A + Model B)

[0037] Model A is a data model for wind power forecasting that uses a multi-level feature extraction algorithm that integrates the Transformer and convolutional neural networks. Based on Model A, an independent AI model, Model B, is constructed, forming a dual-model architecture. Model B optimizes all adjustable parameters in Model A (here, Model A, after pre-training and iterative optimization).

[0038] Specifically, the core of model B is to build a special data model that integrates the following two elements: element x: meteorological spatiotemporal dynamic characteristics; element y: wind power generation scenario.

[0039] Factor x mainly considers the spatiotemporal order in which the spatiotemporal dynamic characteristics of meteorological conditions affect different factors y. For example, cold wave weather will affect wind power generation scenarios in different geographical locations in a certain spatiotemporal order. At this time, the spatiotemporal geographical dynamic characteristics of the current cold wave weather, namely factor x, and the different wind power generation scenarios it affects in succession are an ordered set, recorded as factor y = (yi, i = 1, 2, 3, ...).

[0040] The working process of Model B is as follows:

[0041] First, for the optimized A model, the B model pre-adjusts the adjustable parameters. The pre-adjustment generally adjusts the point value to a value range. Of course, various types of pre-adjustments can be performed according to the specific situation. In any case, after the pre-adjustment, the A model actually corresponds to multiple models, which are recorded as the variant model set A′=(A′ j , j=1, 2, 3, ...).

[0042] Then, each element in the mutation model set A′ will act on y=(y i , i = 1, 2, 3, ...) each element y i Because the cold wave weather in element x acts on each element y in the order of time and space, i Therefore, model B can perform data observation afterwards, that is, observe each element A′ in the variant model set A′ j After applying the first few elements of y (the first few elements of y represent wind power generation scenarios where a cold wave has already arrived), we identify one or several variant models whose prediction results are closest to the actual wind power data, and obtain a new set of variant models, denoted as A″.

[0043] A″ includes an optimized A model with better predictive efficiency in the current cold wave weather. In this way, the A model can be adjusted in real time based on A″, so that the predictive efficiency of the optimization model can be optimized in real time and dynamically for the last few elements of factor y (the last few elements of y are wind power generation scenarios where the cold wave has not yet arrived).

[0044] Overall, for the wind power data model under the two-layer data architecture, in terms of the fusion construction of model A, the method of the present invention realizes multi-level feature extraction of the time series of renewable energy power generation by fusing convolutional neural networks and Transformer models, which can effectively capture local and global time-dependent features, thereby significantly improving the accuracy of power prediction. The present invention adopts a multi-step prediction strategy, and by setting multiple output nodes, it can simultaneously predict multiple future time steps, effectively avoiding the problem of error accumulation in step-by-step prediction, and improving the efficiency and timeliness of prediction. By introducing the self-attention mechanism and position encoding, the model can more flexibly capture long-range dependencies in complex time series, making up for the shortcomings of traditional machine learning methods in processing nonlinear and multidimensional time series data. The multi-level feature extraction algorithm based on CNN and Transformer can effectively improve the accuracy of renewable energy power prediction, solve the shortcomings of traditional prediction methods and single deep learning methods in modeling complex time series dependencies, and provide more accurate data support for the scheduling and planning of renewable energy grid-connected power generation.

[0045] At the same time, it is worth mentioning that this fusion model is constructed based on the consideration of wind power scenarios, but it can be seen that it is also applicable to processing various energy scenarios such as wind power and photovoltaics in new energy power generation, and has broad application prospects.

[0046] Furthermore, with regard to the construction of the two-layer data architecture of the A+B model, in the A model, the CNN convolution kernel sweeps across the spatial map of meteorological data, capturing spatial features of different scales, such as capturing micro-meteorological features such as local wind shear and temperature gradient; the Transformer multi-head attention mechanism cruises through the long river of time, keeping a close eye on the complex relationship between wind power and meteorology over time. The fusion of the two lays a solid feature foundation for wind power prediction, fully outlines the blueprint of the spatiotemporal impact of meteorology-power, makes the input model information richer and more accurate, and improves the prediction resolution from the root; on this basis, considering the orderly adaptation of dynamic and complex meteorological scenes, the B model is constructed to optimize the parameters of the A model, closely following the dynamic spatiotemporal characteristics of meteorology (such as the rhythm of cold waves invading wind farms in geographical order), and ensures that the parameters of the A model keep pace with the times through pre-adjustment and practical screening of variant models, and flexibly responds to extreme and gradual meteorological changes, breaking the static shackles of traditional models, making the prediction model a "smart" model that dances with the weather while enhancing the model robustness and generalization in different wind power scenarios. In this way, in the field of real-time wind power prediction, this dual-model architecture relies on real-time observation, evaluation, and screening by Model B to continuously inject optimization genes into Model A. Whenever there is a weather fluctuation, it quickly adjusts its pace and accurately checks the subsequent wind power output, nipping prediction errors in the bud. This provides highly reliable and high-precision power forecasts for downstream links such as grid dispatching and power market transactions, ensuring the efficient grid connection of wind power and the stable operation of the power system.

[0047] Example 4: Data-based Example of the A+B Model in an Application Scenario (Taking a Cold Wave Meteorological Event as an Example)

[0048] In the basic model architecture, Model A combines a Transformer and convolutional neural network approach, utilizing mathematical operations such as convolution and multi-head attention to process historical wind power time series data and meteorological data. For Model B, integrating meteorological spatiotemporal dynamics with wind power generation scenarios requires data-driven representation of meteorological elements, calculation of meteorological propagation, and pre-adjustment of Model A parameters. The data processing process is divided into an offline training phase and a real-time prediction and optimization phase. In the offline training phase, Model A is first trained using a mean squared error loss function and backpropagation, with parameters updated using stochastic gradient descent and its variants. Model B is then constructed and initialized, determining the spatiotemporal dynamics of meteorology and the sequence of power generation scenarios. Model A parameters are pre-adjusted to generate a set of variant models. In the real-time prediction and optimization phase, meteorological data is monitored in real time, power generation scenarios are ranked, and variant models are applied to these scenarios for evaluation. The variant models are then selected and optimized, and finally, real-time predictions are performed using the updated models. This cycle continues until the meteorological event ends. The entire process combines deep learning model architecture design, meteorological physical process modeling and mathematical optimization methods, and is one of the core development projects of this technical team.

[0049] ①、A model related settings and processing

[0050] Assume that the data input to model A include the historical time series of wind power P(t) (t=1, 2,…, T) and the corresponding meteorological data M(t) (containing a variety of meteorological information, recorded in time steps).

[0051] For meteorological data M(t) (the dimension is assumed to be m×n), a convolutional neural network (CNN) is used to extract local spatial features. s (size is k×k, s=1,2,…,S) and perform convolution operation to obtain the feature map F s (t), the formula is: F s (t)=Conv(M(t),K s ), where Conv represents the convolution operation, which is calculated by sliding the convolution kernel with a step size of p.

[0052] The wind power time series P(t) and the meteorological features extracted by convolution are concatenated into the input sequence X(t) = [P(t), F1(t), F2(t), ..., F S (t)] (after the dimension is adjusted to d), which is input to the Transformer encoder.

[0053] Transformer processing and output:

[0054] In the Transformer encoder, the Multi-HeadAttention mechanism is calculated as follows:

[0055] First calculate Q = X(t)W Q , K=X(t)W K , V=X(t)W V (W Q , W K , W V , W O is the learnable weight matrix).

[0056] Then head i =Attention(QW i Q ,KW i K ,VW i V ),in (d k is the dimension of K).

[0057] Finally, MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O (h is the number of heads).

[0058] The multi-head attention mechanism is followed by a feedforward neural network (FFN) for nonlinear transformation to obtain the wind power prediction value output by model A. The set of adjustable parameters of model A is denoted as θ A .

[0059] ②. B model related settings and processing

[0060] Element x (meteorological spatiotemporal dynamic characteristics) represents:

[0061] Meteorological data are gridded in the geographic space G (divided into I×J grid cells, i=1,2,…,I, j=1,2,…,J). For meteorological elements (such as wind speed, wind direction, temperature, etc.) at time step t, the meteorological characteristic vector of cell (i,j) is expressed as x ij (t)=[v ij (t),ω ij (t),T ij (t)].

[0062] Considering the meteorological propagation dynamics, such as the cold wave propagation speed c (two-dimensional vector), for the target wind farm location set (N is the number of power plants), the cold wave starts to propagate from the starting grid unit (i0, j0) at time t0 and reaches the power plant l k Time t k satisfy: (Here |·| represents a two-dimensional space distance metric), the power plants are sorted by arrival time to obtain an ordered set y=(y i ,i=1,2,…,N), corresponding to element y (wind power generation scenario).

[0063] B model core optimization process:

[0064] For the parameter θ of model A A Perform pre-adjustment and set each parameter θ a ∈θ A (a=1,2,…,A, A is the total number of parameters) Convert the point value to a range of values, such as using the normal distribution perturbation method to generate the parameter set θ A′ (corresponding to the mutation model set A'), where each element satisfy: Here j ~N(0,σ 2 I)(j=1,2,…,J, J is the number of pre-adjusted generated variant models, σ is the standard deviation control disturbance amplitude, and I is the unit matrix).

[0065] ③. Data processing flow

[0066] Offline training phase:

[0067] A model training:

[0068] Using historical wind power data P(t) and meteorological data M(t) (divided into training set, validation set, and test set), the mean square error (MSE) loss function is adopted:

[0069] The parameters θ are updated using the backpropagation algorithm (gradients are calculated according to the chain rule, such as the derivation of the convolutional layer weights involves the derivation rule of the convolution operation, and the derivation of the Transformer weights is performed module by module according to its structure) using stochastic gradient descent (SGD) and its variants (such as the Adam optimizer). A , train until convergence (validation set loss no longer decreases) and obtain the initial better A model.

[0070] B model construction and initialization:

[0071] Based on the statistical analysis of historical meteorological data, the spatiotemporal dynamic characteristics of typical meteorological processes such as cold waves are analyzed, and the starting grid, propagation speed and other parameters are determined to construct the element x representation method, and the sequence set y corresponding to different power generation scenarios is sorted out.

[0072] The parameters of the trained A model are pre-adjusted in the above manner to generate an initial mutation model set. When meteorological events such as cold waves are monitored, the arrival time at each wind farm is calculated based on the meteorological spatiotemporal dynamic characteristic function, and the power generation scenario is sorted and updated with the factor y.

[0073] Role and evaluation of mutation models:

[0074] Each model A in the variant model set A' j′ Act on the elements of y in turn i (The corresponding meteorological and historical power data are input into each variation model to obtain the predicted power series ).

[0075] Using a rolling window evaluation method, the MSE error between the predicted power and the actual power of each variant model in the first few (set as K) power generation scenarios (already affected by meteorological conditions) is calculated:

[0076] Screening and optimizing mutation models:

[0077] Based on the calculated MSE j Sort the variant models, select the first M (M < J) variant models with the smallest errors to form a new set A", extract the mean or weighted mean of the corresponding A model parameters (the weight is set inversely according to the error) as the optimized parameter Update A model.

[0078] Real-time prediction for wind power applications: Utilizing the updated model, combined with real-time meteorological and historical power data, power predictions are made for subsequent wind power generation scenarios (not fully affected by the weather), dynamically optimizing prediction performance. The steps of the above real-time prediction and optimization phases are continuously cycled until the weather event ends.

[0079] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0080] In various embodiments, the hardware implementation of the technology can directly utilize existing intelligent devices, including but not limited to industrial computers, personal computers, smart phones, handheld computers, floor-standing computers, etc. The input device preferably utilizes an on-screen keyboard, the data storage and calculation modules utilize existing memories, calculators, and controllers, the internal communication modules utilize existing communication ports and protocols, and remote communication utilizes existing GPRS networks, the World Wide Web, etc.

[0081] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0082] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and methods can be implemented in other ways. For example, the apparatus / terminal equipment embodiments described above are merely schematic. For example, the division of modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0083] The functional units in the various embodiments of the present invention may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units. If the integrated modules / units are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on this understanding, the present invention may implement all or part of the processes in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program may be stored in a computer-readable storage medium, and when executed by a processor, the computer program may implement the steps of the above-mentioned method embodiments. The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. Computer-readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium.

[0084] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A training method for a hierarchical dual-model fusion wind power prediction data model is characterized by: A data layer is constructed based on the Transformer architecture and convolutional neural network. This data layer is pre-trained using a pre-built annotated dataset to obtain a wind power prediction data model that integrates two models, denoted as Model A. Based on Model A, an independent data layer is constructed, denoted as Model B, thus forming a hierarchical data model architecture. The B model is constructed as a data model that integrates the following two data elements: element x: meteorological spatiotemporal dynamic characteristics; element y: wind power generation scenario; the B model is constructed as the upper structure of the A model. First, for the pre-trained A model, the B model pre-adjusts the adjustable parameters therein, and obtains the variation model set A′ after pre-adjustment; then, each element in the variation model set A′ acts on each element in the ordered data set corresponding to the element y; since the element x acts on each element of the element y in sequence according to the spatiotemporal order, the B model can perform data observation, that is, after observing each element in the variation model set A′ acting on the first few elements of y, identify one or several variation models whose prediction effect is closest to the actual wind power data, and obtain a new variation model set, recorded as A′′; based on A′′, the A model is adjusted in real time, thereby realizing real-time dynamic optimization of wind power prediction for the last few elements in the ordered data set corresponding to the element y; The relationship between factor x and factor y is: factor x considers the spatiotemporal order of the impact on different factors y due to the spatiotemporal dynamic characteristics of meteorological conditions; factor x affects wind power generation scenarios in different geographical locations in sequence according to the spatiotemporal order, and the different wind power generation scenarios affected in sequence correspond to an ordered data set.

2. The training method of the hierarchical dual-model fusion wind power prediction data model according to claim 1 is characterized by: The model A integrates the multi-level feature extraction of Transformer and convolutional neural network to construct a process-based data structure to obtain a wind power prediction data model that integrates the two models; the process-based data structure of model A includes: data process structure one, preprocessing historical wind power data and related meteorological data and generating an input feature matrix; data process structure two, using convolutional neural network CNN to extract local features of the preprocessed input data, capturing short-term dependencies in the wind power time series and generating a preliminary feature representation; data process structure three, inputting the preliminary feature representation in data process structure two into the Transformer model, extracting long-term time dependency features through the self-attention mechanism and enhancing the global representation capability of the features; data process structure four, performing hierarchical feature fusion on the local features extracted by CNN and the global features extracted by Transformer to generate a comprehensive feature representation; data process structure five, performing multi-step prediction of wind power based on the fused features and using a fully connected layer to output the final prediction result.

3. The training method of the hierarchical dual-model fusion wind power prediction data model according to claim 2 is characterized by: The convolutional neural network in the data process structure 2 adopts a multi-layer convolution structure to abstract the local features in the time series layer by layer and use the pooling layer to perform feature dimensionality reduction.

4. The method for training a hierarchical dual-model fusion wind power prediction data model according to claim 2, characterized in that: The Transformer model in data process structure three adopts a multi-layer encoder structure. Each layer of the encoder includes a self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism is used to capture the dependencies between different time steps in the input sequence, and the global information of the sequence is captured by calculating the correlation weights between each time step and other time steps. At the same time, the position information in the time series is explicitly encoded into the input features in combination with position encoding.

5. The training method of the hierarchical dual-model fusion wind power prediction data model according to claim 2 is characterized in that: The multi-level feature fusion in the data process structure four effectively fuses the local features extracted by CNN and the global features extracted by Transformer through weighted summation or splicing; CNN extracts short-term dependency features from the local window and captures the local patterns and detail features in the wind power time series. Transformer can extract global information and model dependencies over a long period of time through the self-attention mechanism; by fusing the features of the two and utilizing both short-term and long-term information, the accuracy of the prediction is improved; in the process of feature fusion, the weighted summation method is used to dynamically adjust the weights according to the importance of local and global features, or feature splicing is used to retain more original information; in addition, an additional attention mechanism is introduced after the fusion to improve the model's sensitivity to wind power fluctuations and prediction accuracy by learning the correlation between different features.

6. The method for training a hierarchical dual-model fusion wind power prediction data model according to claim 2, characterized in that: The prediction part of the data process structure five adopts a multi-step prediction algorithm structure. By setting multiple output nodes, it simultaneously predicts the changes in wind power in multiple time steps in the future, avoiding the problem of error accumulation in step-by-step prediction and directly giving the power prediction results for multiple time points in the future within one prediction cycle; each output node corresponds to the prediction value of a specific time step in the future. The model learns the dependency between each time step in the time series through training and then makes a comprehensive prediction of the power trend in the future time period.