Ultra-short-term irradiance estimation method based on feature mining and seq2seq model

Abstract

The application relates to a kind of super-short-term irradiance measurement methods based on feature mining and Seq2Seq model, the method comprises the following steps: through the abnormal data cleaning rule, the abnormal data of night or continuous zero irradiance is excluded, the difference feature is generated using multivariate scale self-adaptive difference construction method, and the solar elevation angle time sequence feature is reconstructed in combination with space-time dynamics, the negative value is retained to enhance the difference of adjacent date; the local nonlinear relationship of solar elevation angle and meteorological variable is extracted using learnable convolution kernel, and the feature is fused and spliced through full connection layer; a CDAtt-LSTM hybrid model is constructed, based on Seq2Seq architecture, a double-layer LSTM is used to process historical data in the encoder, the encoder and decoder historical state are fused through the cross-stage dynamic attention mechanism in the decoder, and a stage weight function is introduced. The application significantly improves the accuracy and stability of irradiance prediction through the dynamic coupling of physical characteristics and deep learning.

Description

Technical Field

[0001] This invention belongs to the field of electrical digital data processing, specifically relating to a method for calculating ultra-short-term irradiance based on feature mining and a Seq2Seq model. Background Technology

[0002] Existing methods for ultra-short-term irradiance prediction mainly include numerical weather prediction, cloud image-based neural network methods, and traditional machine learning methods. Numerical weather prediction methods are limited by spatiotemporal resolution and have high computational costs, making them unsuitable for ultra-short-term irradiance prediction. Cloud image-based neural network methods are costly to acquire historical data, making them unsuitable for large-scale applications. Traditional machine learning methods lack sufficient learning ability when dealing with high-dimensional and complex meteorological data, making it difficult to learn the inherent patterns of weather systems. Summary of the Invention

[0003] This invention proposes a method for calculating ultra-short-term irradiance based on feature mining and a Seq2Seq model, which includes the following steps:

[0004] Data preprocessing: Historical meteorological data are cleaned, differential features are generated using a multivariate scale adaptive difference construction method, solar altitude angle is calculated using the spatiotemporally dynamically reconstructed solar altitude angle time series features, and the nonlinear coupling features between solar altitude angle and meteorological variables are extracted.

[0005] Data reading and sample normalization: A sliding window is used to divide the samples into multiple consecutive time points and batch normalization is performed.

[0006] CDAtt-LSTM hybrid model prediction: Construct a deep learning model based on Seq2Seq structure. The encoder uses a two-layer LSTM to process historical time data. The decoder sub-module includes LSTM, cross-stage dynamic attention mechanism and linear output layer. The cross-stage dynamic attention mechanism dynamically weights and fuses the encoder hidden state and the decoder historical hidden state. The stage weight function distinguishes the prediction focus of different stages and finally outputs the irradiance prediction result.

[0007] Specifically, the nonlinear coupling features between solar elevation angle and meteorological variables are extracted by using a sliding window to locally interact with the solar elevation angle and meteorological variables, combining learnable convolutional kernels to capture nonlinear relationships, and fusing the concatenated feature vectors through a fully connected layer.

[0008] Specifically, the coupling features include local exchange terms and feature fusion terms.

[0009] Specifically, the coupling characteristics are as follows:

[0010] ;

[0011] Where θs (t+kΔt) represents the solar altitude angle at time t+kΔt; X n (t+kΔt) represents the values ​​of each meteorological variable at time t+kΔt, n∈{1,2,3,4}, corresponding to temperature, air pressure, humidity, and wind speed respectively; k is the offset of the convolution sliding window, with the center point at the current time t, k∈{-2,-1,0,1,2}; W n t+kΔt The learnable convolution kernel relating the nth meteorological variable to the solar altitude angle; [θ s [(t);X(t)] represents the vector concatenated from the solar altitude angle and meteorological variables at time t. LayerNorm is the layer normalization function; W1 and W2 are the weight matrices of the two fully connected layers; b1 and b2 are the bias terms of the two fully connected layers; GELU is the activation function.

[0012] Specifically, the cross-stage dynamic attention mechanism involves performing dot product attention calculations between the current hidden state of the decoder and the hidden states of the encoder and the historical decoder, generating a dynamically weighted hidden state output.

[0013] Specifically, the prediction focus at different stages is as follows: the high-precision stage focuses on the linear layer weights of local mutations, while the extended stage focuses on the linear layer weights of periodic correlations.

[0014] Specifically, the solar altitude angle is calculated using the time-series characteristics of the solar altitude angle reconstructed by spatiotemporal dynamics. Specifically, based on longitude, latitude, date and time, the solar altitude angle is reconstructed through the dynamic relationship between the solar hour angle and the declination angle, and its negative value characteristics are retained to enhance the physical correlation of the time series.

[0015] Specifically, historical meteorological data is cleaned by dividing the data into subsequences by day. If the irradiance at the beginning and end of a subsequence is not zero, or the irradiance is zero throughout the day, it is considered abnormal data and is removed.

[0016] Specifically, differential features are generated through a multivariate scale adaptive difference construction method. Specifically, for each variable among irradiance, temperature, air pressure, humidity, and wind speed, the difference between the current time and the time n hours ago is calculated, and the difference scale parameter n is optimized through grid search.

[0017] Specifically, a sliding window is used to divide the samples into multiple consecutive time steps. Specifically, the window size is 98 consecutive time steps, and only samples with consecutive time steps are retained for model training. The sliding step size is 1 time step.

[0018] Beneficial Technical Effects: This invention significantly improves the accuracy and stability of ultra-short-term irradiance prediction by integrating physical features with a deep learning model. Based on the spatiotemporally dynamic reconstruction of solar altitude angle time-series features and a multivariate adaptive difference construction method, it effectively captures the nonlinear coupling relationship between meteorological variables and irradiance, resulting in a reduced mean absolute error compared to traditional LSTM models. The cross-stage dynamic attention mechanism combined with staged weight adjustment enables the model to adapt to the dynamic changes of different weather systems, reducing the fluctuation range of the root mean square error in cross-seasonal data testing. Attached Figure Description

[0019] The invention will now be further described with reference to the accompanying drawings.

[0020] Figure 1 The flowchart shows the ultra-short-term irradiance measurement method based on feature mining and Seq2Seq model of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of the Seq2Seq model of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, this invention discloses a method for calculating ultra-short-term irradiance based on feature mining and a Seq2Seq model, which includes the following steps:

[0024] 1. Data Preprocessing

[0025] (1) Data cleaning. During the data collection process, various errors may occur, such as data entry errors and measurement errors. This will result in various unreasonable or incorrect data in the dataset, and the subsequent prediction performance may be affected by these errors.

[0026] First, divide the entire dataset into subsequences based on daily times. Find all data in the dataset where the "Time" column is 00:00:00 of a specific day. Starting from this data, take data from 96 times forward (including this time) to form one subsequence. All subsequences together form the subsequence set. Where N represents the total number of days in the entire dataset sequence, and the subsequence of day i is... , where x t iThis represents the data at time t on day i.

[0027] For each Wi subsequence, if the data at both ends (x1) i x 96 i If any one of the values ​​in the sequence is not zero, the subsequence can be considered abnormal data. If the above condition is not met, then... If all values ​​are 0, then this subsequence can also be considered as abnormal data.

[0028] Based on the above two conditions, we can identify outlier data in the entire dataset, remove these outlier data, and complete the data cleaning process.

[0029] (2) Multivariate scale-adaptive differencing construction method. Using the diff function in the Pandas library, the data corresponding to each time step of various variables such as irradiance, direct irradiance, diffuse irradiance, temperature, air pressure, humidity, and wind speed in the dataset are differencing the data n hours ago (n×4 time steps ago) to obtain a series of differencing features. At the same time, based on the results, parameter optimization methods such as grid search are used to specialize the differencing scale and differencing objects, so that the differencing method has the characteristics of multivariate and scale adaptation.

[0030] (3) Temporal characteristics of solar altitude angle reconstructed by spatiotemporal dynamics. Unlike traditional methods that use solar altitude angle as a basis for calculating theoretical ideal irradiance, this invention uses it as a novel physically meaningful temporal feature in the field of deep learning and applies it during deep learning training.

[0031] The method for calculating the solar altitude angle at a certain location and time (given longitude lon, latitude lat, 24-hour Beijing time t, and Beijing date N) is as follows:

[0032] h=15*(t+(lon-120°) / 15°-12)

[0033] δ=-22.34°*cos((N+10)*360 / 365)

[0034] θ s =sin -1 (coshcosδcos(lat)-sinδsin(lat))

[0035] Where h represents the solar hour angle, calculated based on 24-hour Beijing time t and local longitude lon; δ represents the solar declination, calculated based on Beijing date N; θ s This is the calculated solar altitude angle.

[0036] Unlike traditional methods of using solar altitude angle, this method utilizes the trigonometric-like properties of solar altitude angle, retaining the negative values ​​in the calculation process. This results in a time characteristic that is consistent with reality, characterized by "small differences at the same time on adjacent dates, significant differences on distant dates, and huge differences at different times on the same date." By integrating physical methods into the field of deep learning, the model achieves better predictive performance.

[0037] (4) Extraction of coupling features between solar altitude angle and meteorological variables. The attenuation coefficients of meteorological variables such as temperature, air pressure, humidity, and wind speed to radiation are nonlinearly related to the solar altitude angle. Therefore, a learnable feature coupling method can be designed to couple the solar altitude angle with meteorological variables and capture their nonlinear relationship. The coupling feature extraction is achieved through the following steps: using the solar altitude angle within a sliding window to interact locally with meteorological variables, combining a learnable convolutional kernel to capture the nonlinear relationship, and fusing the concatenated feature vector through a fully connected layer. Coupling feature C t This includes local exchange terms and feature fusion terms, specifically:

[0038] ;

[0039] Where θ s (t+kΔt) represents the solar altitude angle at time t+kΔt; X n (t+kΔt) represents the values ​​of each meteorological variable at time t+kΔt, n∈{1,2,3,4}, corresponding to temperature, air pressure, humidity, and wind speed respectively; k is the offset of the convolution sliding window, with the center point at the current time t, k∈{-2,-1,0,1,2}, i.e., covering a 75-minute window; W n t+kΔt The learnable convolution kernel relating the nth meteorological variable to the solar altitude angle; [θ s [(t);X(t)] represents the vector concatenated from the solar altitude angle and meteorological variables at time t. LayerNorm is the layer normalization function; W1 and W2 are the weight matrices of the two fully connected layers; b1 and b2 are the bias terms of the two fully connected layers; GELU is the activation function.

[0040] 2. Data Reading and Sample Normalization

[0041] (1) Data reading. Since the meteorological data is saved as a txt table, the read_csv function in the Pandas library is used to read the meteorological data file. After the meteorological data is read, it is stored in memory as a DataFrame type variable, and various operations can be performed on it using various functions in the Pandas library.

[0042] Because the goal of the forecasting task is to predict the irradiance for the next 7 hours (28 times) based on relevant meteorological data from 17.5 hours (70 times) of history, the data for the entire year must be divided into multiple samples containing 98 consecutive times.

[0043] A sliding window method is used to partition the samples. First, the window size is set to 98, meaning 98 consecutive rows of data are retrieved from the DataFrame variable each time. Second, the Datetime column is read from each of the 98 consecutive rows. The time information in the Datetime column determines whether the data is temporally consecutive. If it is, the sample can be used for model training and added to a list for later use; otherwise, it is discarded. Third, the starting row of the window for sample extraction is incremented by one, and the above steps are repeated until the entire dataset has been traversed by the sliding window, resulting in a list of qualified samples.

[0044] (2) Sample Normalization. The Dataset and DataLoader functions from the torch.utils.data library are used to read the above sample list, with the batch_size parameter controlling the number of samples in each batch. After reading a batch of samples, normalization is required before feeding them into the model for training. We adopted a normalization method different from global normalization, namely batch normalization, which performs normalization within a single batch of data. This normalization method can effectively improve prediction performance under extreme weather conditions. The batch normalization formula is shown below:

[0045] X batch_norm =(X-min batch ) / (max batch -min batch )

[0046] Where X represents a variable in the original data, max batch min batch X represents the maximum and minimum values ​​of this variable in a single batch of data. batch_norm These are the normalized variables. All variables in the data are normalized using the above normalization formula.

[0047] 3. Hybrid model structure combining cross-stage dynamic attention LSTM (CDAtt-LSTM)

[0048] like Figure 2As shown, the deep learning model used in this invention is a novel deep learning fusion model based on a Seq2Seq structure containing an encoder and a decoder. Both the encoder and decoder employ Long Short-Term Memory (LSTM) networks, combined with a cross-stage dynamic attention mechanism module. Specifically, the model structure is as follows: a 70-layer, 2-stage LSTM model is used as the encoder. After extracting features from historical data, the encoder uses these features as the initial state input to the decoder. The decoder contains 28 sub-modules, each consisting of a 1-layer, 2-stage LSTM model, a cross-stage dynamic dot product attention mechanism module, and a linear output layer. The output of each sub-module predicts the irradiance at the corresponding time step. Prediction between sub-modules is performed using an autoregressive approach, where the output of one sub-module serves as the data input for the next.

[0049] (1) Seq2Seq model architecture. The Seq2Seq model consists of two main components: an encoder and a decoder, both of which use the LSTM model.

[0050] The encoder is responsible for processing the input sequence. , where x t ∈R d Let d represent the input feature vector at time step t, and d represent the number of features in the input feature vector. The encoder consists of a series of recurrent units, where each recurrent unit receives the current input x at each time step t. t The hidden state h of the previous time step t-1 And generate a new hidden state h. t The encoder's operation can be represented by the following formula: h t =f enc (h t-1 ,x t c), where f enc It is an optional loop function (LSTM algorithm is used in this invention). The encoder's final hidden state is passed to the decoder as a context vector, which captures information from the entire input sequence.

[0051] Decoder generates output sequence , where y t ∈R d This represents the predicted output vector at time step t, with a sequence length of T'. The decoder is also a recurrent network; unlike the encoder, the decoder generates an output y at each step. t And the output is based on the context vector h' t-1 and the output y generated in the previous step t-1 The decoder's operation can be represented by the following formula: h' t =fdec (h' t-1 ,y t-1 ,c),y t =f out (h' t ).

[0052] (2) LSTM is a special type of recurrent neural network designed to solve the gradient vanishing and exploding problems faced by traditional recurrent neural networks (RNNs) when training on long sequences. By introducing a "gating" mechanism, LSTM can effectively capture long-term dependencies, making it particularly suitable for sequence prediction tasks.

[0053] Each LSTM cell contains three types of gates: input gate, forget gate, and output gate. The forget gate uses a sigmoid activation function to determine which information should be discarded from the cell state, which can be expressed by the following formula: f t =σ(W f ∙[h t-1 ,x t ]+b f ), where h t-1 x represents the hidden state of the LSTM unit output at the previous time step. t W is the input of the LSTM unit at the current time. f b f For the weights and biases of the learnable linear layer with forget gate, f t Information that needs to be forgotten is extracted by the forget gate.

[0054] The input gate determines which parts of the current input information will update the cell state. This can be expressed by the following formula:

[0055]

[0056] Among them, C t-1 It is the cell state output by the LSTM cell at the previous time step. It is the temporary unit state learned based on the current input. and C t-1 Combine the information to be forgotten f t and input information i t The cell state of the LSTM cell at the current moment is obtained by combining the data.

[0057] The output gate learns h again through a learnable linear layer. t-1 and x t Combined with the current unit state C t The current hidden state is obtained. It can be represented by the following formula: , where h t The hidden state representing the output of the LSTM cell at this moment, o tRepresents from h t-1 and x t The output information extracted from it.

[0058] (3) Cross-stage dynamic attention mechanism. The core idea of ​​the attention mechanism is that when processing the input sequence, the model can dynamically assign a different weight to each input element according to the needs of the current task. These weights reflect the degree of influence of different input elements on the current output.

[0059] In the CDAtt-LSTM hybrid model, an attention mechanism is added to the decoder, and the LSTM output h in each sub-module of the decoder... t This involves dynamically combining the hidden state outputs of all the encoders across stages, as well as the hidden state outputs of all previous autoregressive modules in the decoder, into a sequence. A cross-stage dynamic attention mechanism is implemented to obtain a new hidden state output h'. t , where N is the encoder length plus the length in the decoder up to time t. This can be expressed by the following formula:

[0060] ;

[0061] Among them, W q W k ,b q ,b k All are learnable parameters, score t It is a function that calculates the similarity between two vectors, typically a dot product attention function, where D is the dimension of the input vector, and α is the similarity between the two vectors. t for h t and All attention scores between, h' t This represents the hidden state output after attention weighting at time t.

[0062] (4) After the above-mentioned model components, the attention-weighted output vector set is obtained. For different stages of prediction, including the high-precision stage and the extended stage, a coupling layer allows for different emphases:

[0063] Where F1 and F2 represent two learnable linear layers, used to extract the local mutation and periodic correlation of the hidden state, respectively; α(s) is the stage weight function, designed as follows:

[0064]

[0065] In the initial high-precision stage, local abrupt changes dominate, which can improve the capture rate of high irradiance and shorten the detection time of sudden changes in irradiance. In the later extended stage, periodic correlation dominates, which can reduce daily periodic phase error and improve prediction stability.

[0066] (5) After the above-mentioned model parts, the output vector group is obtained. The vector is mapped to the target irradiance through a learnable linear output layer, thereby mapping the dimension of each vector in the vector group H'' to 1. That is, each vector is finally transformed into an irradiance scalar after matrix multiplication with the learnable parameter matrix.

[0067] Based on meteorological principles, this invention employs a CDAtt-LSTM fusion deep learning model to learn the nonlinear coupling relationship between meteorological variables and related structural features, thereby achieving ultra-short-term irradiance prediction. Furthermore, this method utilizes multivariate scale adaptive difference construction and spatiotemporally dynamically reconstructed solar altitude angle time-series features to construct new features. These features are combined with the original meteorological variable features to form a hybrid feature space that combines physical interpretability with data-driven advantages, further improving prediction accuracy. This invention obtains qualified irradiance prediction samples through data preprocessing, data reading, and sample normalization. Then, leveraging the powerful feature extraction capabilities of the proposed CDAtt-LSTM hybrid model, it achieves accurate prediction of irradiance for the next 7 hours. Each part of this invention has undergone multiple experimental verifications, and their combination effectively improves the accuracy of irradiance prediction.

Claims

1. A method for calculating ultra-short-term irradiance based on feature mining and a Seq2Seq model, characterized in that, The method comprises the following steps: Data preprocessing: cleaning historical meteorological data, removing abnormal data, generating differential features through a multivariate scale adaptive differential construction method, calculating the solar elevation angle using spatiotemporal dynamic reconstruction of the solar elevation angle time series feature, and extracting the nonlinear coupling features of the solar elevation angle and meteorological variables; The multivariate scale adaptive differential construction method comprises the following steps: using the diff function in the Pandas library to perform differential processing on the data at each time point in the multiple meteorological variables in the data set and the data n hours ago, thereby obtaining a series of differential features; and using a grid search parameter optimization method, the differential scale and differential object are subjected to special processing, so that the differential method has the characteristics of multivariate and scale adaptivity; The calculation of the solar elevation angle using spatiotemporal dynamic reconstruction of the solar elevation angle time series feature comprises the following steps: h = 15 * (t + (lon-120°) / 15°-12); δ = -22.34°*cos((N+10)*360 / 365); θ s = sin -1 (coshcosδcos(lat)-sinδsin(lat)), where h represents the hour angle of the sun, which is calculated according to the Beijing time t and the local longitude lon; lat represents the local latitude; δ represents the solar declination, which is calculated according to the Beijing date N; θ s is the calculated solar elevation angle; Data reading and sample normalization: dividing samples containing multiple consecutive time points by using a sliding window and performing batch normalization processing; CDAtt-LSTM hybrid model prediction: constructing a deep learning model based on a Seq2Seq structure, using a double-layer LSTM to process historical time data in the encoder, and using an LSTM, a cross-stage dynamic attention mechanism, and a linear output layer in the decoder; the cross-stage dynamic attention mechanism dynamically weights and fuses the encoder hidden state and the decoder historical hidden state, distinguishes the prediction emphasis of different stages through a stage weight function, and finally outputs the irradiance prediction result.

2. The method of claim 1, wherein, The nonlinear coupling features of the solar elevation angle and the meteorological variables are extracted by using the local interaction of the solar elevation angle and the meteorological variables in the sliding window, capturing the nonlinear relationship by using a learnable convolution kernel, and fusing the feature vectors after concatenation through a fully connected layer.

3. The method of claim 1, wherein, The coupling features include local exchange terms and feature fusion terms.

4. The method of claim 1, wherein, The coupling features are as follows: , where θ s (t+kΔt) represents the solar altitude angle at time t+kΔt; X n (t+kΔt) represents the values ​​of each meteorological variable at time t+kΔt, n∈{1,2,3,4}, corresponding to temperature, air pressure, humidity, and wind speed respectively; k is the offset of the convolution sliding window, with the center point at the current time t, k∈{-2,-1,0,1,2}; W n t+kΔt The learnable convolution kernel relating the nth meteorological variable to the solar altitude angle; [θ s [θ(t); X(t)] represents the vector [θ] concatenated from the solar altitude angle and meteorological variables at time t. s (t),X 1 (t),…,X 4 [(t)]; LayerNorm is the layer normalization, W1 and W2 are the weight matrices of the two fully connected layers, b1 and b2 are the bias terms of the two fully connected layers; GELU is the activation function.

5. The method of claim 1, wherein, The cross-stage dynamic attention mechanism is as follows: performing dot product attention calculation on the current hidden state of the decoder, the encoder, and the historical decoder hidden state to generate a dynamically weighted hidden state output.

6. The method of claim 1, wherein, The prediction emphasis of different stages is as follows: the high-precision stage focuses on the linear layer weight of local mutations, and the expansion stage focuses on the linear layer weight of cycle correlation.

7. The method of claim 1, wherein, The calculation of the solar elevation angle using spatiotemporal dynamic reconstruction of the solar elevation angle time series feature is as follows: based on longitude, latitude, date, and time, the solar elevation angle is reconstructed through the dynamic relationship between the solar hour angle and the declination angle, and the negative value feature is retained to enhance the physical correlation of the time series.

8. The method of claim 1, wherein, The cleaning of historical meteorological data is as follows: dividing the data into subsequences by day, and if the irradiance at the beginning and end of the subsequence is non-zero or the irradiance throughout the day is zero, the data is determined to be abnormal and is removed.

9. The method of claim 1, wherein, The difference features are generated by a multivariate scale adaptive difference construction method, specifically, for each variable in the irradiance, air temperature, air pressure, humidity, and wind speed, a difference value between the current time and the time n hours ago is calculated, and the difference scale parameter n is optimized by grid search.

10. The method of claim 1, wherein, The samples containing multiple continuous time points are divided by a sliding window, specifically, the window size is 98 continuous time points, only the time-continuous samples are retained for model training, and the sliding step is 1 time point.

Images