Ultra-short-term irradiance measuring and calculating method based on feature mining and Seq2Seq model

Abstract

The invention relates to an ultra-short-term irradiance measurement and calculation method based on feature mining and a Seq2Seq model, and the method comprises the steps: eliminating night or continuous zero irradiance abnormal data through an abnormal data cleaning rule, generating a difference feature through employing a multivariable scale adaptive difference construction method, and combining with a time-space dynamic reconstruction solar elevation angle time sequence feature, thereby obtaining the ultra-short-term irradiance measurement and calculation method. The negative value is reserved to enhance the difference between adjacent dates; utilizing a learnable convolution kernel to extract a local nonlinear relationship between the solar altitude and the meteorological variable, and fusing the splicing features through a full connection layer; a CDAtt-LSTM hybrid model is constructed, based on a Seq2Seq architecture, an encoder adopts double-layer LSTM to process historical data, a decoder fuses the historical states of the encoder and the decoder through a cross-stage dynamic attention mechanism, and a stage weighting function is introduced. According to the invention, through dynamic coupling of physical characteristics and deep learning, the accuracy and stability of irradiance prediction are significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of electronic digital data processing, and particularly relates to a method for calculating ultra-short-term irradiance based on feature mining and a Seq2Seq model. Background Art

[0002] Existing ultra-short-term irradiance prediction methods mainly include numerical prediction methods, neural network methods based on cloud images, and traditional machine learning methods. The numerical prediction method is limited by spatio-temporal resolution and has a high calculation cost, and is not suitable for ultra-short-term irradiance prediction. The neural network method based on cloud images has a high cost for obtaining historical data and is not suitable for large-scale applications. The traditional machine learning method has insufficient learning ability when dealing with high-dimensional and complex data such as meteorological data, and it is difficult to learn the internal laws of weather systems. Summary of the Invention

[0003] The present invention proposes a method for calculating ultra-short-term irradiance based on feature mining and a Seq2Seq model, which includes the following steps: Data preprocessing: cleaning historical meteorological data, generating differential features through a multi-variable scale adaptive differential construction method, calculating the solar altitude angle using the spatio-temporal dynamic reconstruction of the solar altitude angle time series feature, and extracting the non-linear coupling feature between the solar altitude angle and meteorological variables; Data reading and sample normalization: dividing samples containing consecutive multiple moments by using a sliding window and performing batch normalization processing; CDAtt-LSTM hybrid model prediction: constructing a deep learning model based on the Seq2Seq structure, using a double-layer LSTM in the encoder to process historical moment data, and the sub-module of the decoder includes an LSTM, a cross-stage dynamic attention mechanism, and a linear output layer; the cross-stage dynamic attention mechanism dynamically weights and fuses the encoder hidden state and the decoder historical hidden state, differentiates the prediction focus of different stages through a stage weight function, and finally outputs the irradiance prediction result.

[0004] Specifically, extracting the non-linear coupling feature between the solar altitude angle and meteorological variables specifically is: performing local interaction on the solar altitude angle and meteorological variables within the sliding window, capturing non-linear relationships by combining learnable convolutional kernels, and fusing the concatenated feature vectors through a fully connected layer.

[0005] Specifically, the coupling feature includes a local exchange term and a feature fusion term.

[0006] Specifically, the coupling feature is specifically: ; where θ s (t + kΔt) represents the solar altitude angle at the moment of t + kΔt; X n(t + kΔt) represents the values of each meteorological variable at the moment of t + kΔt, where n ∈ {1, 2, 3, 4}, corresponding to air temperature, air pressure, humidity, and wind speed respectively; k is the offset of the convolutional sliding window, with the center point being the current moment t, and k ∈ {-2, -1, 0, 1, 2}; W n t+kΔt is the learnable convolutional kernel associated with the solar altitude angle for the nth meteorological variable; [θ s (t); X(t)] represents the vector obtained by concatenating the solar altitude angle and meteorological variables at the moment of t ; LayerNorm is layer normalization, and W 1 、W 2 are the weight matrices of two fully connected layers, and b 1 、b 2 are the bias terms of two fully connected layers; GELU is the activation function.

[0007] Specifically, the cross-stage dynamic attention mechanism is as follows: The decoder's current hidden state is subjected to dot product attention calculation with the encoder and historical decoder hidden states to generate the output of the dynamically weighted hidden state.

[0008] Specifically, the prediction focuses at different stages are as follows: The high-precision stage focuses on the linear layer weights of local mutations, and the expansion stage focuses on the linear layer weights of periodic correlations.

[0009] Specifically, the solar altitude angle is calculated using the spatio-temporal dynamic reconstructed solar altitude angle time series features, specifically: Based on longitude, latitude, date, and time, the solar altitude angle is reconstructed through the dynamic relationship between the solar hour angle and the declination angle, and its negative value feature is retained to enhance the physical correlation of the time series.

[0010] Specifically, the historical meteorological data is cleaned, specifically: The data is divided into subsequences by day. If the irradiance at the beginning and end moments of the subsequence is non-zero, or the irradiance throughout the day is zero, it is determined as abnormal data and excluded.

[0011] Specifically, the differential features are generated through the multi-variable scale adaptive difference construction method, specifically: For each variable among irradiance, air temperature, air pressure, humidity, and wind speed, the difference value between the current moment and the moment n hours ago is calculated, and the differential scale parameter n is optimized through grid search.

[0012] Specifically, a sliding window is used to divide the samples containing multiple consecutive moments, specifically: The window size is 98 consecutive moments, and only the samples with continuous time are retained for model training, and the sliding step is 1 moment.

[0013] Beneficial technical effects: By integrating physical features with a deep learning model, the present invention significantly improves the accuracy and stability of ultra-short-term irradiance prediction. Based on the time series features of the solar altitude angle reconstructed by spatio-temporal dynamic reconstruction and the multivariate adaptive differential construction method, the non-linear coupling relationship between meteorological variables and irradiance is effectively captured. Compared with the traditional LSTM model, the mean absolute error is reduced; the cross-stage dynamic attention mechanism combined with the phased weight adjustment enables the model to adapt to the dynamic changes of different weather systems, and the root mean square error fluctuation range is narrowed in the cross-season data test. Description of the Drawings

[0014] The present invention will be further described below with reference to the accompanying drawings.

[0015] Figure 1 It is a flowchart of the ultra-short-term irradiance measurement method based on feature mining and Seq2Seq model of the present invention; Figure 2 It is a schematic diagram of the structure of the Seq2Seq model of the present invention. Detailed Embodiments

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] As Figure 1 shown, the present invention discloses an ultra-short-term irradiance measurement method based on feature mining and Seq2Seq model, which includes the following steps: 1. Data preprocessing (1) Data cleaning. During the data collection process, various errors may occur, such as input errors, measurement errors, etc. This will result in various unreasonable or incorrect data in the dataset, and the subsequent prediction performance may be affected by these errors.

[0018] First, the entire dataset sequence is divided into subsequences on a daily basis. Find all the data in the dataset where the "time" column is 00:00:00 on a certain day. Starting from the data at this moment, take 96 moments (including this moment) of data backward to form 1 subsequence, and all subsequences together form a subsequence set , where N represents how many days there are in the entire dataset sequence, and the subsequence on the i-th day , where x t i represents the data at the t-th moment on the i-th day.

[0019] For each Wi subsequence, if one of the two end data (x 1 i , x 96 i ) is not 0, this subsequence can be determined as abnormal data. If the above condition is not satisfied, check whether are all 0. If this condition is met, this subsequence can also be considered as abnormal data.

[0020] Based on the determination according to the above two conditions, the abnormal data in the entire dataset can be found, and these abnormal data can be removed to complete data cleaning.

[0021] (2) Multivariate scale adaptive difference construction method. Using the diff function in the Pandas library, perform difference processing on the data corresponding to each moment in multiple variables such as irradiance, direct irradiance, diffuse irradiance, temperature, air pressure, humidity, and wind speed in the dataset with the data n hours ago (n×4 moments ago), obtaining a series of difference features. At the same time, guided by the results, through parameter optimization methods such as grid search, make special treatment for the difference scale and difference object, making the difference method have the characteristics of multi-variable and scale adaptability.

[0022] (3) Temporal and spatial dynamic reconstruction of solar altitude angle time series features. Different from the traditional method of using the solar altitude angle as the calculation basis for a theoretical ideal irradiance, the present invention applies it as a new time feature with physical significance in the field of deep learning during deep learning training.

[0023] The calculation method of the solar altitude angle at a certain place and time (longitude lon, latitude lat, 24-hour Beijing time t, Beijing date N) is as follows: h = 15 * (t + (lon - 120°) / 15° - 12) δ = -22.34° * cos((N + 10) * 360 / 365) θ s = sin -1 (cosh cosδ cos(lat) - sinδ sin(lat)) where h represents the solar hour angle, calculated based on the 24-hour Beijing time t and the local longitude lon; δ represents the solar declination, calculated based on the Beijing date N; θ s is the calculated solar altitude angle.

[0024] Different from the use of the traditional solar altitude angle, this method utilizes the trigonometric function-like property of the solar altitude angle, retains the negative values in the calculation process of the solar altitude angle, forms an actual time feature with "tiny differences at the same time of adjacent dates, obvious differences at dates far apart, and huge differences at different times of the same date", integrates the physical method into the field of deep learning, and enables the model to have better prediction performance.

[0025] (4)Coupled feature extraction of solar altitude angle and meteorological variables. Meteorological variables such as air temperature, air pressure, humidity, and wind speed have a non-linear correlation with the attenuation coefficient of radiation with respect to the solar altitude angle. Therefore, a learnable feature coupling method can be designed to couple the solar altitude angle and meteorological variables to capture their non-linear relationship. The coupled feature extraction is achieved through the following steps: Use the solar altitude angle and meteorological variables within the sliding window for local interaction, combine the learnable convolutional kernel to capture the non-linear relationship, and fuse the concatenated feature vectors through a fully connected layer. The coupled feature C t includes a local exchange term and a feature fusion term, specifically: ; where θ s (t + kΔt) represents the solar altitude angle at time t + kΔt; X n (t + kΔt) is the value of each meteorological variable at time t + kΔt, n ∈ {1, 2, 3, 4}, corresponding to air temperature, air pressure, humidity, and wind speed respectively; k is the offset of the convolutional sliding window, with the center point being the current time t, k ∈ {-2, -1, 0, 1, 2}, that is, covering a 75-minute window; W n t+kΔt is the learnable convolutional kernel associated with the nth meteorological variable and the solar altitude angle; [θ s (t); X(t)] represents the vector obtained by concatenating the solar altitude angle and meteorological variables at time t ; LayerNorm is layer normalization, and W 1 、W 2 are the weight matrices of two fully connected layers, and b 1 、b 2 are the bias terms of two fully connected layers; GELU is the activation function.

[0026] 2. Data reading and sample normalization (1)Data reading. Since the meteorological data is saved in the form of a txt table, the read_csv function in the Pandas library is used to read the meteorological data file. After the meteorological data is read, it is stored in memory as a variable of the DataFrame type, and various operations can be performed on it through various functions in the Pandas library.

[0027] Since the goal of the prediction task is to predict the irradiance for the next 7 hours (28 time instants) based on the relevant meteorological data for the past 17.5 hours (70 time instants), the overall annual data needs to be divided into multiple samples containing 98 consecutive time instants.

[0028] The sliding window method is used to divide the samples. First step, set the window size to 98, that is, take out 98 consecutive rows of data from the DataFrame variable each time. Second step, read the Datetime column from the 98 consecutive rows of data taken out, and judge whether these data are temporally continuous data through the time information in the Datetime column. If the data are temporally continuous, then this sample can be used for model training and put into a list for use during model training, otherwise discard this sample. Third step, increment the starting row of the window for intercepting the sample by one, and repeat the above steps until the entire data set is traversed by the sliding window to obtain a list of qualified samples.

[0029] (2)Sample normalization. Use the Dataset and DataLoader functions in the torch.utils.data library to read the above sample list, and control the number of samples per batch through the batch_size parameter. After reading a batch of samples, before sending them into the model for training, normalization processing is required. We adopt a normalization method different from global normalization, that is, batch normalization, which normalizes within a single batch of data. This normalization method can effectively improve the prediction performance under extreme weather conditions. The batch normalization formula is as follows: X batch_norm =(X - min batch ) / (max batch - min batch ) Among them, X represents a certain variable of the original data, max batch , min batch are the maximum and minimum values of this variable in a single batch of data, and X batch_norm is the normalized variable. All variables in the data are normalized through the above normalization formula.

[0030] 3. LSTM (CDAtt-LSTM) hybrid model structure combined with cross-stage dynamic attention As Figure 2As shown in the figure, the deep learning model used in the present invention is a new deep learning fusion model based on the Seq2Seq structure including an encoder and a decoder. Both the encoder and the decoder adopt the long short-term memory network model (LSTM) and are combined with a cross-stage dynamic attention mechanism module. The specific model structure is as follows: An LSTM model with a length of 70 and 2 layers is used as the encoder. After the encoder extracts features from historical data, the extracted features are input as the initial state of the decoder. The decoder contains 28 sub-modules. Each sub-module is a combination of an LSTM model with a length of 1 and 2 layers, a cross-stage dot product attention mechanism module, and a linear output layer. The output of each sub-module of the decoder is the predicted irradiance at the corresponding moment. The sub-modules use an autoregressive method for prediction, that is, the output of the previous sub-module is used as the data input of the next sub-module.

[0031] (1) Seq2Seq model architecture. The Seq2Seq model has two main components: an encoder and a decoder, and both of these components use the LSTM model.

[0032] The encoder is responsible for processing the input sequence , where x t ∈R d represents the input feature vector at time step t, and d represents the number of features of the input feature vector. The encoder consists of a series of recurrent units. At each time step t, each recurrent unit receives the current input x t and the hidden state h t-1 from the previous time step, and generates a new hidden state h t . The process of the encoder working can be represented by the following formula: h t =f enc (h t-1 , x t , c), where f enc is an optional recurrent function (the LSTM algorithm is adopted in the present invention). The final hidden state of the encoder is passed to the decoder as the context vector, which captures the information of the entire input sequence.

[0033] The decoder generates the output sequence , where y t ∈R d represents the predicted output vector at time step t, and the sequence length is T'. The decoder is also a recurrent network. Different from the encoder, the decoder generates an output y t at each step, and this output is based on the context vector h' t-1 and the output y t-1 generated in the previous step. The process of the decoder working can be represented by the following formula: h' t =fdec (h' t-1 ,y t-1 ,c), y t =f out (h' t )。

[0034] (2) The LSTM is a special type of recurrent neural network designed to address the vanishing and exploding gradient problems faced by traditional recurrent neural networks (RNNs) during training on long sequences. By introducing a "gating" mechanism, the LSTM can effectively capture long-term dependencies and is particularly suitable for sequence prediction tasks.

[0035] Each LSTM cell contains three gates, namely the input gate, the forget gate, and the output gate. The forget gate determines which information should be discarded from the cell state through the sigmoid activation function and can be expressed by the following formula: f t =σ(W f ∙[h t-1 , x t +b f ), where h t-1 represents the hidden state output by the LSTM cell at the previous time step, x t is the input to the LSTM cell at the current time step, W f , b f are the weights and biases of the learnable linear layer of the forget gate, and f t is the information to be forgotten extracted by the forget gate.

[0036] The input gate determines which parts of the current input information will update the cell state. It can be expressed by the following formula: where, C t-1 is the cell state output by the LSTM cell at the previous time step, is the temporary cell state learned based on the current input and C t-1 combined with the information to be forgotten f t and the input information i t together to obtain the cell state of the LSTM cell at the current time step.

[0037] The output gate re-learns h t-1 and x t through a learnable linear layer, and combines the current cell state C t to obtain the current hidden state. It can be expressed by the following formula: , where h t represents the hidden state output by the LSTM cell at this time step, o t represents the output from h t-1 and xt The output information extracted therein.

[0038] (3) Cross-stage dynamic attention mechanism. The core idea of the attention mechanism is that when processing the input sequence, the model can dynamically assign a different weight to each input element according to the needs of the current task, and these weights reflect the influence degree of different input elements on the current output.

[0039] In the CDAtt-LSTM hybrid model, the attention mechanism is added to the decoder. The output h of the LSTM in each sub-module of the decoder t , cross-stage, is combined with all the hidden state outputs of the encoder, and the hidden state outputs generated autoregressively before this sub-module in the decoder are dynamically composed into a sequence to perform the cross-stage dynamic attention mechanism to obtain a new hidden state output h' t , where N is the length of the encoder + the length up to time t in the decoder. It can be expressed by the following formula: ; Among them, W q , W k , b q , b k are all learnable parameters, score t is a function for calculating the similarity between two vectors, generally a dot product attention function, D is the dimension number of the input vector, and α t is all the attention scores between h t and , and h' t represents the hidden state output after attention weighting at time t.

[0040] (4) After passing through each part of the above model, an attention-weighted output vector group is obtained. For different prediction stages, including the high-precision stage and the expansion stage, a coupling layer is used to make their focuses different:

[0041] Among them, F 1 , F 2 represent two learnable linear layers, which are respectively used to extract the local mutability and periodic correlation of the hidden state; α(s) is the stage weight function, designed as: In the initial high-precision stage, the local mutability dominates, which can improve the capture rate of high irradiance and shorten the detection time of sudden irradiance changes. In the later expansion stage, the periodic correlation dominates, which can reduce the daily cycle phase error and improve the prediction stability.

[0042] (5) After passing through each part of the above model, an output vector group is obtained , which is mapped to the target irradiance through a learnable output linear layer, so that the dimension of each vector in the vector group H'' is mapped to 1, that is, each vector is finally transformed into an irradiance scalar after matrix multiplication with the learnable parameter matrix.

[0043] The method of the present invention is based on the relevant principles of meteorology, uses the CDAtt-LSTM fusion deep learning model to learn the non-linear coupling relationship between meteorological variables and related structural features, so as to realize the ultra-short-term prediction of irradiance. In addition, this method also uses feature construction methods such as multi-variable scale adaptive difference construction method and temporal-spatial dynamic reconstruction of solar altitude angle time series features to construct new features, which are combined with the original meteorological variable features to form a hybrid feature space with both physical interpretability and data-driven advantages, further improving the prediction accuracy. The present invention obtains qualified irradiance prediction samples through data preprocessing, data reading, sample normalization and other links, and then uses the powerful feature extraction ability of the proposed CDAtt-LSTM hybrid model to realize the accurate prediction of irradiance in the next 7 hours. Each part of the present invention has been verified by multiple experiments, and its combination can effectively improve the accuracy of irradiance prediction.

Claims

1. A method for calculating ultra-short-term irradiance based on feature mining and Seq2Seq model, characterized in that: The following steps are involved: Data preprocessing: clean the historical meteorological data, generate differential features through multivariate scale adaptive difference construction method, calculate the solar altitude angle using the solar altitude angle time series features reconstructed dynamically in time and space, and extract the nonlinear coupling characteristics of the solar altitude angle and meteorological variables; Data reading and sample normalization: Use sliding windows to divide samples containing multiple consecutive moments and perform batch normalization. CDAtt-LSTM hybrid model prediction: Build a deep learning model based on Seq2Seq structure. The encoder uses a two-layer LSTM to process historical moment data. The decoder submodule includes LSTM, cross-stage dynamic attention mechanism and linear output layer. The cross-stage dynamic attention mechanism dynamically weights and fuses the encoder hidden state with the decoder historical hidden state, distinguishes the prediction focus of different stages through the stage weight function, and finally outputs the irradiance prediction result.

2. The method according to claim 1, characterized in that The nonlinear coupling characteristics of the solar altitude angle and meteorological variables are extracted. Specifically, the solar altitude angle in the sliding window is used to locally interact with the meteorological variables, and the learnable convolution kernel is combined to capture the nonlinear relationship, and the concatenated feature vectors are fused through the fully connected layer.

3. The method according to claim 1, characterized in that The coupling features include local exchange terms and feature fusion terms.

4. The method according to claim 1, characterized in that The coupling characteristics are as follows: where θ s (t+kΔt) represents the solar altitude angle at time t+kΔt; X n (t+kΔt) is the value of each meteorological variable at time t+kΔt, n∈{1,2,3,4}, corresponding to temperature, pressure, humidity, and wind speed respectively; k is the convolution sliding window offset, the center point is the current time t, k∈{-2,-1,0,1,2}; W n t+kΔt is the learnable convolution kernel that associates the nth meteorological variable with the solar altitude angle; [θ s (t); X(t)] represents the vector of the solar altitude angle and meteorological variables at time t ;LayerNorm is layer normalization, W1 and W2 are the weight matrices of the two fully connected layers, b1 and b2 are the bias items of the two fully connected layers; GELU is the activation function.

5. The method according to claim 1, characterized in that The cross-stage dynamic attention mechanism is as follows: the current hidden state of the decoder is subjected to dot product attention calculation with the hidden state of the encoder and the historical decoder to generate a dynamically weighted hidden state output.

6. The method according to claim 1, characterized in that The prediction focuses at different stages are as follows: the high-precision stage focuses on the linear layer weights of local mutations, and the expansion stage focuses on the linear layer weights of periodic correlations.

7. The method according to claim 1, characterized in that The solar altitude angle is calculated using the time series characteristics of the solar altitude angle reconstructed dynamically in space and time. Specifically, based on longitude, latitude, date and time, the solar altitude angle is reconstructed through the dynamic relationship between the solar hour angle and the declination angle, and its negative value characteristics are retained to enhance the physical correlation of the time series.

8. The method according to claim 1, characterized in that The historical meteorological data are cleaned, specifically: the data are divided into subsequences by day. If the irradiance at the beginning and end of the subsequence is non-zero, or the irradiance for the whole day is zero, it is judged as abnormal data and removed.

9. The method according to claim 1, characterized in that The differential features are generated through a multivariate scale adaptive difference construction method. Specifically, for each variable in irradiance, temperature, air pressure, humidity, and wind speed, the difference value between the current moment and the moment n hours ago is calculated, and the differential scale parameter n is optimized through grid search.

10. The method according to claim 1, characterized in that A sliding window is used to divide samples containing multiple consecutive moments. Specifically, the window size is 98 consecutive moments, only samples with continuous time are retained for model training, and the sliding step size is 1 moment.

Images