Solar radiation prediction method based on RMN-GLUformer

Abstract

The invention discloses an RMN-GLUformer-based solar radiation prediction method, which learns potential features of data through an RMN preprocessing module and removes noise, thereby realizing effective noise reduction of the data, pays attention to each position in a sequence through a self-attention mechanism, captures a global dependency relationship between the positions, and finally determines a solar radiation prediction result. And the understanding of the prediction model on the context information is enhanced, so that the prediction accuracy is improved. In addition, GLU dynamic control information flow is adopted, the nonlinear modeling capability of the prediction model is enhanced, and the problems of gradient disappearance and explosion are avoided. According to the prediction method, verification and evaluation are carried out on a real data set of a part of regions of Beijing, Tianjin and Hebei, a comparison test is carried out based on the real data set and the five baseline models, and a result shows that the prediction method has smaller prediction errors and higher generalization ability.

Description

Technical Field

[0001] The present invention relates to the field of solar energy technology, and in particular to a solar radiation prediction method based on RMN-GLUformer. Background Art

[0002] With accelerating urbanization and a growing population, the building industry has become a significant contributor to global energy consumption, accounting for approximately 40% of global energy consumption. Heating systems in buildings are often the primary component of building energy consumption, with energy demand particularly high during colder months. Currently, coal and natural gas are the primary heating sources, resulting in significant carbon emissions and environmental pollution. Therefore, promoting sustainable, renewable energy alternatives has become a global consensus. Solar energy, as a clean, pollution-free, renewable energy source, has attracted significant attention.

[0003] However, solar radiation is highly nonlinear, intermittent, randomly fluctuating, and uncertain, influenced by factors such as geographic location, seasonal variations, and weather conditions, posing significant challenges to its accurate prediction. Accurate solar radiation forecasting can effectively reduce this uncertainty, providing reliable data support for the optimized design and operation of solar co-heating systems, thereby improving energy efficiency and promoting the widespread adoption of clean energy.

[0004] Currently, researchers' methods for predicting solar radiation are primarily categorized into four categories: physical modeling, statistical modeling, machine learning, and deep learning. Physical modeling, based on physical equations and meteorological principles, creates prediction models by simulating the effects of various factors on solar radiation, such as the atmosphere, clouds, and topography. Statistical modeling, on the other hand, relies on historical data and observations, analyzing the changing patterns of solar radiation through statistical models and predicting future irradiance accordingly. Machine learning, through training algorithmic models, automatically learns complex patterns and relationships from large amounts of data, enabling accurate predictions and possessing strong adaptive capabilities.

[0005] In recent years, deep learning methods have gradually become the mainstream in solar radiation forecasting. Unlike traditional machine learning methods, deep learning can automatically extract high-level features from data without the need for manual feature design, making it particularly suitable for processing large-scale, complex time series data. Common deep learning models such as convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and self-attention mechanisms can effectively capture the spatiotemporal characteristics and long-term dependencies in the data, significantly improving the accuracy and robustness of forecasts. Deep learning methods, supported by big data, have demonstrated significant advantages, particularly in environments with dynamic meteorological changes and nonlinear complexity, and have promoted the development of solar radiation forecasting technology.

[0006] However, existing models often struggle to effectively capture nonlinear relationships and deep features in high-dimensional, complex data. Furthermore, these models often require significant computational resources and are prone to vanishing or exploding gradients, resulting in inefficient training. Summary of the Invention

[0007] To address the limitations of the existing technology, the present invention proposes a solar radiation prediction method based on RMN-GLUformer. This prediction method uses RMN for denoising in the data preprocessing stage. RMN is an innovative technology that combines residual networks and sliding smoothing methods. It can effectively suppress observation noise and process noise, improve the signal-to-noise ratio of the data, and provide cleaner and more accurate data input for subsequent prediction models. The data is then input into the prediction model, and multi-dimensional information from different subspaces is captured in parallel through a multi-head self-attention mechanism, enabling the prediction model to better understand the complex relationships in time series data. At the same time, the GLU mechanism is introduced to dynamically control the information flow, solving the gradient vanishing and explosion problems faced by traditional models during training.

[0008] The technical solution of the present invention to solve the technical problem is to design a solar radiation prediction method based on RMN-GLUformer, characterized in that the method comprises the following steps:

[0009] Step S1: Obtain historical solar radiation and meteorological data for a target area for no less than one year, wherein the historical solar radiation data is total radiation data, and the historical meteorological data is temperature, dew point temperature, relative humidity, surface albedo, wind speed, air pressure, precipitation, wind direction, and aerosol optical depth; collect the historical solar radiation data and historical meteorological data at a fixed time resolution ΔT to obtain an original data set; then perform quality control and cleaning on the original data set to obtain a training data set; then split the training data set in chronological order, using no less than 60% of the data as a training set and the remaining data as a validation set;

[0010] Step S2: Establish RMN preprocessing module

[0011] The RMN preprocessing module consists of three residual blocks, a fully connected output layer, and a smoothing operation. Each residual block consists of two fully connected layers and a ReLU activation function. For a residual block, the input x is first transformed by the first fully connected layer (fc1), and then nonlinearly transformed by the ReLU activation function, outputting the intermediate result z1.

[0012] z1=ReLU(fc1(x))

[0013] Then, it is transformed through the second fully connected layer (fc2) to obtain the output z2:

[0014] z2=fc2(z1)

[0015] Finally, the output z2 of the second fully connected layer (fc2) is added to the original input x to form the final output:

[0016] y=z²+x

[0017] The output y of the residual block is:

[0018] y=F(x)+x

[0019] Where F(x)=fc2(ReLU(fc1(x))) is the residual term after a series of transformations;

[0020] For the RMN preprocessing module, assume that the input data x0 passes through the first residual block and obtains the output x1:

[0021] x1=F1(x0)+x0

[0022] Then, x1 is used as the input of the second residual block to obtain x2:

[0023] x2=F2(x1)+x1

[0024] Finally, after the third residual block, we get x3:

[0025] x3=F3(x2)+x2

[0026] The output after three residual blocks is:

[0027] x3=F3(F2(F1(x0)+x0)+F1(x0)+x0)+F2(F1(x0)+x0)+F1(x0)+x0

[0028] F1(·), F2(·), and F3(·) represent the operations processed by the first residual block, the second residual block, and the third residual block, respectively;

[0029] Then x3 is processed by the fully connected output layer to obtain the denoised data, and then the denoised signal is smoothed using a smoothing operation of k sliding windows to obtain the output data of the RMN preprocessing module. This output signal is the input of the GLUformer model;

[0030] Step S3: Establish a prediction model based on GLUformer

[0031] The GLUformer-based prediction model includes an embedding layer, multiple encoders, and a fully connected layer. The data input to the prediction model is first processed by the embedding layer to form multidimensional sequence data. The time steps of the multidimensional sequence data are then positionally encoded and then input to the encoder. The multiple encoders are connected in sequence, with the output of the previous encoder serving as the input of the next encoder. The output of the last encoder is then processed by the fully connected layer to obtain the output of the prediction model.

[0032] The input of the prediction model is the output of the RMN preprocessing module; each encoder includes a multi-head attention module, a first residual connection and layer normalization operation, a GLU module and a second residual connection and layer normalization operation; the data P0 input to the encoder is first processed by the multi-head attention module to obtain data P1; the result obtained by performing a residual connection on data P1 and data P0 is then subjected to a layer normalization operation to obtain data P2; data P2 is input to the GLU module for processing to obtain data P3; the result obtained by performing a residual connection on data P3 and data P2 is then subjected to a layer normalization operation to obtain data P4, which is the output of the encoder;

[0033] The encoder's multi-head attention module first projects the input into multiple subspaces, then calculates the attention weight of each subspace, and finally combines the representations of multiple subspaces to obtain the final output;

[0034] In the implementation, the multi-head attention module maps the input data Z0 into Q, K, and V through three different linear transformations, and then splits Q, K, and V into h parts respectively, where h is the number of attention heads. The specific operation is as follows:

[0035]

[0036] in, is the projection matrix of the i-th subspace; then the self-attention output is calculated for each subspace, where the self-attention output head of the i-th subspace is i for:

[0037]

[0038] In the above formula, is the scaling factor, d k K i Dimension value of ;

[0039] Then the outputs of all subspaces are concatenated and the final output is obtained through linear transformation:

[0040] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O

[0041] Where W O It is a projection matrix used for linear transformation to enhance the expressiveness of data;

[0042] Then the output MultiHead(Q,K,V) is subjected to the first residual connection and layer normalization operation:

[0043] Z r =LayerNorm(Z0+MultiHead(Q,K,V))

[0044] Where Z0 represents the input of the multi-head attention module, and MultiHead(Q,K,V) represents the output of the multi-head attention module;

[0045] Then the output Z obtained after the first residual connection and layer normalization operation is r Input into the GLU module. The principle formula of the GLU module is as follows:

[0046] GLU(Z r )=(Q1Z r +b1)⊙σ(W2Z r +b2)

[0047] Among them, W1 and W2 are two different linear transformation matrices, b1 and b2 are corresponding bias terms, σ is the GELU activation function, and ⊙ represents the Hadamard product, that is, element-by-element multiplication;

[0048] The output of the GLU module is connected to the second residual and normalized layer to obtain the output Z of the encoder. p :

[0049] Z p =LayerNorm(Z r +GLU(Z r ))

[0050] The output of the previous encoder is used as the input of the next encoder, and the output of the last encoder is sent to a fully connected layer for linear transformation to obtain the final prediction result;

[0051] Step S4: Training the RMN preprocessing module

[0052] First, the random assignment method is used to initialize the network parameters in the RMN preprocessing module. Then, a fixed-size sliding window k is used to slice the training set and validation set data in step S1. Each slice is a data sample. A batch of data samples in the training set is sequentially input into the RMN preprocessing module. A data sample X i After being processed by the RMN preprocessing module, the output is denoised and then smoothed to obtain Data sample X i After the same smoothing process, we can get Y i ; Y i As the true value, As the predicted value, the training loss of a batch of training sets uses the mean square error between the true value and the predicted value of each feature dimension data of all data samples;

[0053] According to the training loss of a batch of the training set, the Adam optimizer with adaptive learning rate is used for back propagation to update the network parameters of the RMN preprocessing module once, completing the training of a batch of samples; then the data samples of the next batch in the training set are input, and the network parameters when the previous batch of samples is trained are used as the initial parameters when the next batch of samples is trained. The training process of a batch of samples is repeated continuously until the last batch of samples in the training set is trained, completing a round of training; the network parameters when the previous round of training is completed are used as the initial parameters when the next round of training is trained, and the process of a round of training is repeated continuously; after each round of training, the data samples in the validation set are sequentially input into the RMN preprocessing module of the current parameters, and the training loss of the data samples in the validation set is calculated; when the training loss of the data samples in the validation set decreases by less than the preset value for three consecutive rounds or reaches the preset upper limit of the iteration round, the training of the RMN preprocessing module is completed;

[0054] Step S5: Training the GLUformer-based prediction model

[0055] Step S5.1: Slice the original training set and validation set data in step S1 using a fixed-size sliding window k. Then, process each of the slices through the RMN preprocessing module pre-trained in step S4 to obtain a temporally continuous preprocessed training set and preprocessed validation set. Then, calculate the mean and standard deviation of each feature dimension of the data in the preprocessed training set, and standardize each feature dimension of the preprocessed training set and preprocessed validation set according to the mean and standard deviation of each feature dimension to obtain a standardized training set and validation set.

[0056] A sliding window of T+t time steps is used to slice the data of the standardized training set and validation set respectively to generate the training sample set and validation sample set;

[0057] Step S5.2 uses the Xavier initialization method to initialize the weight matrix in the prediction model to ensure that the variance of the input of each layer remains consistent. At the same time, all bias terms are initialized to zero and the hyperparameters of the prediction model are set. For the positional encoding in the GLUformer module, sine and cosine functions are used for initialization.

[0058] Step S5.3: A training sample in the training sample set includes historical meteorological data W and corresponding historical solar radiation data H, where W = {w1, w2, ..., w T}, is the input of the prediction model; H = {h T+1 ,h T+2 ,...,h T+t}, which is the reference value of the output of the prediction model; the historical meteorological data W of a training sample is output to the initialized prediction model, and the output of the prediction model is the predicted solar radiation related data in is the predicted value of the prediction model for the historical solar radiation related data at time step T+t; the mean square error between the reference value output by the prediction model of a training sample and the predicted solar radiation related data output by the prediction model is calculated, and then the result is added to the L2 regularization term. The result is used as the training loss. The calculation formula of the loss function is:

[0059]

[0060] in, is the mean square error between the reference value of the prediction model output of the training sample and the predicted solar radiation related data output by the prediction model, λ is the regularization coefficient, θ i is the i-th parameter of the prediction model, M is the total number of parameters of the prediction model;

[0061] According to the training loss value of a training sample, the Adam optimizer is used to update the prediction model parameters once to complete the training of a training sample; then the next training sample is input, and the network parameters when the previous training sample is completed are used as the initial parameters for the next training sample training, and the training process of a training sample is repeated until the last sample in the training sample set is trained, completing a round of training; the network parameters when the previous round of training is completed are used as the initial parameters for the next round of training, and the training process of a round is repeated continuously; after each round of training, all training samples in the validation sample set are input in turn, and the mean of the training loss of all training samples in the validation sample set is calculated. When the mean of the training loss does not decrease for 5 consecutive rounds or the training round reaches the maximum preset value, the prediction model training is completed, and the parameter values of the prediction model with the smallest training loss of the validation sample set are saved to obtain the optimal prediction model;

[0062] Step S6: Solar radiation prediction

[0063] Predict the solar radiation data of the target area at time point t after time point P in step S1; first obtain the solar radiation data of time point P and the solar radiation data after time point P. time points and the time points before P The meteorological data of k+T time points with a time interval of ΔT are first subjected to quality control and cleaning as described in step S1, and the obtained results are input into the RMN preprocessing module that completes the training in step S4 to obtain a data sequence of T time steps; then the mean and standard deviation of each feature dimension of the data in the preprocessing training set in step S5 are selected, and the same standardization operation is performed on the data sequence, and the obtained results are input into the optimal prediction model in step S5, which outputs solar radiation-related data at t time points after time point P; the parameters contained in the meteorological data are the same as those of the historical meteorological data in step S1.

[0064] Compared with the prior art, the beneficial effects of the present invention are as follows: the solar radiation prediction method based on RMN-GLUformer of the present invention learns the potential features of the data and removes noise through the RMN preprocessing module, thereby achieving effective noise reduction of the data, and then focuses on each position in the sequence through the self-attention mechanism, and captures the global dependencies between positions, thereby enhancing the prediction model's understanding of contextual information, thereby improving prediction accuracy. In addition, GLU is used to dynamically control the information flow, enhance the nonlinear modeling capability of the prediction model, and avoid the problems of gradient disappearance and explosion. The prediction method of the present invention was verified and evaluated on real data sets in parts of the Beijing-Tianjin-Hebei region, and was compared with 5 baseline models based on real data sets. The results show that the prediction method of the present invention has smaller prediction errors and stronger generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a schematic diagram of the structure and principle of the RMN preprocessing module of an embodiment of a solar radiation prediction method based on RMN-GLUformer of the present invention.

[0066] Figure 2 This is a schematic diagram of the structure and principle of a GLUformer-based prediction model in an embodiment of a solar radiation prediction method based on RMN-GLUformer of the present invention.

[0067] Figure 3 This is a schematic diagram of the principle of a multi-head attention module based on a GLUformer prediction model in an embodiment of a solar radiation prediction method based on RMN-GLUformer of the present invention.

[0068] Figure 4 This is a comparison chart of the future 1-hour prediction results (RMN-GLUformer) and the true value (TRUE) of the solar radiation time series using the solar radiation prediction method based on RMN-GLUformer of the present invention.

[0069] Figure 5 This is a comparison chart of the prediction results of the solar radiation time series for the next hour using the solar radiation prediction method based on RMN-GLUformer of the present invention and the other five methods and the true value (TRUE).

[0070] Figure 6 This is a diagram showing the evaluation results of the solar radiation time series prediction results using the solar radiation prediction method based on RMN-GLUformer of the present invention and five other methods. DETAILED DESCRIPTION

[0071] To facilitate understanding of this application, the following drawings and a comprehensive description of this application are provided. The drawings illustrate preferred embodiments of this application. However, the present invention can be implemented in various ways and is not limited to the embodiments shown herein.

[0072] The present invention provides a solar radiation prediction method based on RMN-GLUformer, which comprises the following steps:

[0073] Step S1: Obtain at least one year of historical solar radiation and meteorological data for the target area. The historical solar radiation data includes global irradiance (GHI) data, and the historical meteorological data includes temperature (T), dew point temperature (DP), relative humidity (RH), surface albedo (SA), wind speed (WS), air pressure (P), precipitation (PW), wind direction (WD), and aerosol optical depth (AOD). Collect the historical solar radiation and meteorological data at a fixed temporal resolution ΔT (i.e., a fixed time interval) to obtain a raw dataset. The raw dataset is then quality-controlled and cleaned to obtain a training dataset. The quality control and cleaning steps include: directly removing outliers and filling in missing values.

[0074] For single missing value problems, linear interpolation is used to fill the missing values. For long-term data missing, spatial interpolation of neighboring sites is used to fill the missing values.

[0075] The training dataset is then split in chronological order, with the first 80% of the data used as the training set for model training and parameter optimization; the remaining 20% of the data is used as the validation set to evaluate the model's generalization ability and prediction performance;

[0076] As an example, solar radiation and related meteorological data from 22 observation stations in the Beijing-Tianjin-Hebei region from March 21, 2019 to March 20, 2020 were obtained through the National Solar Radiation Database (NSRDB). The data collection time resolution was 10 minutes, and a total of 12 key meteorological parameters were included: direct radiation (DNI), diffuse radiation (DHI), global radiation (GHI), temperature (T), dew point temperature (DP), relative humidity (RH), surface albedo (SA), wind speed (WS), air pressure (P), precipitation (PW), wind direction (WD), and aerosol optical depth (AOD).

[0077] Step S2: Establish RMN preprocessing module

[0078] The RMN preprocessing module consists of three residual blocks, a fully connected output layer and a smoothing operation, see Figure 1 , where each residual block consists of two fully connected layers (Weight layer) and a ReLU activation function; for a residual block, the input x is first transformed by the first fully connected layer (fc1), and then nonlinearly transformed by the ReLU activation function to output the intermediate result z1.

[0079] z1=ReLU(fc1(x))

[0080] Then, it is transformed through the second fully connected layer (fc2) to obtain the output z2:

[0081] z2=fc2(z1)

[0082] Finally, the output z2 of the second fully connected layer (fc2) is added to the original input x to form the final output:

[0083] y=z²+x

[0084] The output y of the residual block is:

[0085] y=F(x)+x

[0086] Where F(x)=fc2(ReLU(fc1(x))) is the residual term after a series of transformations.

[0087] For the RMN preprocessing module, assume that the input data x0 passes through the first residual block and obtains the output x1:

[0088] x1=F1(x0)+x0

[0089] Then, x1 is used as the input of the second residual block to obtain x2:

[0090] x2=F2(x1)+x1

[0091] Finally, after the third residual block, we get x3:

[0092] x3=F3(x2)+x2

[0093] The output after three residual blocks is:

[0094] x3=F3(F2(F1(x0)+x0)+F1(x0)+x0)+F2(F1(x0)+x0)+F1(x0)+x0

[0095] F1(·), F2(·), and F3(·) represent operations processed by the first residual block, the second residual block, and the third residual block, respectively.

[0096] Then x3 is processed by the fully connected output layer to obtain the denoised data, and then the denoised signal is smoothed using a smoothing operation (MovingAverage) of k sliding windows to obtain the output data of the RMN preprocessing module. This output signal is the input of the GLUformer model.

[0097]

[0098] Among them, x denoise(i) is the value of the denoised data at time point i; k is the size of the sliding window, which is an odd number and determines the range of smoothing. The smoothing operation of the sliding window size of k is to take the mean of the values of the k time points before, after and at time point t as the result of the smoothing operation at time point t; x res (t) is the result obtained by smoothing the value at time point t.

[0099] Step S3: Establish a prediction model based on GLUformer

[0100] The GLUformer-based prediction model includes an embedding layer, multiple encoders, and a fully connected layer. Figure 2 The data input to the prediction model is first processed by the embedding layer to form multidimensional sequence data. The time steps of this multidimensional sequence data are then positionally encoded and input to the encoder. Multiple encoders are connected in sequence, with the output of the previous encoder serving as the input to the next encoder. The output of the last encoder is then processed by the fully connected layer to obtain the output of the prediction model.

[0101] The input of the prediction model is the output of the RMN preprocessing module; assuming that the input of the prediction model is the output of the RMN preprocessing module of n time steps, after processing by the embedding layer, the sequence X is obtained. res =[x res (1),x res (2),…,x res (t),…,x res (n)],X res ∈R n×d , n is the sequence length, d is the feature dimension. In order to make the prediction model aware of the order of elements in the sequence, position encoding is introduced:

[0102] Z0=X res +PE

[0103] Among them, PE is the position code, and Z0 is the input of the first encoder.

[0104] The commonly used position encoding form is based on the sine and cosine functions:

[0105]

[0106] Where pos is the position and i is the dimension index.

[0107] Each encoder includes a multi-head attention module, a first residual connection and layer normalization operation, a GLU module and a second residual connection and layer normalization operation; the data P0 input to the encoder is first processed by the multi-head attention module to obtain data P1; the result obtained after residual connection of data P1 and data P0 is then subjected to layer normalization operation to obtain data P2; data P2 is input to the GLU module for processing to obtain data P3; the result obtained after residual connection of data P3 and data P2 is then subjected to layer normalization operation to obtain data P4, which is the output of the encoder.

[0108] The encoder's multi-head attention module is composed of multiple self-attention modules. Its core mechanism is to map input features into multiple independent subspaces for parallel processing. Figure 3 Specifically, the multi-head attention module first projects the input into multiple different subspaces, then calculates the attention weight of each subspace, and finally combines the representations of multiple subspaces to obtain the final output.

[0109] In the implementation process, the multi-head attention module maps the input data Z0 into Q (Query), K (Key), and V (Value) through three different linear transformations, and then splits Q, K, and V into h parts (h is the number of attention heads). The specific operations are as follows:

[0110]

[0111] in, is the projection matrix of the i-th subspace. Then the self-attention output is calculated for each subspace, where the self-attention output head of the i-th subspace is i for:

[0112]

[0113] In the above formula, is the scaling factor, d k K i Dimension value of .

[0114] Then the outputs of all subspaces are concatenated and the final output is obtained through linear transformation:

[0115] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O

[0116] Where W O It is a projection matrix used for linear transformation to enhance the expressiveness of data.

[0117] Then the output MultiHead(Q,K,V) is subjected to the first residual connection and layer normalization operation:

[0118] Z r =LayerNorm(Z0+MultiHead(Q,K,V))

[0119] Where Z0 represents the input of the multi-head attention module, and MultiHead(Q,K,V) represents the output of the multi-head attention module.

[0120] Then the output Z obtained after the first residual connection and layer normalization operation is r Input to the GLU module. The GLU module performs two operations on the input. The first operation is to calculate the gate signal, and the other operation is to calculate the linear transformation. The results of the two operations are then combined by element-by-element multiplication. The principle formula of the GLU module is as follows:

[0121] GLU(Z r )=(W1Z r +b1)⊙σ(W2Z r +b2)

[0122] Among them, W1 and W2 are two different linear transformation matrices, b1 and b2 are corresponding bias terms, σ is the GELU activation function, and ⊙ represents the Hadamard product, that is, element-by-element multiplication. 2 are trainable parameters.

[0123] GLU dynamically controls the flow of information based on the input by introducing a gating mechanism and two linear transformations. Essentially, the input data is processed in two ways: one undergoes a linear transformation, the other undergoes a linear transformation, and then passes the GELU activation function through the other, before being element-wise multiplied with the result of the first. This design allows the model to selectively pass information, avoiding excessive information attenuation or explosion.

[0124] The output of the GLU module is connected to the second residual and normalized layer to obtain the output Z of the encoder. p :

[0125] Z p =LayerNorm(Z r +GLU(Z r ))

[0126] The output of the previous encoder is used as the input of the next encoder, and the output of the last encoder is sent to a fully connected layer for linear transformation to obtain the final prediction result.

[0127] Step S4: Training the RMN preprocessing module

[0128] First, the random assignment method is used to initialize the network parameters in the RMN preprocessing module. Then, a fixed-size sliding window k is used to slice the training set and validation set data in step S1. Each slice is a data sample. A batch of data samples in the training set is sequentially input into the RMN preprocessing module. A data sample X i After being processed by the RMN preprocessing module, the output is denoised and then smoothed to obtain Data sample X i After the same smoothing process, we can get Y i ; Y i As the true value, As the prediction value, the training loss of a batch of training sets adopts the mean of the mean squared error (MSE) between the true value of each feature dimension data of all data samples and the predicted value.

[0129] According to the training loss of a batch of training sets, the Adam optimizer with adaptive learning rate is used for back propagation to update the network parameters of the RMN preprocessing module once to complete the training of a batch of samples; then the data samples of the next batch in the training set are input, and the network parameters when the previous batch of samples is trained are used as the initial parameters when the next batch of samples is trained. The training process of a batch of samples is repeated continuously until the last batch of samples in the training set is trained and a round of training is completed; the network parameters when the previous round of training is completed are used as the initial parameters when the next round of training is trained, and the process of a round of training is repeated continuously; after each round of training, the data samples in the validation set are input into the RMN preprocessing module of the current parameters in turn, and the training loss of the data samples in the validation set is calculated. When the training loss of the data samples in the validation set decreases by less than 1e for three consecutive rounds -4 Or when the preset upper limit of 200 iterations is reached, the training of the RMN preprocessing module is completed.

[0130] Step S5: Training the GLUformer-based prediction model

[0131] Step S5.1 slices the original training set and validation set data in step S1 using a fixed-size sliding window k, and then processes them through the RMN preprocessing module pre-trained in step S4 to obtain a time-series continuous preprocessed training set and preprocessed validation set; then calculate the mean and standard deviation of each feature dimension of the data in the preprocessed training set, and perform z-score normalization on each feature dimension of the preprocessed training set and preprocessed validation set according to the mean and standard deviation of each feature dimension to obtain a standardized training set and validation set.

[0132] A sliding window of T+t time steps is used to slice the data of the standardized training set and validation set, generating training sample sets and validation sample sets respectively. During the sliding slicing process, overlapping areas of adjacent windows are allowed to expand the sample size.

[0133] Step S5.2 uses the Xavier initialization method to initialize the weight matrix in the prediction model to ensure that the variance of the input of each layer remains consistent, thereby accelerating convergence. At the same time, all bias terms are initialized to zero and the hyperparameters of the prediction model are set. For the position encoding in the GLUformer module, sine and cosine functions are used for initialization to capture the position information in the sequence. Among the hyperparameters, dropout is set to 0.1, batch size is set to 32, the number of encoders in the GLUformer module is set to 3, and the number of attention heads in the multi-head attention module is set to 8.

[0134] Step S5.3: A training sample in the training sample set includes historical meteorological data W and corresponding historical solar radiation data H, where W = {w1, w2, ..., w T}, is the input of the prediction model; H = {h T+1 ,h T+2 ,...,h T+t}, which is the reference value of the output of the prediction model; the historical meteorological data W of a training sample is output to the initialized prediction model, and the output of the prediction model is the predicted solar radiation related data in is the predicted value of the prediction model for the historical solar radiation related data at time step T+t. Calculate the mean square error (MSE) between the reference value output by the prediction model of a training sample and the predicted solar radiation related data output by the prediction model, and then add it to the L2 regularization term to obtain the result as the training loss. The calculation formula of the loss function is:

[0135]

[0136] in, is the mean square error between the reference value of the prediction model output of the training sample and the predicted solar radiation related data output by the prediction model, λ is the regularization coefficient, θ i is the i-th parameter of the prediction model, and M is the total number of parameters of the prediction model.

[0137] Based on the training loss value of a training sample, the Adam optimizer is used to update the prediction model parameters once, completing the training of one training sample. Then, the next training sample is input, and the network parameters at the completion of the previous training sample are used as the initial parameters for the next training sample. The training process for one training sample is repeated until the last sample in the training sample set is trained, completing one round of training. The network parameters at the completion of the previous round of training are used as the initial parameters for the next round of training, and the training process is repeated one round at a time. After each round of training, all training samples in the validation sample set are input in sequence, and the mean training loss of all training samples in the validation sample set is calculated. When the mean training loss does not decrease for five consecutive rounds or the training round reaches the maximum preset value, the prediction model training is completed. The parameter values of the prediction model with the minimum training loss for the validation sample set are saved to obtain the optimal prediction model.

[0138] Step S6: Solar radiation prediction

[0139] Predict the solar radiation data at t time points (total duration is ΔT×t) after the time point P in the target area in step S1. First, obtain the time point P and the data after the time point P. time points and the time points before P The meteorological data of k+T time points with a time interval of ΔT are first subjected to quality control and cleaning as described in step S1, and the obtained results are input into the RMN preprocessing module trained in step S4 to obtain a data sequence of T time steps; then the mean and standard deviation of each feature dimension of the data in the preprocessing training set in step S5 are selected, and the same z-score normalization operation is performed on the data sequence, and the obtained results are input into the optimal prediction model in step S5, which outputs solar radiation-related data at t time points after time point P (with a total duration of ΔT×t).

[0140] The parameters included in the meteorological data are the same as those of the historical meteorological data in step S1. The meteorological data at each time point can be obtained by prediction using existing technical methods. The specific prediction method is not within the scope of protection of the claims of this invention, and since only The meteorological data at each time point is the predicted value, while the rest are real historical data, so their impact on the final target result can be ignored. The predicted solar radiation data can be used to consider the intensity of solar radiation. Alternatively, it can be denormalized using the mean and standard deviation of the historical solar radiation data in the preprocessed training set in step S5.1 to obtain the predicted value of solar radiation data.

[0141] Example 1

[0142] The solar radiation dataset used in the present invention comes from the National Solar Radiation Database (NSRDB), and the solar radiation data of 22 regions in Beijing, Tianjin and Hebei from March 21, 2019 to March 20, 2020 are selected, with a total of 26,352 data. In order to better understand the characteristics of the data, Table 1 shows the detailed statistical information of the dataset. It can be seen that the solar radiation (Solar Radiation) data in spring and summer show a larger standard deviation, stronger volatility, and higher instability than those in autumn and winter. In the present invention, in order to evaluate the performance and effectiveness of the model, each dataset is divided into two subsets: a training dataset and a test dataset. Specifically, the first 80% of the data is used for model training and parameter tuning, and the remaining 20% is used for model verification and performance evaluation. This division method ensures that the model can be fully trained on different data subsets and effectively evaluated on the test set, thereby improving the generalization ability of the model.

[0143] Table 1 Basic statistical information of the four seasonal datasets

[0144]

[0145] In deep learning models, hyperparameters play an important role in model performance and efficiency. In this paper, we focused on several key model hyperparameters, as shown in Table 2. To optimize these hyperparameters, we employed a random search tuning method and experimentally determined the optimal value for each hyperparameter.

[0146] Table 2 Optimal parameters of RMN-GLUformer

[0147] parameter value parameter value dropout 0.01 dmodel 512 batchsize 32 Learning rate 0.01 numlayers 3 Activation Function Gelu head 8 Hidden layer size 128

[0148] Solar radiation data from 22 regions in Beijing, Tianjin and Hebei were used to predict the future with a time step of 6 (1 hour in the future). In order to evaluate the performance of the model, this study selected three commonly used performance indicators, namely root mean square error (RMSE), mean absolute error (MAE) and coefficient of determination (R 2 ), the results are shown in Table 3.

[0149]

[0150] Among them, y i is the true value of solar radiation, is the predicted value of solar radiation, and n is the total number of samples. 2 The range is (-∞, 1). The closer the exponent is to 1, the better the model performance is.

[0151] It can be observed from the table that the R 2 All three showed the best prediction performance in the winter dataset, while RMSE and MAE performed best on the full-year dataset. All three showed the worst prediction performance in the spring dataset. This suggests that seasonal differences in prediction performance may be affected by external factors such as the growth status of vegetation. Vegetation is in its growth phase in spring, and its leaves absorb solar radiation more strongly and reflect less sunlight. A large portion of solar radiation is absorbed by vegetation and does not reach the ground surface. In contrast, plants begin to wither in autumn and winter, and their leaves gradually turn yellow, making it easier for solar radiation to reach the ground directly.

[0152] See also Figure 4 、 Figure 5 As shown, Figure 4 A comparison chart of the solar radiation prediction method based on RMN-GLUformer for the solar radiation time series for the next hour and the true value; Figure 5 This is a comparison chart of the prediction results and true values of the solar radiation time series for the next 1 hour by the present invention and other comparison algorithms.

[0153] Table 3 Comparison of prediction results of different seasonal datasets

[0154]

[0155] See also Figure 6 The figure shows a mixed chart of various evaluation indicators of the prediction results of solar radiation time series of the present invention. Taking the full-year data set as an example, the MAE of the proposed model is reduced by 21.1%, 20.62%, 10.17%, 8.69%, and 24.05% compared with Linear, DLinear, Informer, Autoformer, and LSTM, respectively. The RMSE is reduced by 27.29%, 26.85%, 11.6%, 11.11%, and 27.91%, and R 2 They increased by 0.14, 0.14, 0.03, 0.05 and 0.08 respectively.

[0156] Experimental results show that, as a linear method, the Linear model, despite its simple structure, struggles to effectively capture the nonlinear and volatile trends in solar radiation data, leading to large prediction errors. The DLinear model, which introduces a dynamic weighting mechanism, improves performance to some extent, but remains limited in its ability to handle complex seasonal fluctuations and long-term forecasting tasks. Informer and Autoformer models have limitations in capturing long-term dependencies, especially when processing year-round datasets, where complex seasonal variations often affect their prediction performance. While the LSTM model can effectively handle dependencies in time series data, its low training efficiency and susceptibility to vanishing and exploding gradients result in low prediction accuracy.

[0157] The RMN-GLUformer model proposed in this invention can effectively capture long-term dependencies in time series through a self-attention mechanism, addressing the shortcomings of the Linear and DLinear models in capturing nonlinear relationships. Furthermore, through multi-head self-attention, it can more accurately model the seasonality and volatility of solar radiation data. The RMN-GLUformer also enhances its ability to model nonlinear relationships in data by introducing the GLU layer, effectively addressing the shortcomings of the Informer, Autoformer, and LSTM in handling complex seasonal fluctuations. This allows it to more comprehensively capture the changing patterns of solar radiation data and improve the model's predictive capabilities.

[0158] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A solar radiation prediction method based on RMN-GLUformer, characterized in that: The prediction method comprises the following steps: Step S1: Obtain historical solar radiation and meteorological data for a target area for no less than one year, wherein the historical solar radiation data is total radiation data, and the historical meteorological data is temperature, dew point temperature, relative humidity, surface albedo, wind speed, air pressure, precipitation, wind direction, and aerosol optical depth; collect the historical solar radiation data and historical meteorological data at a fixed time resolution ΔT to obtain an original data set; then perform quality control and cleaning on the original data set to obtain a training data set; then split the training data set in chronological order, using no less than 60% of the data as a training set and the remaining data as a validation set; Step S2: Establish RMN preprocessing module The RMN preprocessing module consists of three residual blocks, a fully connected output layer, and a smoothing operation. Each residual block consists of two fully connected layers and a ReLU activation function. For a residual block, the input x is first transformed by the first fully connected layer (fc1), and then nonlinearly transformed by the ReLU activation function, outputting the intermediate result z1. z1=ReLU(fc1(x)) Then, it is transformed through the second fully connected layer (fc2) to obtain the output z2: z2=fc2(z1) Finally, the output z2 of the second fully connected layer (fc2) is added to the original input x to form the final output: y=a²+x The output y of the residual block is: y=F(x)+x Where F(x)=fc2(ReLU(fc1(x))) is the residual term after a series of transformations; For the RMN preprocessing module, assume that the input data x0 passes through the first residual block and obtains the output x1: x1=F1(x0)+x0 Then, x1 is used as the input of the second residual block to obtain x2: x2=F2(x1)+x1 Finally, after the third residual block, we get x3: x3=F3(x2)+x2 The output after three residual blocks is: x3=F3(F2(F1(x0)+x0)+F1(x0)+x0)+F2(F1(x0)+x0)+F1(x0)+x0 F1(·), F2(·), and F3(·) represent the operations processed by the first residual block, the second residual block, and the third residual block, respectively; Then x3 is processed by the fully connected output layer to obtain the denoised data, and then the denoised signal is smoothed using a smoothing operation of k sliding windows to obtain the output data of the RMN preprocessing module. This output signal is the input of the GLUformer model; Step S3: Establish a prediction model based on GLUformer The GLUformer-based prediction model includes an embedding layer, multiple encoders, and a fully connected layer. The data input to the prediction model is first processed by the embedding layer to form multidimensional sequence data. The time steps of the multidimensional sequence data are then positionally encoded and then input to the encoder. The multiple encoders are connected in sequence, with the output of the previous encoder serving as the input of the next encoder. The output of the last encoder is then processed by the fully connected layer to obtain the output of the prediction model. The input of the prediction model is the output of the RMN preprocessing module; each encoder includes a multi-head attention module, a first residual connection and layer normalization operation, a GLU module and a second residual connection and layer normalization operation; the data P0 input to the encoder is first processed by the multi-head attention module to obtain data P1; the result obtained by performing a residual connection on data P1 and data P0 is then subjected to a layer normalization operation to obtain data P2; data P2 is input to the GLU module for processing to obtain data P3; the result obtained by performing a residual connection on data P3 and data P2 is then subjected to a layer normalization operation to obtain data P4, which is the output of the encoder; The encoder's multi-head attention module first projects the input into multiple subspaces, then calculates the attention weight of each subspace, and finally combines the representations of multiple subspaces to obtain the final output; In the implementation, the multi-head attention module maps the input data Z0 into Q, K, and V through three different linear transformations, and then splits Q, K, and V into h parts respectively, where h is the number of attention heads. The specific operation is as follows: in, is the projection matrix of the i-th subspace; then the self-attention output is calculated for each subspace, where the self-attention output head of the i-th subspace is i for: In the above formula, is the scaling factor, d k K i Dimension value of ; Then the outputs of all subspaces are concatenated and the final output is obtained through linear transformation: MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O Where W O It is a projection matrix used for linear transformation to enhance the expressiveness of data; Then the output MultiHead(Q,K,V) is subjected to the first residual connection and layer normalization operation: From r =LayerNorm(Z0+MultiHead(Q,K,V)) Where Z0 represents the input of the multi-head attention module, and MultiHead(Q,K,V) represents the output of the multi-head attention module; Then the output Z obtained after the first residual connection and layer normalization operation is r Input into the GLU module. The principle formula of the GLU module is as follows: <h2 style=";text-align:left;direction:ltr">GLU(Z<h2 style=";text-align:left;direction:ltr"> r <h2 style=";text-align:left;direction:ltr"> )=(W1Z<h2 style=";text-align:left;direction:ltr"> r <h2 style=";text-align:left;direction:ltr"> +b1)⊙σ(W2Z<h2 style=";text-align:left;direction:ltr"> r <h2 style=";text-align:left;direction:ltr"> +b2) Among them, W1 and W2 are two different linear transformation matrices, b1 and b2 are corresponding bias terms, σ is the GELU activation function, and ⊙ represents the Hadamard product, that is, element-by-element multiplication; The output of the GLU module is connected to the second residual and normalized layer to obtain the output Z of the encoder. p : WITH p =LaterNorm(Z r +GLU(Z r )) The output of the previous encoder is used as the input of the next encoder, and the output of the last encoder is sent to a fully connected layer for linear transformation to obtain the final prediction result; Step S4: Training the RMN preprocessing module First, the random assignment method is used to initialize the network parameters in the RMN preprocessing module. Then, a fixed-size sliding window k is used to slice the training set and validation set data in step S1. Each slice is a data sample. A batch of data samples in the training set is sequentially input into the RMN preprocessing module. A data sample X i After being processed by the RMN preprocessing module, the output is denoised and then smoothed to obtain Data sample X i After the same smoothing process, we can get Y i ; Y i As the true value, As the predicted value, the training loss of a batch of training sets uses the mean square error between the true value and the predicted value of each feature dimension data of all data samples; According to the training loss of a batch of the training set, the Adam optimizer with adaptive learning rate is used for back propagation to update the network parameters of the RMN preprocessing module once, completing the training of a batch of samples; then the data samples of the next batch in the training set are input, and the network parameters when the previous batch of samples is trained are used as the initial parameters when the next batch of samples is trained. The training process of a batch of samples is repeated continuously until the last batch of samples in the training set is trained, completing a round of training; the network parameters when the previous round of training is completed are used as the initial parameters when the next round of training is trained, and the process of a round of training is repeated continuously; after each round of training, the data samples in the validation set are sequentially input into the RMN preprocessing module of the current parameters, and the training loss of the data samples in the validation set is calculated; when the training loss of the data samples in the validation set decreases by less than the preset value for three consecutive rounds or reaches the preset upper limit of the iteration round, the training of the RMN preprocessing module is completed; Step S5: Training the GLUformer-based prediction model Step S5.1: Slice the original training set and validation set data in step S1 using a fixed-size sliding window k. Then, process each of the slices through the RMN preprocessing module pre-trained in step S4 to obtain a temporally continuous preprocessed training set and preprocessed validation set. Then, calculate the mean and standard deviation of each feature dimension of the data in the preprocessed training set, and standardize each feature dimension of the preprocessed training set and preprocessed validation set according to the mean and standard deviation of each feature dimension to obtain a standardized training set and validation set. A sliding window of T+t time steps is used to slice the data of the standardized training set and validation set respectively to generate the training sample set and validation sample set; Step S5.2 uses the Xavier initialization method to initialize the weight matrix in the prediction model to ensure that the variance of the input of each layer remains consistent. At the same time, all bias terms are initialized to zero and the hyperparameters of the prediction model are set. For the positional encoding in the GLUformer module, sine and cosine functions are used for initialization. Step S5.3: A training sample in the training sample set includes historical meteorological data W and corresponding historical solar radiation data H, where W = {w1, w2, ..., w T }, is the input of the prediction model; H = {h T+1 ,h T+2 ,...,h T+t }, which is the reference value of the output of the prediction model; the historical meteorological data W of a training sample is output to the initialized prediction model, and the output of the prediction model is the predicted solar radiation related data in is the predicted value of the prediction model for the historical solar radiation related data at time step T+t; the mean square error between the reference value output by the prediction model of a training sample and the predicted solar radiation related data output by the prediction model is calculated, and then the result is added to the L2 regularization term. The result is used as the training loss. The calculation formula of the loss function is: in, is the mean square error between the reference value of the prediction model output of the training sample and the predicted solar radiation related data output by the prediction model, λ is the regularization coefficient, θ i is the i-th parameter of the prediction model, M is the total number of parameters of the prediction model; According to the training loss value of a training sample, the Adam optimizer is used to update the prediction model parameters once to complete the training of a training sample; then the next training sample is input, and the network parameters when the previous training sample is completed are used as the initial parameters for the next training sample training, and the training process of a training sample is repeated until the last sample in the training sample set is trained, completing a round of training; the network parameters when the previous round of training is completed are used as the initial parameters for the next round of training, and the training process of a round is repeated continuously; after each round of training, all training samples in the validation sample set are input in turn, and the mean of the training loss of all training samples in the validation sample set is calculated. When the mean of the training loss does not decrease for 5 consecutive rounds or the training round reaches the maximum preset value, the prediction model training is completed, and the parameter values of the prediction model with the smallest training loss of the validation sample set are saved to obtain the optimal prediction model; Step S6: Solar radiation prediction Predict the solar radiation data of the target area at time point t after time point P in step S1; first obtain the solar radiation data of time point P and the solar radiation data after time point P. time points and the time points before P The meteorological data of k+T time points with a time interval of ΔT are first subjected to quality control and cleaning as described in step S1, and the obtained results are input into the RMN preprocessing module that completes the training in step S4 to obtain a data sequence of T time steps; then the mean and standard deviation of each feature dimension of the data in the preprocessing training set in step S5 are selected, and the same standardization operation is performed on the data sequence, and the obtained results are input into the optimal prediction model in step S5, which outputs solar radiation-related data at t time points after time point P; the parameters contained in the meteorological data are the same as those of the historical meteorological data in step S1.

2. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S1, the time resolution is 10 minutes.

3. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S1, the training data set is divided in chronological order, the first 80% of the data is used as the training set, and the remaining 20% of the data is used as the validation set.

4. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S1, the quality control and cleaning are specifically as follows: outliers are directly eliminated; missing values are filled.

5. The solar radiation prediction method based on RMN-GLUformer according to claim 4, characterized in that: For single missing value problems, linear interpolation is used to fill the missing values. For long-term data missing, spatial interpolation of neighboring sites is used to fill the missing values.

6. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S2, the smoothing formula is: Among them, x denoise (i) is the value of the denoised data at time point i; k is the size of the sliding window, which is an odd number; x res (t) is the result obtained by smoothing the value at time point t.

7. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S4, the default value of the decrease in the training loss of the data samples of the validation set for three consecutive rounds is 1e -4 , the preset upper limit of iteration rounds is 200.

8. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S5, among the hyperparameters, dropout is set to 0.1, batch size is set to 32, the number of encoders in the GLUformer module is set to 3, and the number of attention heads in the multi-head attention module is set to 8.

9. The solar radiation prediction method based on RMN-GLUformer according to claim 1, characterized in that: In step S5.1, the normalization is a z-score normalization operation.

Images