A weather time series prediction method balancing stationary and non-stationary information

Abstract

The application discloses a meteorological time series prediction method balancing stationary and non-stationary information, and belongs to the field of atmospheric science and ocean science. The channel grouping and aggregation operation proposed by the application groups meteorological elements according to similar time sequence rules, alleviates the problem of overfitting caused by complete fusion or independence of channels and the problem of inability to capture the correlation between different variables, and extracts local features in the time series through the aggregation operation, thereby reducing the parameter quantity and calculation amount of the network. The gating normalization module proposed by the application dynamically adjusts the weight of different features in the time sequence data through gating weights, realizes effective screening and enhancement of the features, and thereby improves the stability and accuracy of the model in processing complex meteorological data; the balanced attention mechanism proposed by the application reintegrates non-stationary information by introducing a bias term, avoids the loss of key data distribution characteristics caused by excessive normalization, and is more applicable in actual scenarios.

Description

Technical Field

[0001] This invention relates to the fields of atmospheric science and ocean science, and specifically to a method for predicting meteorological time series information that balances stationary and non-stationary information. Background Technology

[0002] Meteorological time series forecasting, a field jointly studied by atmospheric and oceanographic sciences, is a crucial factor affecting people's lives, transportation, and economic development. Traditional time series forecasting methods typically assume that changes in time series follow a predefined pattern; however, real-world time series are often complex and variable, limiting the effectiveness of traditional models in practical applications.

[0003] In recent years, deep learning technology has played a crucial role in the field of meteorological time series forecasting. Through the analysis of massive amounts of meteorological data, deep learning can uncover complex patterns and regularities, providing new perspectives and methods for meteorological forecasting research. Initially, Recurrent Neural Networks (RNNs), capable of capturing time dependencies, achieved significant results in meteorological forecasting. RNNs can effectively process sequential data, capturing dependencies in time series through their recurrent structure. However, RNNs have certain limitations in capturing long-range dependencies. Transformer models have demonstrated superior performance compared to RNNs in time series analysis. Transformers process data in parallel through a self-attention mechanism, efficiently capturing long-range dependencies. The self-attention mechanism allows the model to consider data from all other points in the sequence while processing data at the current time point, thus overcoming the limitations of RNNs in processing long-sequence data. Furthermore, the parallel processing capability of Transformers makes them more efficient when processing large-scale data.

[0004] Despite its achievements in weather forecasting, Transformer still faces numerous limitations and challenges. First, excessive fusion of channels in meteorological data can lead to overfitting. For meteorological data containing multivariate time series, most methods employ complete channel fusion, aggregating all time series feature vectors and mapping them into the embedding space. This approach is susceptible to noise and prone to overfitting. To address this issue, Nie et al. proposed a completely independent channel strategy in their paper "A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers." While this avoids overfitting, it introduces a new problem: it fails to effectively capture the correlations between different variables. Different variables in meteorological data often exhibit complex interactions, and completely independent processing struggles to reflect these dependencies, thus affecting forecast accuracy. Furthermore, Liu et al., in their paper "Non-Stationary Transformers: Exploring the Stationarity in Time Series Forecasting," point out that excessive elimination of time-series differences in meteorological data can lead to excessive stationarization. Meteorological data typically exhibits non-stationarity, possessing complex and difficult-to-capture temporal dependencies. Traditional stabilization techniques attempt to simplify the modeling process by eliminating differences in data distribution over time. However, when using stabilized data for model training, it may lead to over-normalization, causing the model to learn only similar attention maps and weak temporal dependencies, resulting in overly stable predictions and large prediction errors.

[0005] In summary, the main problems currently facing meteorological time series forecasting include:

[0006] (1) Although the input method with fully fused channels can make full use of the information of all variables, its sensitivity to noise makes the model difficult to generalize in practical applications.

[0007] (2) Although the input method with completely independent channels avoids overfitting, it ignores the mutual influence between variables and cannot effectively capture the correlation between different variables.

[0008] (3) Although the stabilization technique reduces the impact of noise, it weakens the model’s ability to capture complex temporal dependencies, leading to over-normalization of the model. Summary of the Invention

[0009] To address the aforementioned problems, this invention provides a meteorological time series forecasting method that balances stationary and non-stationary information, thereby improving and enhancing the forecasting performance of meteorological data with non-stationary characteristics, and providing a more reliable and accurate forecasting tool for the field of meteorological forecasting.

[0010] To achieve the above objectives, the specific technical solution adopted by the present invention is as follows:

[0011] A meteorological time series forecasting method that balances stationary and non-stationary information includes the following steps:

[0012] S1: Acquire multivariate meteorological time series data, and use the constructed channel grouping operation to group meteorological elements according to similar time series patterns, so that the model can focus on learning the unique characteristics of variables within each group;

[0013] S2: Perform aggregation operations on the grouped data to capture time-related features in the time series data;

[0014] S3: Utilize the constructed gated normalization module to filter irrelevant or invalid features in time series data based on gated weights, while enhancing the correlation between valid features;

[0015] S4: By using a balanced attention-based encoder-decoder to introduce bias terms to re-integrate non-stationary information, the model learns time dependencies from stationary and non-stationary sequences.

[0016] S5: Generate the prediction results for each group through the inverse normalization module;

[0017] S6: Combine the prediction results of all groups to obtain the final prediction result.

[0018] Furthermore, the channel grouping operation described in step S1 utilizes principal component analysis to group meteorological elements with similar temporal patterns. Specifically, it determines the optimal number of groups by analyzing the variance interpretation table and the reduced factor loading quadrant diagram. This includes:

[0019] S1.1 Calculate the variance explanation table and assess the contribution rate of the principal components to the explanation of the variables;

[0020] S1.2, perform factor loading quadrant analysis to reduce multiple principal components to a two-dimensional space and display the spatial distribution of principal components in the form of a quadrant diagram;

[0021] S1.3, based on the variance interpretation table and factor loading quadrant analysis, consider the appropriate number of groups.

[0022] Furthermore, the input to the gating normalization module mentioned in step S3 is... D represents the number of channels, N represents the time series length, and the specific process of the gated normalization module is as follows:

[0023] S3.1, global context information is incorporated into each channel of the temporal feature, and a trainable embedding weight α = [α1,…,α] is introduced. D The formula is as follows:

[0024]

[0025] Among them, s d It is the calculated global context vector, where ∈ is a constant term to prevent differentiation at zero;

[0026] S3.2, performing L2 normalization on the global context vector yields:

[0027]

[0028] Where s = [s1, ..., s D When D is large, in order to prevent The scale was too small, so a scale factor was added.

[0029] S3.3 employs a gating mechanism to promote competition and cooperation among features, designing trainable weights γ = [γ1, ..., γ2]. D ] and bias β = [β1,…,β D When the weight γ d When the weight γ is positive, it will promote the "competition" between the features of this channel and the features of other channels; when the weight γ is positive... d When the value is negative, it encourages the features of this channel to "cooperate" with the features of other channels, as shown in the following formula:

[0030]

[0031] in, The gating weights output by the gating mechanism;

[0032] S3.4, for y d Perform standardized operations and coordinate with gating weights Multiplication, the formula is as follows:

[0033]

[0034] Among them, E(y) d ) is y d The mean of Var(y) d ) is y d variance It is the final output of gating normalization.

[0035] Furthermore, the balanced attention-based encoding and decoding described in step S4 assumes that each variable in the time series has the same variance during normalization, thus simplifying the variance to a single scalar σ. The specific process of the balanced attention mechanism is as follows:

[0036] S4.1, after gating and normalizing the data mapping, we obtain Q′ and K′:

[0037]

[0038] Among them, the data without gated normalization is obtained after linear mapping. The gating weights after dimensional changes. This represents the mean of Q, K, and V over the time dimension;

[0039] S4.2, the attention mechanism first performs a dot product of Q′ and K′, as shown in the following formula:

[0040]

[0041] in,

[0042] S4.3, use the Softmax function for the dot product of Q and K;

[0043] S4.4, the balanced attention mechanism performs attention calculations on the gated normalized sequences Q′, K′ and the unnormalized sequence V, as shown in the following formula:

[0044]

[0045] Furthermore, in step S4.3, the specific process of the dot product of Q and K is as follows:

[0046] S4.3.1, the formula for the dot product of Q and K is as follows:

[0047]

[0048] in, This is equivalent to adding the same value to every element in that row.

[0049] S4.3.2, since Softmax operates on each column of the matrix, adding the same value to each element in a row has no effect on the result of Softmax and can be omitted. Therefore, it can be simplified to:

[0050]

[0051] Where, σ 2 、(Kμ Q ) TIt cannot be obtained from the gated normalized sequence;

[0052] S4.3.3, using a simple multilayer perceptron, directly extracts σ from the unnormalized statistical data μ and σ of y. 2 、(Kμ Q ) T Learn separately, that is, learn τ = σ 2 Δ=Kμ Q :

[0053] logτ=MLP(σ,y), Δ=MLP(μ,y)

[0054] S4.3.4, returns the optimized dot product formula for Q and K:

[0055]

[0056] A meteorological time series forecasting model that balances stationary and non-stationary information includes the following modules:

[0057] The channel grouping module groups multivariate meteorological time-series data.

[0058] An aggregation module to capture time-related features in time series data;

[0059] The gating normalization module filters irrelevant or invalid features in time series data based on gating weights.

[0060] The encoder-decoder module based on balanced attention introduces a bias term to re-integrate non-stationary information, thereby better capturing the temporal dependencies in time series.

[0061] The denormalization module restores the original scale of the data and generates prediction results for each group;

[0062] The splicing module splices the prediction results of all groups to obtain the final prediction result.

[0063] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0064] The channel grouping and aggregation operations proposed in this invention group meteorological elements according to similar temporal patterns, alleviating the overfitting caused by complete channel fusion or independence and the inability to capture correlations between different variables. Aggregation operations extract local features from time series, reducing the number of network parameters and computational cost. The gated normalization module proposed in this invention dynamically adjusts the weights of different features in time series data through gate weights, achieving effective feature selection and enhancement, thereby improving the stability and accuracy of the model when processing complex meteorological data. The balanced attention mechanism proposed in this invention reintegrates non-stationary information by introducing a bias term, avoiding the loss of key data distribution features caused by over-normalization, making it more applicable in practical scenarios.

[0065] The meteorological time series forecasting method proposed in this invention exhibits good performance in both short-term and long-term forecasts. When the lookback window is 96, the MSE for short-term meteorological time series forecasts (forecast length of 96) is less than 0.17, and the MSE for long-term meteorological time series forecasts (forecast length of 720) is less than 0.33. Attached Figure Description

[0066] Figure 1 This is a network structure diagram of the meteorological time series prediction method of the present invention.

[0067] Figure 2 This is a quadrant analysis diagram of the factor loadings for channel grouping operations in this invention.

[0068] Figure 3 This is a schematic diagram of the gating normalization module in this invention.

[0069] Figure 4 This is a schematic diagram of the balanced attention mechanism in this invention.

[0070] Figure 5 This is a visualization of the prediction results obtained from an example of the present invention. Detailed Implementation

[0071] To enable those skilled in the art to better understand the technical solution of the present invention, the technical implementation of the present invention will be described and analyzed in more detail below with reference to the accompanying drawings and practical examples.

[0072] Example 1:

[0073] A meteorological time series forecasting method that balances stationary and non-stationary information, such as Figure 1 As shown, it includes the following steps:

[0074] Step S1: Based on similar temporal patterns, the multivariate meteorological time series data is grouped by channel. That is, principal component analysis is used to determine the optimal number of groups by analyzing the variance explanation table and the factor loading quadrant plot after dimensionality reduction. This allows the Weather dataset to be grouped by channel, enabling the model to focus on learning the unique characteristics of variables within each group. Weather is a meteorological time series dataset that collects 21 meteorological factors, such as humidity and air temperature, from the Max Planck Institute for Biogeochemistry meteorological station in Germany every 10 minutes in 2020, for a total of 52,696 time steps.

[0075] The specific process of channel grouping is as follows:

[0076] S1.1 Calculate the variance explanation table for the Weather dataset and evaluate the contribution rate of the principal components to the explanation of the variables;

[0077] S1.2, perform factor loading quadrant analysis, reduce multiple principal components to two-dimensional space, and display the spatial distribution of these principal components in the form of quadrant plots, so as to intuitively understand the internal structure and characteristics of the data;

[0078] S1.3, as shown in Table 1, when the number of principal components is 7, the eigenvalues ​​explained by the total variance are lower than 1 for the first time, and the contribution rate of the variables explained reaches 88.264%, which can ensure that the model can explain most of the variance, while avoiding the noise problem that may be introduced by the eigenvalues ​​being lower than 1. Figure 2 The distribution of each variable is presented in the form of a quadrant plot. The distance between variables reflects their similarity in the principal component space; the closer the distance, the higher the correlation between them. (Table 1 and...) Figure 2 It can be seen that dividing the input factors into 7 groups is a more reasonable choice;

[0079] Among them, the eigenvalues ​​reflect the contribution of each principal component to the total variance. The larger the eigenvalue, the more variance the principal component can explain. The variance explained rate is a measure of the ability of a principal component to explain the total data variance, while the cumulative variance explained rate is the total explanatory power of the first few principal components. When the cumulative variance explained rate is high, these principal components can summarize the main information of the data well.

[0080] Table 1 Explanation of Variance

[0081]

[0082]

[0083] Step S2 involves performing an aggregation operation on the grouped data to capture time-related features in the time series data. This involves sliding the convolution kernel across the time series data and calculating the weighted sum of the data within the window. The formula for the length of the aggregated time series is as follows:

[0084]

[0085] Where W is the lookback window size, F is the kernel size, S is the stride, and P is the padding. Note that the last value of the time series is used for padding to ensure that the padding value is as close as possible to the actual value of the series, thereby avoiding the introduction of too much noise.

[0086] Step S3: Since the meteorological time series obtained in step S2 may contain some extreme or outlier values, thus affecting the calculation of the mean and standard deviation during normalization, the proposed gated normalization is used to process the meteorological time series data. The gated normalization is as follows: Figure 3 As shown, let the input of the gated normalization be... Where D is the number of channels and N is the time series length, the specific process of the gated normalization is as follows:

[0087] S3.1, global context information is incorporated into each channel of the temporal feature, and a trainable embedding weight α = [α1,…,α] is introduced. D The formula is as follows:

[0088]

[0089] Among them, s d It is the calculated global context vector, where ∈ is a constant term to prevent differentiation at zero;

[0090] S3.2, performing L2 normalization on the global context vector yields:

[0091]

[0092] Where s = [s1, ..., s D When D is large, in order to prevent The scale was too small, so a scale factor was added.

[0093] S3.3 employs a gating mechanism to promote competition and cooperation among features, designing trainable weights γ = [γ1, ..., γ2]. D ] and bias β = [β1,…,β D When the weight γ d When the weight γ is positive, it will promote the "competition" between the features of this channel and the features of other channels; when the weight γ is positive... dWhen the value is negative, it encourages the features of this channel to "cooperate" with the features of other channels, as shown in the following formula:

[0094]

[0095] in, The gating weights output by the gating mechanism;

[0096] S3.4, for y d Perform standardized operations and coordinate with gating weights Multiplication, the formula is as follows:

[0097]

[0098] Among them, E(y) d ) is y d The mean of Var(y) d ) is y d variance It is the final output of gating normalization.

[0099] Step S4: Embed the normalized meteorological time-series data obtained in step S3 using a linear projection W. f and learnable addition position encoding W pos The features are mapped to a D-dimensional Transformer latent space and the order information of the time series data is added to obtain... Embedded data The signal is fed into the encoder and decoder to obtain feature z. (i) The balanced attention mechanism in the encoder reintroduces non-stationary information through a multilayer perceptron, enabling the model to learn temporal dependencies from both stationary and non-stationary sequences, thereby mitigating over-stationarity. The balanced attention mechanism, for example... Figure 4 As shown, assuming that each variable in the time series has the same variance during normalization, the variance can be simplified to a single scalar σ. The specific process of the balanced attention mechanism is as follows:

[0100] S4.1, after gating and normalizing the data mapping, we obtain Q′ and K′:

[0101]

[0102] Among them, the data without gated normalization is obtained after linear mapping. The gating weights after dimensional changes. This represents the mean of Q, K, and V over the time dimension;

[0103] S4.2, the attention mechanism first performs a dot product of Q′ and K′, as shown in the following formula:

[0104]

[0105] in,

[0106] S4.3, the Softmax function is used to perform the dot product of Q and K. The specific process of the dot product of Q and K is as follows:

[0107] S4.3.1, the formula for the dot product of Q and K is as follows:

[0108]

[0109] in, This is equivalent to adding the same value to every element in that row.

[0110] S4.3.2, since Softmax operates on each column of the matrix, adding the same value to each element in a row has no effect on the result of Softmax and can be omitted. Therefore, it can be simplified to:

[0111]

[0112] Where, σ 2 、(Kμ Q ) T It cannot be obtained from the gated normalized sequence;

[0113] S4.3.3, using a simple multilayer perceptron, directly extracts σ from the unnormalized statistical data μ and σ of y. 2 、(Kμ Q ) T Learn separately, that is, learn τ = σ 2 Δ=Kμ Q :

[0114] logτ=MLP(σ,y), Δ=MLP(μ,y)

[0115] S4.3.4, returns the optimized dot product formula for Q and K:

[0116]

[0117] S4.4, the balanced attention mechanism performs attention calculations on the gated normalized sequences Q′, K′ and the unnormalized sequence V, as shown in the following formula:

[0118]

[0119] Step S5, use the feature z obtained in step S4 (i)Flattening is achieved through an inverse normalization module, which uses a linear head to generate the prediction results for each group, thereby eliminating the scaling transformation introduced in the preprocessing stage and making the prediction results the same scale as the original data.

[0120] Step S6: Concatenate the prediction results of each group in step S5 to obtain the final prediction result.

[0121] Example 2

[0122] To demonstrate the effectiveness of the meteorological time series forecasting method of this invention, experiments were conducted on the Wearther dataset. All experiments were performed on a single NVIDIA GeForce RTX 4090 graphics card using the PyTorch framework. The Adam optimizer was used to train the model. The learning rate was set to 10⁻⁴, and the batch size was set to 32. The model consists of 6 encoder layers and 1 decoder layer. For the implementation of the balanced attention mechanism, a two-layer multilayer perceptron was used, with the hidden layer dimension set to 128. The loss functions used during training were mean squared error (MSE) and mean absolute error (MAE), with lower MSE / MAE indicating better performance.

[0123] Table 2 compares the prediction results with a lookback window of 96 and a prediction length T∈{96,192,336,720}.

[0124]

[0125] Table 2 shows the performance comparison of the meteorological time series forecasting method of this invention with eleven other benchmark models for forecasting meteorological elements with a lookback window of 96 and a forecast length T∈{96,192,336,720}. The best result is indicated in bold, and the second best result is indicated by underline. Table 2 shows that the meteorological time series forecasting method of this invention achieves the best performance on all MSE indices, which significantly demonstrates its advantage in forecast accuracy. Compared with PatchTST, although the MAE is slightly lower at T=336, the meteorological time series forecasting method of this invention achieves a higher improvement in MSE, which means that the meteorological time series forecasting method of this invention is superior in reducing forecast bias. Compared with iTransformer, although the MAE of the meteorological time series forecasting method of this invention is slightly lower at T=96, in terms of long-term time series forecasting, the meteorological time series forecasting method of this invention outperforms iTransformer in both MSE and MAE, which further proves the advantage of the meteorological time series forecasting method of this invention in capturing global features and long-term dependencies in time series data.

[0126] Example 3

[0127] With a lookback window of 96, the forecasting performance of the meteorological time series forecasting method of this invention on different forecast lengths was tested. Figure 5 As can be seen, the meteorological time series forecasting method of this invention exhibits excellent performance across various forecast lengths. The forecast results not only show minimal error compared to the actual series, but also accurately reflect the actual temporal patterns of meteorological elements, whether for short-term or long-term forecasts. Notably, with the increase in forecast series length, the meteorological time series forecasting method of this invention does not show any collapse in forecast performance or other performance degradation; on the contrary, it maintains high forecast accuracy and stability, fully demonstrating the unique advantages of this meteorological time series forecasting method in capturing and utilizing the periodic characteristics of meteorological data.

[0128] The above description details specific embodiments of the present invention, including its objectives, technical solutions, and beneficial effects, but does not limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations based on the technical solutions of the present invention, without inventive effort, still fall within the scope of protection of the present invention.

Claims

1. A method for forecasting meteorological time series information that balances stationary and non-stationary information, characterized in that, Includes the following steps: S1: Acquire multivariate meteorological time-series data, and use the constructed channel grouping operation to group meteorological elements according to similar time-series patterns; S2: Perform aggregation operations on the grouped data to capture time-related features in the time series data; S3: Utilize the constructed gated normalization module to filter irrelevant or invalid features in time series data based on gated weights, while enhancing the correlation between valid features; S4: Introduce bias terms using a balanced attention-based encoder-decoder to re-integrate non-stationary information; In S4, the balanced attention-based encoding and decoding assumes that each variable in the time series has the same variance during normalization, thus simplifying the variance to a single scalar. The specific process of the attention balancing mechanism is as follows: S4.1, obtained after gating and normalizing the data mapping. , : ， Among them, the data without gated normalization is obtained after linear mapping. , The gating weights after dimensional changes. , , express The mean over time; S4.2, Attention mechanism first and The formula for dot product is as follows: in, ; S4.3, for and The dot product is obtained using the Softmax function; S4.4, Balanced attention mechanism on gating normalized sequences , and unnormalized sequences The formula for calculating attention is as follows: Attn( Softmax Softmax S5: Generate the prediction results for each group through the inverse normalization module; S6: Combine the prediction results of all groups to obtain the final prediction result.

2. The meteorological time series forecasting method as described in claim 1, characterized in that, The channel grouping operation in S1 uses principal component analysis to group meteorological elements with similar temporal patterns. Specifically, the number of groups is determined by analyzing the variance interpretation table and the factor loading quadrant diagram after dimensionality reduction.

3. The meteorological time series forecasting method as described in claim 2, characterized in that, S1 specifically includes: S1.1 Calculate the variance explanation table and assess the contribution rate of the principal components to the explanation of the variables; S1.2, perform factor loading quadrant analysis to reduce multiple principal components to a two-dimensional space and display the spatial distribution of principal components in the form of a quadrant diagram; S1.3, determine the number of groups based on the variance interpretation table and factor loading quadrant analysis.

4. The meteorological time series forecasting method as described in claim 1, characterized in that, The input to the gate normalization module mentioned in S3 is , , For the number of channels, Given the time series length, the specific process of the gated normalization module is as follows: S3.1 integrates global context information into each channel of the temporal feature and introduces an embedding weight. The formula is as follows: in, It is the calculated global context vector. It is a constant term to prevent differentiation at zero; S3.2, performing L2 normalization on the global context vector yields: in, ,when When it is large, in order to prevent The scale was too small, so a scale factor was added. ; S3.3 employs a gating mechanism to promote competition and cooperation among features, and designs weights. and bias When weight When the weight is positive, it will encourage the features of this channel to compete with the features of other channels; when the weight... When the value is negative, it encourages the features of this channel to cooperate with the features of other channels, as shown in the following formula: in, The gating weights output by the gating mechanism; S3.4, for Perform standardized operations and coordinate with gating weights Multiplication, the formula is as follows: ； in, yes The mean, yes variance It is the final output of gating normalization.

5. The meteorological time series forecasting method as described in claim 1, characterized in that, In step S4.3, and The specific process of dot product is as follows: S4.3.1, and The dot product formula is as follows: Softmax Softmax in, , This is equivalent to adding the same value to every element in that row; S4.3.2, since Softmax operates on each column of the matrix, adding the same value to each element in a row has no effect on the result of Softmax, so it simplifies to: Softmax Softmax in, , It cannot be obtained from the gated normalized sequence; S4.3.3, directly from the unnormalized data using a simple multilayer perceptron. Statistical data and China , Learn separately, that is, learn , : S4.3.4, Return the optimized version and Dot product formula: Softmax Softmax 。 6. A forecasting model for a meteorological time series forecasting method based on the balanced stationary and non-stationary information of claim 1, characterized in that, It includes the following modules: The channel grouping module groups multivariate meteorological time-series data. An aggregation module to capture time-related features in time series data; The gating normalization module filters irrelevant or invalid features in time series data based on gating weights. The encoder-decoder module based on balanced attention introduces a bias term to re-integrate non-stationary information, thereby capturing the temporal dependencies in time series. The denormalization module restores the original scale of the data and generates prediction results for each group; The splicing module splices the prediction results of all groups to obtain the final prediction result.

Images