Meteorological time sequence prediction method for balancing stationary and non-stationary information

Through the combination of channel grouping, gated normalization and balanced attention codec, the problems of overfitting and overstationary in meteorological time series prediction are solved, improving the accuracy and stability of the prediction, and significantly improving the meteorological conditions Predictive performance of data.

CN119150227AActive Publication Date: 2024-12-17OCEAN UNIV OF CHINA

Patent Information

Application Number
CN202411186725.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-17
Estimated Expiration
2044-08-28

AI Technical Summary

Technical Problem

Existing meteorological time series prediction methods are prone to overfitting or over-stationary when processing non-stationary meteorological data, resulting in large prediction errors and inability to effectively capture the complex mutual influences between different variables.

Method used

Use channel grouping operations to group meteorological elements with similar timing laws, combine the gated normalization module and balanced attention codec to filter irrelevant features and re-fusion of non-stationary information, generate prediction results through reverse normalization, and reduce noise impact and enhance feature correlation.

Benefits of technology

It improves the accuracy and stability of meteorological time series prediction, reduces the amount of model parameters and calculations, avoids the problems of overfitting and over-normalization, and significantly improves the performance of short-term and long-term predictions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119150227A_ABST
    Figure CN119150227A_ABST
Patent Text Reader

Abstract

The invention discloses a meteorological time sequence prediction method for balancing stationary and non-stationary information, and belongs to the field of atmospheric science and marine science. According to the channel grouping and aggregation operation provided by the invention, meteorological elements are grouped according to a similar time sequence rule, the problems of overfitting and incapability of capturing correlation among different variables caused by complete channel fusion or independence are relieved, and local features in a time sequence are extracted through aggregation operation, so that the accuracy of the time sequence is improved. And the parameter quantity and the calculation quantity of the network are reduced. According to the invention, the gating normalization module dynamically adjusts the weights of different features in time series data through the gating weight, and achieves the effective screening and enhancement of the features, thereby improving the stability and accuracy of the model in the processing of complex meteorological data. According to the attention balancing mechanism provided by the invention, non-stationary information is re-integrated by introducing an offset item, so that the loss of key data distribution characteristics caused by excessive normalization is avoided, and the method has higher applicability in actual scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the fields of atmospheric science and ocean science, and in particular to a meteorological time series prediction method that balances stationary and non-stationary information. Background Art

[0002] Meteorological time series prediction, as a field of joint research between atmospheric science and ocean science, is an important factor affecting people's lives, transportation, and economic development. Traditional time series prediction methods usually assume that the changes in time series follow a certain predefined pattern. However, time series in reality are often complex and changeable. This assumption limits the effectiveness of traditional models in practical applications.

[0003] In recent years, deep learning technology has occupied an important position in the field of meteorological time series forecasting. Through the analysis of massive meteorological data, deep learning can dig out complex laws and patterns, providing new perspectives and methods for meteorological forecasting research. Initially, the recurrent neural network (RNN) that can capture time dependencies has achieved remarkable results in the field of meteorological forecasting. RNN can effectively process sequence data and capture dependencies in time series through its cyclic structure. However, RNN has certain limitations in capturing long-distance dependencies. The Transformer model has shown superior performance than RNN in the field of time series analysis. Transformer processes data in parallel through the self-attention mechanism and can efficiently capture long-distance dependencies. The self-attention mechanism allows the model to consider the data of all other time points in the sequence when processing the data at the current time point, thereby overcoming the limitations of RNN in processing long sequence data. In addition, the parallel processing capability of Transformer makes it more efficient when processing large-scale data.

[0004] Although Transformer has made certain achievements in the field of meteorological forecasting, it still faces many limitations and challenges. First, excessive fusion between channels in meteorological data may lead to overfitting. For meteorological data containing multivariate time series, most methods adopt the method of complete channel fusion, that is, aggregating all time series feature vectors and mapping them into the embedding space, which is susceptible to noise and overfitting. To solve this problem, Nie et al. proposed a channel-completely independent strategy in the paper "A Time Series Is Worth 64 Words: Long-Term Forecasting with Transformers". Although this method avoids overfitting, it also brings new problems, namely, it cannot effectively capture the correlation between different variables. There are often complex interactions between different variables in meteorological data, and completely independent processing methods are difficult to reflect the dependencies between these variables, thus affecting the accuracy of prediction. In addition, Liu et al. pointed out in the paper "Non-Stationary Transformers: Exploring the Stationarity in Time Series Forecasting" that excessive elimination of differences between meteorological data time series may lead to excessive stabilization of data. Meteorological data usually exhibits non-stationary characteristics and has complex and difficult-to-capture time series dependencies. Traditional smoothing techniques attempt to simplify the modeling process by eliminating differences in data distribution in the time dimension. However, when using smoothed data for model training, it may cause the model to be over-normalized, so that the model can only learn similar attention maps and weak temporal dependencies, resulting in overly smooth prediction results and large prediction errors.

[0005] In summary, the main problems facing current meteorological time series forecasting include:

[0006] (1) Although the channel-fully fused input method can fully utilize the information of all variables, its sensitivity to noise makes it difficult to generalize the model in practical applications.

[0007] (2) Although the channel-independent input method avoids overfitting, it ignores the mutual influence between variables and cannot effectively capture the correlation between different variables.

[0008] (3) Although the stabilization technology reduces the impact of noise, it weakens the model's ability to capture complex time series dependencies, resulting in the problem of over-normalization of the model. Summary of the invention

[0009] In response to the above-mentioned problems, the present invention provides a meteorological time series prediction method that balances stationary and non-stationary information, so as to improve and enhance the prediction performance of meteorological data with non-stationary characteristics, and provide a more reliable and accurate prediction tool for the field of meteorological prediction.

[0010] In order to achieve the above object, the specific technical solution adopted by the present invention is:

[0011] A meteorological time series prediction method that balances stationary and non-stationary information includes the following steps:

[0012] S1: Obtain multivariate meteorological time series data, use the constructed channel grouping operation to group meteorological elements according to similar time series rules, so that the model can focus on learning the unique characteristics of the variables in each group;

[0013] S2: Perform aggregation operations on the grouped data to capture time-related features in the time series data;

[0014] S3: Using the constructed gated normalization module, irrelevant or invalid features in the time series data are filtered based on the gated weights, while enhancing the correlation between valid features;

[0015] S4: A balanced attention-based encoder-decoder is used to introduce a bias term to reintegrate non-stationary information, allowing the model to learn temporal dependencies from stationary and non-stationary sequences;

[0016] S5: Generate the prediction results of each group through the anti-normalization module;

[0017] S6: Concatenate the prediction results of all groups to obtain the final prediction result.

[0018] Furthermore, the channel grouping operation described in step S1 utilizes the principal component analysis method to group meteorological elements with similar temporal regularities, that is, by analyzing the variance explanation table and the factor loading quadrant diagram after dimensionality reduction, the number of optimal groups is determined; specifically, it includes:

[0019] S1.1, calculate the variance explanation table and evaluate the contribution of the principal components to the explanation of the variables;

[0020] S1.2, perform factor loading quadrant analysis to reduce multiple principal components to two-dimensional space and present the spatial distribution of principal components in the form of quadrant diagram;

[0021] S1.3, consider the appropriate number of groups based on the variance explanation table and factor loading quadrant analysis.

[0022] Furthermore, the input of the gated normalization module described in step S3 is D is the number of channels, N is the length of the time series, and the specific process of the gated normalization module is as follows:

[0023] S3.1, integrate the global context information into each channel of the temporal feature, and introduce a trainable embedding weight α = [α 1 ,…,α D ], the formula is as follows:

[0024]

[0025] Among them, s d is the calculated global context vector, ∈ is a constant term to prevent differentiation at zero;

[0026] S3.2, perform L2 normalization on the global context vector, and we get:

[0027]

[0028] Where s = [s 1 ,…,s D ], when D is large, in order to prevent The scale is too small, add a scale factor

[0029] S3.3, adopt the gating mechanism to promote competition and cooperation between features, and design trainable weights γ = [γ 1 ,…,γ D ] and bias β = [β 1 ,…,β D ], when the weight γ d When it is positive, it will promote the characteristics of this channel to "compete" with the characteristics of other channels; when the weight γ d When it is negative, the features of this channel will be encouraged to "cooperate" with the features of other channels. The formula is as follows:

[0030]

[0031] in, The gating weight output by the gating mechanism;

[0032] S3.4, for y d Perform normalization and gate weights Multiplying, the formula is as follows:

[0033]

[0034] Among them, E(y d ) is y d The mean value of Var(y d ) is y d The variance of is the final output of gated normalization.

[0035] Furthermore, the balanced attention-based encoding and decoding described in step S4 assumes that each variable of the time series has the same variance in normalization, so that the variance can be simplified to a single scalar σ. The specific process of the balanced attention mechanism is as follows:

[0036] S4.1, after the gated normalized data mapping, we get Q′ and K′:

[0037]

[0038] Among them, the data without gated normalization is obtained after linear mapping. is the gating weight after dimension change, Represents the mean of Q, K, and V in the time dimension;

[0039] S4.2, the attention mechanism first performs a dot product on Q′ and K′, the formula is as follows:

[0040]

[0041] in,

[0042] S4.3, use the Softmax function for the dot product of Q and K;

[0043] S4.4, the balanced attention mechanism performs attention calculation on the gated normalized sequence Q′, K′ and the unnormalized sequence V, the formula is as follows:

[0044]

[0045] Furthermore, in step S4.3, the specific process of the dot multiplication of Q and K is as follows:

[0046] S4.3.1, the dot product formula of Q and K is as follows:

[0047]

[0048] in, This is equivalent to adding the same value to each element in the row.

[0049] S4.3.2, since Softmax operates on each column of the matrix, adding the same value to each element of a row has no effect on the result of Softmax and can be omitted, so it can be simplified to:

[0050]

[0051] Among them, σ 2 、(KμQ ) T It cannot be obtained from the gated normalized sequence;

[0052] S4.3.3, we use a simple multilayer perceptron to directly extract σ from the unnormalized statistics μ and σ of y. 2 、(Kμ Q ) T Learn separately, that is, learn τ=σ 2 , Δ=Kμ Q :

[0053] logτ=MLP(σ,y), Δ=MLP(μ,y)

[0054] S4.3.4, returns the optimized Q and K dot product formula:

[0055]

[0056] A meteorological time series prediction model that balances stationary and non-stationary information includes the following modules:

[0057] Channel grouping module, which groups multivariate meteorological time series data;

[0058] Aggregation module to capture time-dependent features in time series data;

[0059] The gated normalization module filters irrelevant or invalid features in the time series data based on the gated weights;

[0060] Based on the balanced attention encoder-decoder module, a bias term is introduced to reintegrate non-stationary information to better capture the temporal dependencies in time series;

[0061] The denormalization module restores the original scale of the data and generates prediction results for each group;

[0062] The splicing module splices the prediction results of all groups to obtain the final prediction results.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] The channel grouping and aggregation operations proposed in the present invention group meteorological elements according to similar time series rules, alleviate the overfitting caused by complete fusion or independence of channels and the problem of being unable to capture the correlation between different variables, and extract local features in the time series through aggregation operations, reducing the number of network parameters and the amount of calculation. The gated normalization module proposed in the present invention dynamically adjusts the weights of different features in the time series data through gated weights to achieve effective screening and enhancement of features, thereby improving the stability and accuracy of the model when processing complex meteorological data; the balanced attention mechanism proposed in the present invention introduces a bias term to reintegrate non-stationary information, avoiding the loss of key data distribution features caused by over-normalization, and is more applicable in actual scenarios.

[0065] The meteorological time series prediction method proposed in the present invention shows good performance in both short-term and long-term predictions. When the lookback window is 96, the MSE of the short-term meteorological time series prediction (prediction length is 96) is lower than 0.17, and the MSE of the long-term meteorological time series prediction (prediction length is 720) is lower than 0.33. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a network structure diagram of the meteorological time series prediction method of the present invention.

[0067] Figure 2 This is a factor loading quadrant analysis diagram of the channel grouping operation in the present invention.

[0068] Figure 3 Schematic diagram of the structure of the gated normalization module in the present invention.

[0069] Figure 4 Schematic diagram of the structure of the balanced attention mechanism in the present invention.

[0070] Figure 5 This is a visualization result diagram obtained by predicting an example of the present invention. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical implementation solution of the present invention is described and analyzed in more detail below in combination with the accompanying drawings and actual examples.

[0072] Embodiment 1:

[0073] A meteorological time series forecasting method that balances stationary and non-stationary information, such as Figure 1 As shown, the following steps are included:

[0074] Step S1, according to similar time series rules, the multivariate meteorological time series data are grouped into channels, that is, the principal component analysis method is used to determine the number of optimal groups by analyzing the variance explanation table and the factor loading quadrant diagram after dimensionality reduction, so as to group the Weather data set into channels, so that the model can focus on learning the unique characteristics of the variables in each group; Weather is a meteorological time series data set, which collects 21 meteorological factors such as humidity and air temperature collected every 10 minutes in 2020 by the weather station of the Max Planck Institute for Biogeochemistry in Germany, with a total of 52696 time steps;

[0075] The specific process of the channel grouping is as follows:

[0076] S1.1, calculate the variance explanation table for the Weather data set and evaluate the contribution of the principal components to the explanation of the variables;

[0077] S1.2, factor loading quadrant analysis was performed to reduce multiple principal components to a two-dimensional space and to present the spatial distribution of these principal components in the form of a quadrant diagram, so that the inherent structure and characteristics of the data could be intuitively understood;

[0078] S1.3, as shown in Table 1, when the number of principal components is 7, the characteristic root of the total variance explained is lower than 1 for the first time, and the contribution rate of variable explanation reaches 88.264%, which can ensure that the model can explain most of the variance while avoiding the noise problem that may be introduced when the characteristic root is lower than 1; Figure 2 The distribution of each variable is shown in the form of a quadrant diagram. The distance between variables reflects their similarity in the principal component space. The closer the distance, the higher the correlation between them. Figure 2 It can be seen that dividing the input factors into 7 groups is a more reasonable choice;

[0079] Among them, the characteristic root reflects the contribution of each principal component to the total variance. The larger the characteristic root, the more variance the principal component can explain. The variance explanation rate is a measure of the ability of a principal component to explain the variance of the total data. The cumulative variance explanation rate is the total explanatory power of the first few principal components. When the cumulative variance explanation rate is high, these principal components can well summarize the main information of the data.

[0080] Table 1 Variance explanation table

[0081]

[0082]

[0083] Step S2, perform aggregation operation on the grouped data to capture the time-related features in the time series data, that is, slide the convolution kernel on the time series data and calculate the weighted sum of the data in the window. The formula for the length of the aggregated time series is as follows:

[0084]

[0085] Where W is the lookback window size, F is the convolution kernel size, S is the step size, and P is padding. Note that the last value of the time series is used for padding to ensure that the padding value is as close as possible to the actual value of the sequence to avoid introducing too much noise.

[0086] Step S3: Since the meteorological time series obtained in step S2 may contain some extreme values ​​or outliers, which may affect the calculation of the mean and standard deviation in the normalization process, the proposed gated normalization is used to process the meteorological time series data. Figure 3 As shown, let the input of gated normalization be Where D is the number of channels, N is the length of the time series, and the specific process of the gated normalization is:

[0087] S3.1, integrate the global context information into each channel of the temporal feature, and introduce a trainable embedding weight α = [α 1 ,…,α D ], the formula is as follows:

[0088]

[0089] Among them, s d is the calculated global context vector, ∈ is a constant term to prevent differentiation at zero;

[0090] S3.2, perform L2 normalization on the global context vector, and we get:

[0091]

[0092] Where s = [s 1 ,…,s D ], when D is large, in order to prevent The scale is too small, add a scale factor

[0093] S3.3, adopt the gating mechanism to promote competition and cooperation between features, and design trainable weights γ = [γ 1 ,…,γ D ] and bias β = [β 1 ,…,β D ], when the weight γ dWhen it is positive, it will promote the characteristics of this channel to "compete" with the characteristics of other channels; when the weight γ d When it is negative, the features of this channel will be encouraged to "cooperate" with the features of other channels. The formula is as follows:

[0094]

[0095] in, The gating weight output by the gating mechanism;

[0096] S3.4, for y d Perform normalization and gate weights Multiplying, the formula is as follows:

[0097]

[0098] Among them, E(y d ) is y d The mean value of Var(y d ) is y d The variance of is the final output of gated normalization.

[0099] Step S4, embedding the normalized meteorological time series data obtained in step S3, using a linear projection W f and a learnable additive positional encoding W pos Mapping the features to the D-dimensional Transformer latent space and adding the order information of the time series data, we get The embedded data Send it to the codec to get the feature z (i) ; The balanced attention mechanism in the encoder reintroduces non-stationary information through a multi-layer perceptron, allowing the model to learn temporal dependencies from stationary and non-stationary sequences, thereby alleviating the over-stationary phenomenon; the balanced attention mechanism is as follows Figure 4 As shown in Figure 2, assuming that each variable of the time series has the same variance in normalization, the variance can be simplified to a single scalar σ. The specific process of the balanced attention mechanism is as follows:

[0100] S4.1, after the gated normalized data mapping, we get Q′ and K′:

[0101]

[0102] Among them, the data without gated normalization is obtained after linear mapping. is the gating weight after dimension change, Represents the mean of Q, K, and V in the time dimension;

[0103] S4.2, the attention mechanism first performs a dot product on Q′ and K′, the formula is as follows:

[0104]

[0105] in,

[0106] S4.3, the Softmax function is used for the dot product of Q and K. The specific process of the dot product of Q and K is as follows:

[0107] S4.3.1, the formula for the dot product of Q and K is as follows:

[0108]

[0109] in, This is equivalent to adding the same value to each element in the row.

[0110] S4.3.2, since Softmax operates on each column of the matrix, adding the same value to each element of a row has no effect on the result of Softmax and can be omitted, so it can be simplified to:

[0111]

[0112] Among them, σ 2 、(Kμ Q ) T It cannot be obtained from the gated normalized sequence;

[0113] S4.3.3, we use a simple multilayer perceptron to directly extract σ from the unnormalized statistics μ and σ of y. 2 、(Kμ Q ) T Learn separately, that is, learn τ=σ 2 , Δ=Kμ Q :

[0114] logτ=MLP(σ,y), Δ=MLP(μ,y)

[0115] S4.3.4, returns the optimized Q and K dot product formula:

[0116]

[0117] S4.4, the balanced attention mechanism performs attention calculation on the gated normalized sequence Q′, K′ and the unnormalized sequence V, the formula is as follows:

[0118]

[0119] Step S5, the feature z obtained in step S4 (i)Flattening is performed through the anti-normalization module, that is, a linear head is used to generate the prediction results of each group to eliminate the scale transformation introduced in the preprocessing stage, so that the prediction results are on the same scale as the original data.

[0120] Step S6, concatenating the prediction results of each group in step S5 to obtain the final prediction result.

[0121] Example 2

[0122] In order to demonstrate the effectiveness of the meteorological time series prediction method of the present invention, experiments were conducted on the Wearther dataset. All experiments were implemented on a single NVIDIA GeForce RTX 4090 graphics card using the PyTorch framework. The Adam optimizer was used to train the model. The learning rate was set to 10-4 and the batch size was set to 32. The model consisted of 6 encoder layers and 1 decoder layer. For the implementation of the balanced attention mechanism, a two-layer multilayer perceptron was used, in which the hidden layer dimension was set to 128. The loss functions used in the training process were mean square error (MSE) and mean absolute error (MAE), and lower MSE / MAE indicated better performance.

[0123] Table 2 Comparison of prediction results when the lookback window is 96 and the prediction length T∈{96,192,336,720}

[0124]

[0125] Table 2 shows the performance comparison of the meteorological time series prediction method of the present invention and other eleven benchmark models for predicting meteorological elements under the lookback window of 96 and the prediction length T∈{96,192,336,720}. The best result is indicated in bold and the second best result is indicated by underline. As can be seen from Table 2, the meteorological time series prediction method of the present invention has achieved the best performance in all MSE indicators, which significantly reflects its advantage in prediction accuracy. Compared with PatchTST, although the MAE at T=336 is slightly lower than that of PatchTST, the meteorological time series prediction method of the present invention has achieved a higher improvement in MSE, which means that the meteorological time series prediction method of the present invention is better in reducing prediction bias. Compared with iTransformer, although the MAE of the meteorological time series prediction method of the present invention is slightly lower at T=96, in terms of long-term time series prediction, the performance of the meteorological time series prediction method of the present invention in both MSE and MAE is better than that of iTransformer, which further proves the advantage of the meteorological time series prediction method of the present invention in capturing global features and long-term dependencies in time series data.

[0126] Example 3

[0127] When the lookback window is 96, the prediction effect of the meteorological time series prediction method of the present invention on different prediction lengths is tested. Figure 5 It can be seen that the meteorological time series prediction method of the present invention has shown excellent performance under various prediction lengths. The prediction results not only have a small error with the real sequence, but also can accurately reflect the actual time law of meteorological elements for both short-term and long-term predictions. It is worth noting that with the increase in the length of the prediction sequence, the meteorological time series prediction method of the present invention does not show any phenomenon of prediction effect collapse or other performance degradation, but can maintain a high prediction accuracy and stability, which fully demonstrates the unique advantages of the meteorological time series prediction method of the present invention in capturing and utilizing the periodic characteristics in meteorological data.

[0128] The above describes in detail the specific embodiments of the present invention, including its purpose, technical solutions and beneficial effects, but does not limit the scope of protection of the present invention. It should be understood by those skilled in the art that various modifications or variations based on the technical solutions of the present invention without creative labor still fall within the scope of protection of the present invention.

Claims

1. A meteorological time series prediction method that balances stationary and non-stationary information, characterized in that: The following steps are involved: S1: Obtain multivariate meteorological time series data, use the constructed channel grouping operation to group meteorological elements according to similar time series rules; S2: Perform aggregation operations on the grouped data to capture time-related features in the time series data; S3: Using the constructed gated normalization module, irrelevant or invalid features in the time series data are filtered based on the gated weights, while enhancing the correlation between valid features; S4: Using balanced attention-based encoder-decoder to introduce bias term to reintegrate non-stationary information; S5: Generate the prediction results of each group through the anti-normalization module; S6: Concatenate the prediction results of all groups to obtain the final prediction result.

2. The meteorological time series prediction method according to claim 1, characterized in that: The channel grouping operation in S1 utilizes the principal component analysis method to group meteorological elements with similar temporal regularities, that is, the number of groups is determined by analyzing the variance explanation table and the factor loading quadrant diagram after dimensionality reduction.

3. The meteorological time series prediction method according to claim 2, characterized in that: The S1 specifically includes: S1.1, calculate the variance explanation table and evaluate the contribution of the principal components to the explanation of the variables; S1.2, perform factor loading quadrant analysis to reduce multiple principal components to two-dimensional space and present the spatial distribution of principal components in the form of quadrant diagram; S1.3, determine the number of groups based on the variance explanation table and factor loading quadrant analysis.

4. The meteorological time series prediction method according to claim 1, characterized in that: The input of the gated normalization module described in S3 is D is the number of channels, N is the length of the time series, and the specific process of the gated normalization module is as follows: S3.1, integrate the global context information into each channel of the temporal feature, and introduce an embedding weight α = [α1,…,α D ], the formula is as follows: Among them, s d is the calculated global context vector, ∈ is a constant term to prevent differentiation at zero; S3.2, perform L2 normalization on the global context vector, and we get: Where s=[s1,…,s D ], when D is large, in order to prevent The scale is too small, add a scale factor S3.3, adopt the gating mechanism to promote competition and cooperation among features, and design the weight γ=[γ1,…,γ D ] and bias β=[β1,…,β D ], when the weight γ d When it is positive, it will promote the features of this channel to compete with the features of other channels; when the weight γ d When it is negative, the features of this channel will be encouraged to cooperate with the features of other channels. The formula is as follows: in, The gating weight output by the gating mechanism; S3.4, for y d Perform normalization and gate weights Multiplying, the formula is as follows: Among them, E(y d ) is y d The mean value of Var(y d ) is y d The variance of is the final output of gated normalization.

5. The meteorological time series prediction method according to claim 1, characterized in that: In S4, the balanced attention-based encoding and decoding assumes that each variable of the time series has the same variance in normalization, thereby simplifying the variance to a single scalar σ. The specific process of the balanced attention mechanism is as follows: S4.1, after gated and normalized data mapping, Q ′ , K ′ : Among them, the data without gate normalization is linearly mapped to obtain Q, K, is the gating weight after dimension change, Represents the mean of Q, K, and V in the time dimension; S4.2, the attention mechanism first ′ and K ′ Doing a dot multiplication, the formula is as follows: in, S4.3, use the Softmax function for the dot product of Q and K; S4.4, balanced attention mechanism for the gated normalized sequence Q ′ , K ′ And the unnormalized sequence V is used for attention calculation, the formula is as follows:

6. The meteorological time series prediction method according to claim 5, characterized in that: In step S4.3, the specific process of the dot product of Q and K is as follows: S4.3.1, the dot product formula of Q and K is as follows: in, This is equivalent to adding the same value to each element in the row; S4.3.2, since Softmax operates on each column of the matrix, adding the same value to each element of a row has no effect on the result of Softmax, so it is simplified to: Among them, σ 2 、(Kμ Q ) T It cannot be obtained from the gated normalized sequence; S4.3.3, we use a simple multilayer perceptron to directly extract σ from the unnormalized statistics μ and σ of y. 2 、(Kμ Q ) T Learn separately, that is, learn τ=σ 2 , Δ=Kμ Q : logτ=MLP(σ,y), Δ=MLP(μ,y) S4.3.4, returns the optimized Q and K dot product formula:

7. A prediction model based on the meteorological time series prediction method of balancing stationary and non-stationary information as claimed in claim 1, characterized in that: It includes the following modules: Channel grouping module, which groups multivariate meteorological time series data; Aggregation module to capture time-dependent features in time series data; The gated normalization module filters irrelevant or invalid features in the time series data based on the gated weights; Based on the balanced attention encoder-decoder module, a bias term is introduced to reintegrate non-stationary information to capture the temporal dependency in the time series; The denormalization module restores the original scale of the data and generates prediction results for each group; The splicing module splices the prediction results of all groups to obtain the final prediction results.

Citation Information

Patent Citations

  • Time sequence prediction method and system based on multi-scale fusion and attention mechanism

    CN115907203A

  • Short temporary rainfall prediction method based on multi-scale attention and convolution fusion

    CN118298222A

  • Transform-based long-time multivariable time sequence prediction method

    CN118364966A

Cited By

  • Power load prediction method and system based on frequency domain adaptive filter

    CN120197742A