A weather data prediction method based on sequence segmentation and frequency domain attention

Abstract

The present invention discloses a weather data prediction method based on sequence segmentation and frequency domain attention, including: S1: constructing a weather data set; S2: constructing a weather prediction model based on an informer model; importing weather data into the PatchTST local feature extraction module in the encoder to obtain the segmented local features; S3: importing the local features in step S2 into the DCT frequency domain attention module to obtain enhanced frequency domain features; importing the local features in step S2 into the multi-head sparse attention module at the same time to obtain sparse attention features; S4: the encoder feedforward layer splices the enhanced frequency domain features and sparse attention features in step S3 through learning factors, and obtains the first feedforward layer output through dimensional conversion; S5: importing the first feedforward layer output into the decoder to obtain the predicted weather data; S6: constructing a loss function, minimizing the loss function to optimize the model. The present invention can accurately capture the short-term fluctuations and long-term trends of weather data.

Description

Technical Field

[0001] The present invention belongs to the technical field of weather forecasting, and in particular relates to a weather data forecasting method based on sequence segmentation and frequency domain attention. Background Art

[0002] Weather forecasting plays an important role in meteorology, energy management, agricultural planning and other fields. In particular, accurate long-sequence weather data forecasting can help various industries better cope with weather changes. However, traditional time series forecasting methods (such as ARIMA, LSTM, etc.) often face problems such as high computational complexity and difficulty in capturing long-distance dependencies when processing long-sequence weather data, resulting in low prediction accuracy. In recent years, the Transformer model has gradually been used in time series forecasting due to its global dependency modeling ability of its self-attention mechanism. However, the standard Transformer model faces the problem of low computational efficiency in long-sequence data because its self-attention mechanism has a low efficiency when the sequence length is The computational complexity is , which limits its application in long time series prediction tasks.

[0003] To solve this problem, Informer introduced a sparse attention mechanism (ProbSparse Attention), which greatly reduced the computational complexity and made the processing of long-sequence weather data more efficient. However, when extracting global information, the encoder of Informer may ignore the local characteristics and frequency domain information of weather data, which in turn affects the accuracy of the prediction. Weather data has significant local fluctuations and long-term trend characteristics. Global modeling based on sparse attention alone is difficult to fully capture these complex multi-level information, resulting in limited prediction results.

[0004] To address this problem, in recent years, many improved methods based on deep learning have been proposed, attempting to introduce local features and frequency domain feature extraction in long sequence prediction. Patch segmentation methods are gradually used for time series segmentation analysis. For example, the PatchTST module divides a long sequence into several subsequences to capture local change characteristics. In addition, frequency domain analysis can effectively reduce high-frequency noise interference by extracting low-frequency trend information, thereby retaining the main trend of the data. However, existing models often lack the ability to adaptively fuse these features, making it difficult to achieve high accuracy and robustness in long sequence prediction. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention introduces the PatchTST local feature extraction module, the DCT frequency domain attention module and the residual connection with learnable parameters on the basis of the Informer model, so as to improve the model's ability to extract local features and trend information of weather data, thereby realizing efficient and accurate long-sequence weather forecasting.

[0006] A weather data prediction method based on sequence segmentation and frequency domain attention, comprising:

[0007] Step S1: construct a weather data set, the data set includes a number of weather data;

[0008] Step S2: Construct a weather prediction model based on the informer model, which includes a serially superimposed encoder and decoder, and the encoder includes a PatchTST local feature extraction module, a DCT frequency domain attention module, a multi-head sparse attention module, and an encoder feedforward layer; import weather data into the PatchTST local feature extraction module in the encoder, divide the weather data in a long sequence format into several subsequences to capture local features, and obtain the segmented local features;

[0009] Step S3: import the local features in step S2 into the DCT frequency domain attention module, apply discrete cosine transform to extract low-frequency information in the frequency domain, further filter high-frequency noise, and obtain enhanced frequency domain features; at the same time, import the local features in step S2 into the multi-head sparse attention module to obtain sparse attention features;

[0010] Step S4: The encoder feedforward layer concatenates the enhanced frequency domain features and sparse attention features in step S3 through the learning factor, and obtains the first feedforward layer output through dimension conversion;

[0011] Step S5: import the output of the first feed-forward layer into the decoder to obtain the output of the second feed-forward layer, i.e., the predicted weather data;

[0012] Step S6: Construct a loss function and minimize the loss function to optimize the model.

[0013] Furthermore, the weather data in step S1 is preprocessed, specifically:

[0014] The weather data in step S1 includes 12 feature dimensions; the feature dimensions include:

[0015] Date, visibility, dry bulb temperature, degrees Fahrenheit, degrees Celsius, dew point temperature, relative humidity, wind speed, wind direction, station pressure, altimeter pressure, and wet bulb temperature;

[0016] Perform mean normalization on feature dimensions.

[0017] Furthermore, the weather data is imported into the PatchTST local feature extraction module in the encoder in step S2, and the weather data in the long sequence format is divided into several subsequences to capture local features and obtain the local features after segmentation; specifically:

[0018] The weather data in long sequence format is split into time series to obtain several subsequences, which are expressed as:

[0019] ;

[0020] ;

[0021] ;

[0022] in, represents the i-th subsequence extracted from the weather data X, i.e., the local feature; Indicates the number of subsequences after division; represents the time step; Indicates the sliding step length; Indicates the total length of weather data in long sequence format; Represents the set of real numbers.

[0023] Furthermore, in step S3, the local feature vector in step S2 is imported into the DCT frequency domain attention module, and the discrete cosine transform is applied to extract the low-frequency information in the frequency domain, and the high-frequency noise is further filtered out to obtain the enhanced frequency domain feature; specifically:

[0024] Step S31A: Convert the time domain format into frequency domain coefficients, expressed as:

[0025] ;

[0026] in, Indicates frequency domain coefficients;

[0027] Step S32A: Setting a threshold to filter noise, expressed as:

[0028] ;

[0029] in, Represents the frequency threshold, which is used to distinguish between low-frequency and high-frequency parts. Represents the frequency domain coefficients after denoising, only the frequency domain coefficients below the frequency threshold are retained, and the high-frequency coefficients are set to zero;

[0030] Step S33A: After obtaining the filtered frequency domain set, inverse DCT restoration is performed, which is expressed as:

[0031] ;

[0032] in, represents the time series set after frequency domain filtering, that is, enhancing the frequency domain features;

[0033] Step S3 imports the local feature vector in step S2 into the sparse attention module to obtain the sparse attention feature, specifically:

[0034] Step S31B: The importance of local features is screened through the sparse attention mechanism, which is expressed as:

[0035]

[0036] ;

[0037] ;

[0038] ;

[0039] in, represents the attention score, is the query vector, is the key vector, is a value vector, , and Respectively represent the preset weight matrices;

[0040] In each query, keep the attention score The first u maximum values, the other attention scores The value of is set to zero; finally, the dependency relationship between each local feature is obtained through sparse attention calculation to obtain the sparse attention feature , expressed as:

[0041] ;

[0042] in, represents the sparse multi-head attention mechanism, represents the normalization function; Represents dimension.

[0043] Furthermore, step S4 is specifically as follows:

[0044] Obtaining learning factors through gradient calculation , expressed as:

[0045] ;

[0046] in, represents the loss function, represents the learning rate;

[0047] The enhanced frequency domain features and sparse attention features in step S3 are transformed into Splicing, and then passing through the encoder feedforward layer The layer performs dimension conversion to obtain the output of the first feedforward layer ; expressed as:

[0048] ;

[0049] in, Represents a dimension transformation operation.

[0050] Further, the process of step S5 is expressed as:

[0051] ;

[0052] in, represents the output of the decoder, that is, the output of the second feedforward layer, Represents a splicing operation, represents the sequence processed by the encoder, that is, the output of the first feedforward layer, Indicates additional input, represents the set of real numbers, express Length, represents the length of the additional input, express Dimensions;

[0053] The decoder in step S5 includes a decoder multi-head sparse attention module and a decoder feed-forward layer; specifically:

[0054] The multi-head sparse attention module processes the output of the first feed-forward layer, expressed as:

[0055] ;

[0056] ;

[0057] ;

[0058] ;

[0059] in, represents the output of the multi-head sparse attention module, is the decoder query vector, is the decoder key vector, is the decoder value vector, , and Both represent the weight matrix of the decoder's multi-head sparse attention module;

[0060] The decoder feedforward layer transforms the output of the multi-head sparse attention module and obtains the output of the second feedforward layer, which is expressed as:

[0061] ;

[0062] in, Represents a dimension transformation operation.

[0063] Furthermore, the loss function in step S6 is specifically:

[0064] ;

[0065] ;

[0066] in, is the true value of the weather data, Represents the predicted value of weather data. represents the mean square error loss function, represents the average error loss function, Represents the total number of samples.

[0067] Compared with the prior art, the present invention has the following beneficial effects:

[0068] 1) The present invention combines the PatchTST local feature extraction module and the DCT frequency domain attention module to significantly enhance the temporal feature extraction effect of the model. The PatchTST local feature extraction module helps the model capture local features in the short term by dividing the long sequence format weather data into subsequences, thereby improving the ability to identify data details and short-term fluctuations; at the same time, the DCT frequency domain attention module converts the time domain features of the weather data into frequency domain features, focuses on the low-frequency components of the data, extracts the long-term trends in the data and reduces the interference of high-frequency noise. Based on this, the short-term fluctuations and long-term trends of weather data are accurately captured. Short-term fluctuations include fluctuations in temperature, humidity, wind speed, etc. on the same day or in a short period of time, and long-term trends include low-frequency trend components in weather data, such as periodic temperature rise and fall trends. This fusion of short-term and long-term features can improve the comprehensive understanding of weather data; enhance the recognition of extreme weather events, for example, under extreme weather conditions, such as typhoons, rainstorms or heat waves, the fluctuations of data may be particularly severe, but the extracted local features can identify sharp changes in a short period of time under extreme weather conditions; frequency domain information can provide overall trends, helping the model to more effectively distinguish abnormal weather changes from normal seasonal trends.

[0069] 2) The model provided by the present invention has better prediction performance and generalization ability, that is, it has better trend recognition for complex weather and long-term uncertain complex weather conditions than general models, and due to the diverse characteristics of the data set, it has better prediction effects for the prediction of various weather indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The present invention is a flowchart of the steps of a weather data prediction method based on sequence segmentation and frequency domain attention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Reference Figure 1 , a weather data prediction method based on sequence segmentation and frequency domain attention, in one example, includes:

[0073] Step S1: construct a weather data set, the data set includes a number of weather data;

[0074] Step S2: Construct a weather prediction model based on the informer model, which includes a serially superimposed encoder and decoder, and the encoder includes a PatchTST local feature extraction module, a DCT frequency domain attention module, a multi-head sparse attention module, and an encoder feedforward layer; import weather data into the PatchTST local feature extraction module in the encoder, divide the weather data in a long sequence format into several subsequences to capture local features, and obtain the segmented local features;

[0075] Step S3: import the local features in step S2 into the DCT frequency domain attention module, apply discrete cosine transform to extract low-frequency information in the frequency domain, further filter high-frequency noise, and obtain enhanced frequency domain features; at the same time, import the local features in step S2 into the multi-head sparse attention module to obtain sparse attention features;

[0076] Step S4: The encoder feedforward layer concatenates the enhanced frequency domain features and sparse attention features in step S3 through the learning factor, and obtains the first feedforward layer output through dimension conversion;

[0077] Step S5: import the output of the first feed-forward layer into the decoder to obtain the output of the second feed-forward layer, i.e., the predicted weather data;

[0078] Step S6: Construct a loss function and minimize the loss function to optimize the model.

[0079] Furthermore, in one example, the weather data in step S1 is preprocessed, specifically:

[0080] The weather data in step S1 includes 12 feature dimensions; the feature dimensions include:

[0081] Date, visibility, dry bulb temperature, degrees Fahrenheit, degrees Celsius, dew point temperature, relative humidity, wind speed, wind direction, station pressure, altimeter pressure, and wet bulb temperature;

[0082] Perform mean normalization on feature dimensions.

[0083] Further, in one example, the weather data is imported into the PatchTST local feature extraction module in the encoder in step S2, and the weather data in the long sequence format is divided into several subsequences to capture local features, and the local features after segmentation are obtained; specifically:

[0084] The weather data in long sequence format is split into time series to obtain several subsequences, which are expressed as:

[0085] ;

[0086] ;

[0087] ;

[0088] in, represents the i-th subsequence extracted from the weather data X, i.e., the local feature; Indicates the number of subsequences after division; represents the time step; Indicates the sliding step length; Indicates the total length of weather data in long sequence format; Represents the set of real numbers.

[0089] Further, in one example, in step S3, the local feature vector in step S2 is imported into the DCT frequency domain attention module, discrete cosine transform is applied to extract low-frequency information in the frequency domain, and high-frequency noise is further filtered out to obtain enhanced frequency domain features; specifically:

[0090] Step S31A: Convert the time domain format into frequency domain coefficients, expressed as:

[0091] ;

[0092] in, Indicates frequency domain coefficients;

[0093] Step S32A: Setting a threshold to filter noise, expressed as:

[0094] ;

[0095] in, Represents the frequency threshold, which is used to distinguish between low-frequency and high-frequency parts. Represents the frequency domain coefficients after denoising, only the frequency domain coefficients below the frequency threshold are retained, and the high-frequency coefficients are set to zero;

[0096] Step S33A: After obtaining the filtered frequency domain set, inverse DCT restoration is performed, which is expressed as:

[0097] ;

[0098] in, represents the time series set after frequency domain filtering, that is, enhancing the frequency domain features;

[0099] Step S3 imports the local feature vector in step S2 into the sparse attention module to obtain the sparse attention feature, specifically:

[0100] Step S31B: The importance of local features is screened through the sparse attention mechanism, which is expressed as:

[0101]

[0102] ;

[0103] ;

[0104] ;

[0105] in, represents the attention score, is the query vector, is the key vector, is a value vector, , and Respectively represent the preset weight matrices;

[0106] In each query, keep the attention score The first u maximum values, the other attention scores The value of is set to zero; finally, the dependency relationship between each local feature is obtained through sparse attention calculation to obtain the sparse attention feature , expressed as:

[0107] ;

[0108] in, represents the sparse multi-head attention mechanism, represents the normalization function; Represents dimension.

[0109] Furthermore, in one example, step S4 is specifically as follows:

[0110] Obtaining learning factors through gradient calculation , expressed as:

[0111] ;

[0112] in, represents the loss function, represents the learning rate;

[0113] The enhanced frequency domain features and sparse attention features in step S3 are transformed into Splicing, and then passing through the encoder feedforward layer The layer performs dimension conversion to obtain the output of the first feedforward layer ; expressed as:

[0114] ;

[0115] in, Represents a dimension transformation operation.

[0116] Further, in one example, the process of step S5 is expressed as:

[0117] ;

[0118] in, represents the output of the decoder, that is, the output of the second feedforward layer, Represents a splicing operation, represents the sequence processed by the encoder, that is, the output of the first feedforward layer, Indicates additional input, represents the set of real numbers, express Length, represents the length of the additional input, express Dimensions;

[0119] The decoder in step S5 includes a decoder multi-head sparse attention module and a decoder feed-forward layer; specifically:

[0120] The multi-head sparse attention module processes the output of the first feed-forward layer, expressed as:

[0121] ;

[0122] ;

[0123] ;

[0124] ;

[0125] in, represents the output of the multi-head sparse attention module, is the decoder query vector, is the decoder key vector, is the decoder value vector, , and Both represent the weight matrix of the decoder's multi-head sparse attention module;

[0126] The decoder feedforward layer transforms the output of the multi-head sparse attention module and obtains the output of the second feedforward layer, which is expressed as:

[0127] ;

[0128] in, Represents a dimension transformation operation.

[0129] Furthermore, in one example, the loss function in step S6 is specifically:

[0130] ;

[0131] ;

[0132] in, is the true value of the weather data, Represents the predicted value of weather data. represents the mean square error loss function, represents the average error loss function, Represents the total number of samples.

[0133] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A weather data prediction method based on sequence segmentation and frequency domain attention, characterized in that: include: Step S1: construct a weather data set, the data set includes a number of weather data; Step S2: Construct a weather prediction model based on the informer model, which includes a serially superimposed encoder and decoder, and the encoder includes a PatchTST local feature extraction module, a DCT frequency domain attention module, a multi-head sparse attention module, and an encoder feedforward layer; import weather data into the PatchTST local feature extraction module in the encoder, divide the weather data in a long sequence format into several subsequences to capture local features, and obtain the segmented local features; Step S3: import the local features in step S2 into the DCT frequency domain attention module, apply discrete cosine transform to extract low-frequency information in the frequency domain, further filter high-frequency noise, and obtain enhanced frequency domain features; at the same time, import the local features in step S2 into the multi-head sparse attention module to obtain sparse attention features; Step S4: The encoder feedforward layer concatenates the enhanced frequency domain features and sparse attention features in step S3 through the learning factor, and obtains the first feedforward layer output through dimension conversion; Step S5: import the output of the first feed-forward layer into the decoder to obtain the output of the second feed-forward layer, i.e., the predicted weather data; Step S6: Construct a loss function and minimize the loss function to optimize the model.

2. A weather data prediction method based on sequence segmentation and frequency domain attention according to claim 1, characterized in that: The weather data in step S1 is preprocessed, specifically: The weather data in step S1 includes 12 feature dimensions; Feature dimensions include: Date, visibility, dry bulb temperature, degrees Fahrenheit, degrees Celsius, dew point temperature, relative humidity, wind speed, wind direction, station pressure, altimeter pressure, and wet bulb temperature; Perform mean normalization on feature dimensions.

3. The weather data prediction method based on sequence segmentation and frequency domain attention according to claim 1 is characterized in that: In step S2, the weather data is imported into the PatchTST local feature extraction module in the encoder, and the weather data in the long sequence format is divided into several subsequences to capture local features and obtain the local features after segmentation; specifically: The weather data in long sequence format is split into time series to obtain several subsequences, which are expressed as: ； ； ； in, represents the i-th subsequence extracted from the weather data X, i.e., the local feature; Indicates the number of subsequences after division; represents the time step; Indicates the sliding step length; Indicates the total length of weather data in long sequence format; Represents the set of real numbers.

4. The weather data prediction method based on sequence segmentation and frequency domain attention according to claim 3 is characterized in that: Step S3 imports the local feature vector in step S2 into the DCT frequency domain attention module, applies discrete cosine transform to extract low-frequency information in the frequency domain, further filters out high-frequency noise, and obtains enhanced frequency domain features; Specifically: Step S31A: Convert the time domain format into frequency domain coefficients, expressed as: ； in, Indicates frequency domain coefficients; Step S32A: Setting a threshold to filter noise, expressed as: ； in, Represents the frequency threshold, which is used to distinguish between low-frequency and high-frequency parts. Represents the frequency domain coefficients after denoising, only the frequency domain coefficients below the frequency threshold are retained, and the high-frequency coefficients are set to zero; Step S33A: After obtaining the filtered frequency domain set, inverse DCT restoration is performed, which is expressed as: ； in, represents the time series set after frequency domain filtering, that is, enhancing the frequency domain features; Step S3 imports the local feature vector in step S2 into the sparse attention module to obtain the sparse attention feature, specifically: Step S31B: The importance of local features is screened through the sparse attention mechanism, which is expressed as: ； ； ； in, represents the attention score, is the query vector, is the key vector, is a value vector, , and Respectively represent the preset weight matrices; In each query, keep the attention score The first u maximum values, the other attention scores The value of is set to zero; finally, the dependency relationship between each local feature is obtained through sparse attention calculation to obtain the sparse attention feature , expressed as: ； in, represents the sparse multi-head attention mechanism, represents the normalization function; Represents dimension.

5. The weather data prediction method based on sequence segmentation and frequency domain attention according to claim 4 is characterized in that: Step S4 is specifically as follows: Obtaining learning factors through gradient calculation , expressed as: ； in, represents the loss function, represents the learning rate; The enhanced frequency domain features and sparse attention features in step S3 are transformed into Splicing, and then passing through the encoder feedforward layer The layer performs dimension conversion to obtain the output of the first feedforward layer ; expressed as: ； in, Represents a dimension transformation operation.

6. A weather data prediction method based on sequence segmentation and frequency domain attention according to claim 5, characterized in that: The process of step S5 is expressed as: ； in, represents the output of the decoder, that is, the output of the second feedforward layer, Represents a splicing operation, represents the sequence processed by the encoder, that is, the output of the first feedforward layer, Indicates additional input, represents the set of real numbers, express Length, represents the length of the additional input, express Dimensions; The decoder in step S5 includes a decoder multi-head sparse attention module and a decoder feed-forward layer; specifically: The multi-head sparse attention module processes the output of the first feed-forward layer, expressed as: ； ； ； ； in, represents the output of the multi-head sparse attention module, is the decoder query vector, is the decoder key vector, is the decoder value vector, , and Both represent the weight matrix of the decoder's multi-head sparse attention module; The decoder feedforward layer transforms the output of the multi-head sparse attention module and obtains the output of the second feedforward layer, which is expressed as: ； in, Represents a dimension transformation operation.

7. The weather data prediction method based on sequence segmentation and frequency domain attention according to claim 1, characterized in that: The loss function in step S6 is specifically: ； ； in, is the true value of the weather data, Represents the predicted value of weather data. represents the mean square error loss function, represents the average error loss function, Represents the total number of samples.

Images