A long time series prediction method based on frequency domain cross enhancement

Abstract

The application discloses a long-time sequence prediction method based on frequency domain cross enhancement, and relates to the field of time sequence prediction, which comprises the following steps: reversible instance normalization is performed on an input sequence, and the sequence is mapped to a high-dimensional semantic space through dimension expansion; then, the sequence is transformed to a frequency domain by using a discrete Fourier transform, and is decomposed into a real part sequence and an imaginary part sequence; a real part structure enhancement module and an imaginary part structure enhancement module are introduced into the real part and the imaginary part respectively, so that the structural perception ability and feature diversity of an attention matrix are improved; a bidirectional cross attention mechanism is further constructed to realize information interaction between the real part and the imaginary part; then, the sequence is recovered to a time domain through an inverse Fourier transform, and a prediction result is obtained by combining a residual connection, a linear mapping and inverse normalization. The application can effectively alleviate the problems of low rank, excessive focus and insufficient structure expression of traditional self-attention in frequency domain modeling, enhance the modeling ability of long-distance dependence, and improve the precision and generalization performance of multivariate long-time sequence prediction.

Description

Technical Field

[0001] This invention relates to a long-term series prediction method based on frequency domain cross-enhancement, belonging to the field of time series prediction. Background Technology

[0002] Time series data analysis is a crucial research area in statistics, data science, and machine learning. Its core task is to model and predict future trends based on historical observation data. Time series forecasting techniques are widely used in many key fields such as energy systems, financial markets, weather forecasting, traffic management, and industrial manufacturing, and are of great significance for improving system operating efficiency and supporting decision-making.

[0003] Traditional time series forecasting methods primarily rely on statistical modeling techniques, such as autoregressive models (AR), moving average models (MA), and autoregressive integrated moving average models (ARIMA). These methods infer future trends by modeling the statistical properties of time series, such as mean, variance, and autocorrelation, and have advantages such as simple structure and strong interpretability. However, because they are based on linear and stationary assumptions, they struggle to effectively characterize the prevalent nonlinear relationships, non-stationary dynamics, and complex coupling relationships between multiple variables in the real world. Therefore, their predictive performance is significantly limited in complex scenarios.

[0004] With the development of deep learning technology, time series prediction methods based on neural networks have gradually become mainstream. Convolutional Neural Networks (CNNs) can effectively capture local patterns in time series through their local receptive fields, while Temporal Convolutional Networks (TCNs) expand their receptive fields by dilating convolutions, enhancing their ability to model long-distance dependencies. Recurrent Neural Networks (RNNs) and their variants (such as LSTM and GRU) can capture the temporal dependencies of sequences through recursive structures. These methods have improved the ability to model time series to some extent, but they still have limitations. For example, CNN-type models have difficulty handling long-term dependencies, while RNN-type models face problems such as vanishing or exploding gradients, limiting their performance in long-sequence prediction tasks.

[0005] Transformer models exhibit significant advantages in long-term sequence prediction tasks due to their global modeling capabilities based on self-attention mechanisms. These models can capture dependencies between arbitrary positions in a sequence through parallel computation mechanisms, effectively mitigating the information decay problem of traditional models in long-term sequence modeling. However, the computational complexity of attention in a standard Transformer is [insert value here]. This leads to significant computational and storage overhead when processing long-running series. To address this, researchers have proposed a series of improved methods, such as sparse attention (e.g., LogTrans, Informer), hierarchical structures (e.g., Pyraformer), and block modeling (e.g., PatchTST), to reduce computational complexity and improve efficiency.

[0006] In recent years, with the continuous development of time series feature modeling methods, researchers have gradually shifted from traditional time-domain modeling to frequency-domain modeling. Among these, frequency-domain modeling methods based on Fourier transform map time-domain sequences to the frequency domain, allowing for explicit characterization of the periodic structure, frequency components, and energy distribution within the sequence, thus providing a new perspective for the structural analysis of complex time series. Existing research shows that time series often exhibit more compact energy distribution characteristics in the frequency domain and possess natural global representation capabilities, which helps capture long-distance dependencies and multi-scale periodic patterns, and is of great significance for long-term series prediction tasks.

[0007] Despite the promising potential of frequency domain modeling in time series forecasting, existing methods still suffer from several key shortcomings that limit further performance improvements. First, most current frequency domain methods rely primarily on linear mappings or simple multilayer perceptrons to model spectral features, lacking a deep understanding of the complex structural relationships within the spectrum and failing to effectively uncover the dependency patterns between different frequency components. Second, in complex spectral representations, the real and imaginary parts carry different structural information, but existing methods typically model them independently or simply concatenate them, lacking an effective information exchange mechanism. This makes it difficult to fully utilize the implicit coupling relationships within the spectrum, limiting the model's ability to express complex dynamic patterns.

[0008] Furthermore, in real-world time series, the spectrum typically exhibits significant sparsity, meaning that most energy is concentrated in a few dominant frequency components. This characteristic leads to an "over-focusing" phenomenon in the frequency domain of attention-based models, where attention weights are concentrated on a few high-energy frequencies while ignoring other potentially important frequency components, thus reducing the model's ability to model multi-frequency band dependencies. Simultaneously, traditional attention mechanisms often exhibit low-rank weight distributions when dealing with such sparse spectra, further limiting the model's expressive power. Therefore, it is necessary to propose a novel frequency domain time series prediction method. By constructing a structure-enhanced attention mechanism and introducing a cross-component interactive modeling strategy, this method can fully utilize global information in the frequency domain to improve the model's ability to express complex spectral structures and multivariate dependencies, thereby achieving more accurate and robust time series prediction. Summary of the Invention

[0009] This invention addresses the problems of low-rank attention matrix, over-focusing of weights, and insufficient expressive power of spectral structure in existing time series prediction methods during frequency domain modeling. It proposes a long-term time series prediction method based on frequency domain cross-enhancement. This method effectively models complex periodic structures, phase changes, and long-distance dependencies in time series by constructing a frequency domain structure enhancement modeling mechanism and a cross-component interaction strategy, thereby improving prediction accuracy and model robustness. First, the input multivariate time series undergoes reversible instance normalization. Statistical features are calculated along the time dimension for each sample and standardized, effectively reducing the impact of distribution differences and non-stationarity between different samples, thus improving the stability and generalization ability of model training. Second, dimensionality expansion maps the original two-dimensional time series to a high-dimensional semantic space to enhance feature expressive power and provide richer semantic information for subsequent frequency domain modeling. Subsequently, a discrete Fourier transform is used to map the time-domain sequence to the frequency domain space, obtaining a complex spectral representation, which is then decomposed into real and imaginary parts. Further, real and imaginary structure enhancement modules are constructed separately to independently model the two components of the spectrum. In each branch, a structure-enhanced attention mechanism is introduced to improve the traditional attention computation process, enhancing the expressive power of the attention matrix from two aspects: nonlinear enhancement and structural relationship modeling. This allows the model to more fully characterize the complex dependencies between different components in the spectrum. Based on this, a bidirectional cross-attention mechanism is designed to establish a two-way information exchange channel between the real and imaginary parts. This mechanism constructs a cross-branch attention computation path, enabling the two components to perceive each other's feature information, thereby achieving collaborative modeling between different components in the spectrum and enhancing the model's ability to express complex frequency structures and dynamic change patterns. After completing the frequency domain modeling, the cross-enhanced spectral representation is mapped back to the time domain using an inverse discrete Fourier transform, restoring the time domain feature representation. Subsequently, the reconstructed result is fused with the input features through residual connections, introducing frequency domain enhanced features while preserving the original information, and generating the target prediction result through a linear mapping layer. Finally, the prediction output is subjected to inverse instance normalization, restoring it to the original numerical scale based on the statistical information recorded in the aforementioned normalization stage, yielding the final time series prediction result.

[0010] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0011] A long-time series prediction method based on frequency domain cross-enhancement, the method comprising the following steps:

[0012] Step S1: Obtain the real-world dataset for time series prediction. The dataset includes ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and Traffic datasets, and divide them into training, validation, and test sets according to a preset ratio. In this embodiment, a 7:2:1 ratio is preferred. Further, based on the requirements of the long-term series prediction task, construct historical observation windows and future prediction windows for the original time series. Let the length of the input historical window be... The predicted length is The number of variables is Then a single input sample can be represented as:

[0013]

[0014] The corresponding prediction objective can be expressed as:

[0015]

[0016] in, This represents the multivariate observation at time step t. A sliding window approach can be used to construct a large number of training samples from the original sequence for subsequent model training and validation.

[0017] Step S2: Construct a long-term sequence prediction network model based on frequency domain cross-enhancement; the network model includes: a frequency domain preprocessing module; a real part structure enhancement module; an imaginary part structure enhancement module; a bidirectional interactive attention module; a temporal reconstruction and fusion module; and a prediction output module. The frequency domain preprocessing module is used to normalize, expand the dimension, and transform the frequency domain of the input sequence; the real part structure enhancement module and the imaginary part structure enhancement module are used to perform structural enhancement modeling on the two components of the complex spectrum, respectively; the bidirectional interactive attention module is used to establish a bidirectional information interaction relationship between the two components; the temporal reconstruction and fusion module is used to restore the frequency domain representation to the temporal domain and fuse it with the front-end features; and the prediction output module is used to generate the final output result at the target prediction length.

[0018] Step S3: Input time series The data is fed into the frequency domain preprocessing module. This module comprises a reversible instance normalization unit, a dimension expansion unit, and a discrete Fourier transform unit. Its function is to reduce the influence of distribution drift between input samples, enhance the semantic expressiveness of input features, and convert time-domain information to the frequency domain, providing a foundation for subsequent frequency domain modeling. First, reversible instance normalization is performed on the time series, calculating the mean and standard deviation for each variable along the time dimension to normalize the input sequence. Second, dimension expansion is used to map the normalized two-dimensional time series tensor into a three-dimensional high-dimensional representation. Finally, a discrete Fourier transform is performed on the dimension-expanded time series along the time dimension to obtain a frequency domain complex spectrum representation, which is then decomposed into a sequence of real parts. and imaginary part sequence ;

[0019] Step S4: Convert the real part sequence obtained in step S3 into a sequence of real parts. The input real part structure enhancement module is used to perform structure enhancement modeling on the real part of the spectrum, thereby extracting the global frequency structure information contained in the real part and improving the model's ability to express spectral intensity distribution and dominant frequency modes. Specifically, for real part sequences... The extended and temporal dimensions are flattened, and then mapped to a unified latent space dimension through a learnable linear projection. Thus, input of attention is obtained. Secondly, the computational structure enhances attention, and the intermediate output is obtained by combining residual connections and layer normalization. Finally, the output of the real part structure enhancement module is obtained through the feedforward neural network, which is the real part spectrum representation.

[0020] Step S5: The imaginary part sequence obtained in step S3... The input imaginary part structure enhancement module flattens the extended and time dimensions and maps them to a unified hidden space dimension through a learnable linear projection. Thus, input of attention is obtained. Secondly, the computational structure enhances attention, and the intermediate output is obtained by combining residual connections and layer normalization. Finally, the output of the imaginary part structure enhancement module is obtained through a feedforward neural network, which is the imaginary part spectrum representation.

[0021] Step S6: Input the real part spectral representation and the imaginary part spectral representation into the bidirectional interactive attention module to establish the interaction relationship between the real part and the imaginary part, and obtain the real part representation and the imaginary part representation after cross-enhancement; restore the real part representation and the imaginary part representation to the tensor structure corresponding to the frequency domain input through the projection matrix, and reconstruct the complex spectrum;

[0022] Step S7: The frequency domain complex spectrum representation obtained in step S6 is fed into the time domain reconstruction and fusion module. This module mainly includes two processes: inverse discrete Fourier transform and time domain residual fusion. Specifically, the complex spectrum representation is remapped back to the time domain through inverse discrete Fourier transform, and the basic semantic information of the input stage is preserved through residual connection, thereby obtaining a fused representation that combines the global structure of the frequency domain and the original time domain features;

[0023] Step S8: The prediction output module maps the high-dimensional time domain representation to the output result on the target prediction length, and restores the original numerical scale through inverse normalization, finally generating the prediction sequence;

[0024] Step S9: Calculate the total loss function of the long-term series prediction model based on frequency domain crosstalk enhancement. And complete the training of the time series prediction network model;

[0025] Step S10: Input the test set time series data into the trained time series prediction network model to obtain the final time series prediction results.

[0026] Furthermore, the specific steps of step S3 are as follows:

[0027] Step S3-1: Process the input time series ,in, This represents the observation at time T, where N is the number of channels and T is the sequence length. Given the real number field, calculate the mean along the time dimension. and standard deviation The specific formula is as follows:

[0028]

[0029] in, This represents the mean of the j-th channel throughout the entire historical window.

[0030]

[0031] in, This represents the standard deviation of the j-th channel. To prevent division by zero of extremely small constants, Let be the observation value of channel j at time t. Then, the input time series is normalized using the mean and standard deviation mentioned above to obtain the normalized sequence. :

[0032]

[0033] Through the above processing, the distribution differences between different samples and different time periods can be effectively reduced without destroying the original time series structure, thereby improving the stability of subsequent model building. Since the normalization process is reversible, the original numerical space can be restored at the output using the corresponding statistics.

[0034] Step S3-2: Perform a dimensionality expansion operation on the normalized time series to improve the expressive power of temporal features. Specifically, a learnable projection vector is introduced. Perform dimension mapping on the input:

[0035]

[0036] in, It is a learnable projection vector, d is the extended dimension, and the output is This process maps the original two-dimensional time series representation to a three-dimensional high-dimensional semantic space, enabling each variable to obtain richer feature representations at each time step, thereby providing stronger semantic support for subsequent frequency domain transformation and frequency domain structure modeling.

[0037] Step S3-3: Perform dimensional expansion on the high-dimensional time series. Performing a discrete Fourier transform along the time dimension yields the complex spectrum representation in the frequency domain. Its expression is:

[0038]

[0039] in, Let represent the complex number representation of the k-th frequency component, e denote the exponential function, and j be the imaginary unit. Further, the obtained complex spectrum is decomposed into a sequence of real parts and a sequence of imaginary parts:

[0040]

[0041] in, These represent the real part sequence and the imaginary part sequence of the spectrum, respectively. Through the Discrete Fourier Transform, long-term dependencies and periodic patterns in the time domain can be converted into explicit frequency component representations in the frequency domain, enabling more effective modeling in the global frequency space.

[0042] Furthermore, the specific steps of step S4 are as follows:

[0043] First, for the real part sequence We flatten the extended and time dimensions, and then map them uniformly to the latent space using a learnable linear mapping, resulting in:

[0044]

[0045] in, To hide the feature dimension, the query matrix, key matrix, and value matrix are then constructed using the projection matrix:

[0046]

[0047] in, Both are projection matrices. Further, the standard attention similarity matrix is ​​calculated and enhanced using a power function nonlinearity:

[0048]

[0049] in, Represents a symbolic function. For learnable exponent parameters, , To scale the factor. Query matrix. Bond matrix Mapping each feature to one of the two sub-controls creates two parallel relationship paths. , Based on this, two types of structural relation matrices are constructed respectively:

[0050]

[0051] in, Used to describe a consistent relationship in the same direction. This is used to describe complementary interaction relationships. To achieve adaptive adjustment of the two types of relationships, a learnable gating matrix is ​​introduced. And an enhanced attention matrix is ​​formed through weighted fusion:

[0052]

[0053] in, It is an activation function. This indicates element-wise multiplication. Further, the enhanced attention matrix is ​​row-wise L1 normalized to obtain the final attention output:

[0054]

[0055] in, This involves L1 normalization. Finally, the attention output and input are added together via a residual connection, and then processed through layer normalization and a feedforward neural network to obtain the output of the real part structure enhancement module. :

[0056]

[0057] in, It is layer normalization. It is a feedforward network. It is a projection matrix. It is an activation function. It is a bias term that changes linearly.

[0058] Furthermore, the specific steps of step S5 are as follows:

[0059] First, for the imaginary part sequence Flattening the extended and time dimensions and mapping them to the latent space using a learnable linear projection, we obtain:

[0060]

[0061] Subsequently, the query matrix, key matrix, and value matrix are obtained through the projection matrix:

[0062]

[0063] in, Both are projection matrices. Standard attention similarity is calculated and power-law nonlinear enhancement is applied:

[0064]

[0065] in, Represents a symbolic function. For learnable parameters, , To scale the factor. Query matrix. Bond matrix Mapping each feature to one of the two sub-controls creates two parallel relationship paths. , Based on this, two types of structural relation matrices are constructed respectively:

[0066]

[0067] To achieve adaptive modeling of the two types of relationships, a learnable gating matrix is ​​introduced. Perform a weighted fusion of the two branches:

[0068]

[0069] in, It is an activation function. This indicates element-wise multiplication. The attention output is then further calculated.

[0070]

[0071] in, This involves L1 normalization. Finally, the output of the imaginary part structure enhancement module is obtained through residual connections and a feedforward network. :

[0072]

[0073] in, It is layer normalization. It is a feedforward network. It is a projection matrix. It is an activation function. It is a bias term that changes linearly.

[0074] Furthermore, the specific steps of step S6 are as follows:

[0075] The bidirectional cross-attention module consists of two symmetrical substructures: a cross-attention module guided by the imaginary part and a cross-attention module guided by the real part. In the branch guided by the imaginary part, the imaginary part features are used as the query. Real features as keys Sum :

[0076]

[0077] in, Both are projection matrices. Calculate cross-attention.

[0078]

[0079] The interaction result is consistent with the original representation. Residual fusion and layer normalization are performed to obtain the imaginary part representation after cross-enhancement. :

[0080]

[0081] In the branch of imaginary part guided by the real part, the real part feature is used as the query. The imaginary part features serve as keys Sum :

[0082]

[0083] in, Both are projection matrices. Calculate cross-attention.

[0084]

[0085] The interaction result is consistent with the original representation. After addition and layer normalization, the real part table after cross-enhancement is obtained.

[0086] Show :

[0087]

[0088] Output Reshape into d dimensions The subarray blocks form the intermediate tensor. Then through a linear projection layer Each sub-matrix block undergoes a dimensionality transformation, mapping it to the initial dimensional feature space. The final output of the bidirectional cross-attention module is as follows:

[0089]

[0090] Furthermore, the specific steps of step S7 are as follows:

[0091] Step S7-1: First, output the real part. With imaginary part output Reconstructed as a complex spectrum:

[0092]

[0093] Then, the complex spectrum is restored to the time domain using the inverse discrete Fourier transform, resulting in the reconstructed time-domain feature representation:

[0094]

[0095] in, This represents the inverse discrete Fourier transform.

[0096] Step S7-2: In order to preserve the basic semantic information of the input stage, the reconstructed temporal features are combined with the front-end dimensional extended features. Residual fusion is performed to obtain the final temporal fusion representation:

[0097]

[0098] Furthermore, the specific steps of step S8 are as follows:

[0099] Temporal fusion representation Perform a flattening operation to convert the high-dimensional tensor into a two-dimensional prediction representation:

[0100]

[0101] in, Indicates the flattening operation. Subsequently, the prediction is projected onto the target prediction length H through a linear layer to obtain the prediction result in the normalized space:

[0102]

[0103] Since the original sequence is standardized using reversible instance normalization during the model input stage, the saved mean needs to be utilized during the output stage. and standard deviation The prediction results are inversely normalized to restore the original numerical space. The specific form is:

[0104]

[0105] in, This is the final predicted output.

[0106] Furthermore, the total loss function of the long-term series prediction model based on frequency domain cross-enhancement is calculated, and the network model training is completed. The mean squared error loss function (MSE) is preferably used as the optimization objective, and its expression is:

[0107]

[0108] in, Represents the true value. This represents the predicted value. The entire time series prediction network model is trained through gradient backpropagation and parameter updates.

[0109] The present invention has the following technical effects:

[0110] (1) Time series in the time domain are usually a superposition of multiple dynamic patterns. Their long-term trends, periodic structures, and short-term fluctuations are often intertwined, making it difficult for models to simultaneously characterize global dependencies and local changes within a unified framework. This is especially true in long-term series prediction tasks, where information decay and error accumulation are prone to occur. In the frequency domain, however, time series can be decomposed into several components with clear frequency meanings through Fourier transform, transforming the originally complex coupled time dependencies into structural expressions on different frequency components, thereby significantly reducing the modeling difficulty. Compared to time domain modeling, frequency domain modeling has a natural globality, as each frequency component is determined by the entire time series, thus enabling more effective capture of long-distance dependencies. This invention maps time series to the frequency domain and models the series from a global frequency perspective, effectively enhancing the model's ability to express long-term dependencies and complex periodic patterns, thereby improving the accuracy and stability of long-term series prediction.

[0111] (2) The real and imaginary parts of a complex spectrum carry different structural information, which together determine the amplitude and phase characteristics of the signal. However, existing methods usually model the real and imaginary parts independently or simply splice them together, lacking an effective interaction mechanism and making it difficult to fully explore the coupling relationship within the spectrum. This invention constructs a real part structure enhancement module and an imaginary part structure enhancement module, respectively, and introduces a structure enhancement attention mechanism in each branch to improve the feature representation capability from two aspects: nonlinear enhancement of structural relationship modeling. Furthermore, an information interaction channel is established between the real and imaginary parts through a bidirectional cross-attention mechanism, enabling the two components to perceive each other's feature information, thereby realizing explicit coupling modeling between amplitude and phase information. This mechanism can effectively enhance the model's ability to characterize frequency rotation, phase drift, and complex dynamic modes.

[0112] (3) In the frequency domain modeling process, since the time series spectrum is usually sparse, traditional self-attention mechanisms are prone to the problem of excessive concentration of attention weights on a few high-energy frequency points, resulting in a low-rank attention matrix and limiting the model's ability to express multi-frequency band dependencies. This invention improves the attention calculation process by introducing a structure-enhanced attention mechanism: on the one hand, it uses a learnable power function to perform a nonlinear transformation on the similarity matrix to adjust the attention distribution and reduce attention entropy; on the other hand, it uses a bi-branch structure modeling mechanism to characterize different types of feature relationships and combines a gating mechanism to achieve adaptive fusion, thereby enriching the attention expression at the structural level. The above improvements can effectively alleviate the problem of excessive attention focus and improve the diversity of attention matrix expression. Attached Figure Description

[0113] Figure 1 This is a flowchart illustrating the overall implementation of a long-time series prediction method based on frequency domain cross-enhancement according to the present invention.

[0114] Figure 2 This is a visual comparison of the prediction results of the ETTh1 dataset selected in this invention when the prediction length is 96.

[0115] Figure 3 This is a visual comparison of the prediction results of the ECL dataset selected for this invention when the prediction length is 720. Detailed Implementation

[0116] To better illustrate the objectives, technical solutions, and advantages of the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention.

[0117] like Figure 1As shown, this invention provides a long-time series prediction method based on frequency domain cross-enhancement, comprising a frequency domain preprocessing module, a real part structure enhancement module, an imaginary part structure enhancement module, a bidirectional interactive attention module, a temporal reconstruction fusion module, and a prediction output module. The frequency domain preprocessing module performs normalization, dimensionality expansion, and frequency domain transformation of the input sequence; the real part structure enhancement module and the imaginary part structure enhancement module respectively perform structural enhancement modeling on the two components of the complex spectrum; the bidirectional interactive attention module establishes a bidirectional information interaction relationship between the two components; the temporal reconstruction fusion module restores the frequency domain representation to the temporal domain and fuses it with the front-end features; and the prediction output module generates the final output result at the target prediction length.

[0118] The long-time series prediction method based on frequency domain cross-enhancement described in this invention includes the following:

[0119] Step S1: Obtain the real-world dataset for time series forecasting. The dataset includes ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and Traffic datasets, and divide them into training, validation, and test sets according to a preset ratio. In this embodiment, a 7:2:1 ratio is preferred. Further, based on the requirements of the long-term series forecasting task, construct historical observation windows and future forecast windows for the original time series.

[0120] Step S2: Construct a long-term sequence prediction network model based on frequency domain cross-enhancement; the network model includes: a frequency domain preprocessing module; a real part structure enhancement module; an imaginary part structure enhancement module; a bidirectional interactive attention module; a temporal reconstruction and fusion module; and a prediction output module. The frequency domain preprocessing module is used to normalize, expand the dimension, and transform the frequency domain of the input sequence; the real part structure enhancement module and the imaginary part structure enhancement module are used to perform structural enhancement modeling on the two components of the complex spectrum, respectively; the bidirectional interactive attention module is used to establish a bidirectional information interaction relationship between the two components; the temporal reconstruction and fusion module is used to restore the frequency domain representation to the temporal domain and fuse it with the front-end features; and the prediction output module is used to generate the final output result at the target prediction length.

[0121] Step S3: Input time series The data is fed into the frequency domain preprocessing module. This module comprises a reversible instance normalization unit, a dimension expansion unit, and a discrete Fourier transform unit. Its function is to reduce the influence of distribution drift between input samples, enhance the semantic expressiveness of input features, and convert time-domain information to the frequency domain, providing a foundation for subsequent frequency domain modeling. First, reversible instance normalization is performed on the time series, calculating the mean and standard deviation for each variable along the time dimension to normalize the input sequence. Second, dimension expansion is used to map the normalized two-dimensional time series tensor into a three-dimensional high-dimensional representation. Finally, a discrete Fourier transform is performed on the dimension-expanded time series along the time dimension to obtain a frequency domain complex spectrum representation, which is then decomposed into a sequence of real parts. and imaginary part sequence ;

[0122] Step S4: Convert the real part sequence obtained in step S3 into a sequence of real parts. The input real part structure enhancement module is used to perform structure enhancement modeling on the real part of the spectrum, thereby extracting the global frequency structure information contained in the real part and improving the model's ability to express spectral intensity distribution and dominant frequency modes. Specifically, for real part sequences... The extended and temporal dimensions are flattened, and then mapped to a unified latent space dimension through a learnable linear projection. Thus, input of attention is obtained. Secondly, the computational structure enhances attention, and the intermediate output is obtained by combining residual connections and layer normalization. Finally, the output of the real part structure enhancement module is obtained through the feedforward neural network, which is the real part spectrum representation.

[0123] Step S5: The imaginary part sequence obtained in step S3... The input imaginary part structure enhancement module flattens the extended and time dimensions and maps them to a unified hidden space dimension through a learnable linear projection. Thus, input of attention is obtained. Secondly, the computational structure enhances attention, and the intermediate output is obtained by combining residual connections and layer normalization. Finally, the output of the imaginary part structure enhancement module is obtained through a feedforward neural network, which is the imaginary part spectrum representation.

[0124] Step S6: Input the real part spectral representation and the imaginary part spectral representation into the bidirectional interactive attention module to establish the interaction relationship between the real part and the imaginary part, and obtain the real part representation and the imaginary part representation after cross-enhancement; restore the real part representation and the imaginary part representation to the tensor structure corresponding to the frequency domain input through the projection matrix, and reconstruct the complex spectrum;

[0125] Step S7: The frequency domain complex spectrum representation obtained in step S6 is fed into the time domain reconstruction and fusion module. This module mainly includes two processes: inverse discrete Fourier transform and time domain residual fusion. Specifically, the complex spectrum representation is remapped back to the time domain through inverse discrete Fourier transform, and the basic semantic information of the input stage is preserved through residual connection, thereby obtaining a fused representation that combines the global structure of the frequency domain and the original time domain features.

[0126] Step S8: The prediction output module maps the high-dimensional time domain representation to the output result on the target prediction length, and restores the original numerical scale through inverse normalization, finally generating the prediction sequence.

[0127] Step S9: Calculate the total loss function of the long-term series prediction model based on frequency domain crosstalk enhancement. And complete the training of the time series prediction network model;

[0128] Step S10: Input the test set time series data into the trained time series prediction network model to obtain the final time series prediction results. Figure 2 This is a visual comparison chart of the prediction results of our method with eight other methods—MultiPatchFormer, TimeXer, TimeMixer, PatchTST, iTransformer, DLinear, TimesNet, and Autoformer—on the ETTh1 dataset with an input length of 96 and a prediction length of 96. Figure 3 This is a visualization comparing the prediction results of our method with eight other methods—MultiPatchFormer, TimeXer, TimeMixer, PatchTST, iTransformer, DLinear, TimesNet, and Autoformer—on the ECL dataset with an input length of 96 and a prediction length of 720. The graph shows that, for both short-term and long-term predictions, our method's blue predicted values ​​closely match the orange original values, effectively fitting the changing trends of the real time series.

[0129] Table 1 shows the evaluation metrics of all comparison methods under MAE (mean absolute error) and MSE (mean squared error) in seven real-world datasets.

[0130] Table 1: Prediction results of various methods with prediction lengths of 96, 192, 336, and 720 when the input length is uniformly 96.

[0131]

[0132] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.

Claims

1. A long-term series prediction method based on frequency domain cross-enhancement, characterized in that, Includes the following steps: Step S1: Obtain seven real-world datasets for time series prediction, including ETTh1 and ETTh2, ETTm1 and ETTm2, Weather, ECL, and Traffic, and divide them into training set, validation set, and test set in a ratio of 7:2:

1. Construct historical observation window and future prediction window according to the prediction task. Step S2: Construct a long-term series prediction network model based on frequency domain cross-enhancement; Step S3: Input time series The data is sent to the frequency domain preprocessing module, whereby... This represents the observation at time T, where N is the number of channels and T is the sequence length. For the real number domain, the frequency domain preprocessing module sequentially includes an invertible instance normalization unit, a dimension expansion unit, and a discrete Fourier transform unit. First, invertible instance normalization is performed on the time series, calculating the mean and standard deviation for each variable along the time dimension to normalize the input sequence. Second, dimension expansion is used to map the normalized two-dimensional time series tensor into a three-dimensional high-dimensional representation. Finally, a discrete Fourier transform is performed on the dimension-expanded time series along the time dimension to obtain the frequency domain complex spectrum representation, which is then decomposed into a sequence of real parts. and imaginary part sequence Where d is the extended dimension; Step S4: Convert the real part sequence The input real part structure enhancement module flattens the extended and time dimensions and maps them to a unified hidden space dimension through a learnable linear projection. Thus, input of attention is obtained. Secondly, the computational structure enhances attention, and the intermediate output is obtained by combining residual connections and layer normalization. Finally, the output of the real part structure enhancement module is obtained through the feedforward neural network, which is the real part spectrum representation. Step S5: Convert the imaginary part sequence The input imaginary part structure enhancement module flattens the extended and time dimensions and maps them to a unified hidden space dimension through a learnable linear projection. Thus, input of attention is obtained. Secondly, the computational structure enhances attention, and the intermediate output is obtained by combining residual connections and layer normalization. Finally, the output of the imaginary part structure enhancement module is obtained through a feedforward neural network, which is the imaginary part spectrum representation. Step S6: Input the real part spectral representation and the imaginary part spectral representation into the bidirectional interactive attention module to establish the interaction relationship between the real part and the imaginary part, and obtain the real part representation and the imaginary part representation after cross-enhancement; restore the real part representation and the imaginary part representation to the tensor structure corresponding to the frequency domain input through the projection matrix, and reconstruct the complex spectrum; Step S7: Input the complex spectrum after frequency domain modeling into the time domain reconstruction fusion module, remap the complex spectrum representation back to the time domain through inverse discrete Fourier transform, and retain the basic semantic information of the input stage through residual connection, thereby obtaining a fused representation that has both frequency domain global structure and original time domain features; Step S8: The prediction output module maps the high-dimensional time domain representation to the output result on the target prediction length, and restores the original numerical scale through inverse normalization, finally generating the prediction sequence; Step S9: Calculate the total loss function of the long-term series prediction model based on frequency domain crosstalk enhancement. And complete the training of the time series prediction network model; Step S10: Input the test set time series data into the trained time series prediction network model to obtain the final time series prediction results.

2. The long-time series prediction method based on frequency domain crosstalk enhancement according to claim 1, characterized in that, The specific steps of step S3 are as follows: Step S3-1: Process the input time series ,in This represents the observation at time T, where N is the number of channels and T is the sequence length. For the real number field; calculate the mean along the time dimension. and standard deviation The specific formula is as follows: in, This represents the mean of the j-th channel; in, This represents the standard deviation of the j-th channel. To prevent division by zero of extremely small constants, The input sequence is the observation value of channel j at time t; then the input sequence is normalized to obtain the normalized sequence. : Step S3-2: Expand the dimensionality through learnable projection vectors to improve the ability to represent time series features: in, It is a learnable projection vector, d is the extended dimension, and the output is... ; Step S3-3: Output Performing a discrete Fourier transform along the time dimension yields the complex spectrum in the frequency domain: in, Let e ​​represent the complex number representation of the k-th frequency component, where e represents the exponent and j is the imaginary unit; decompose the complex spectrum into a real part sequence R and an imaginary part sequence I: in, These represent the real part sequence and the imaginary part sequence, respectively.

3. The long-time series prediction method based on frequency domain crosstalk enhancement according to claim 1, characterized in that, The specific steps of step S4 are as follows: For the real part sequence Flatten and linearly map to unify into the hidden space: Obtain the query matrix through the projection matrix. Key matrix Sum matrix : in, Both are projection matrices; standard attention similarity is calculated and exponential activation is performed: in, Represents a symbolic function. For learnable parameters, ; To scale the factor, the query matrix will be... Bond matrix Mapping each feature to one of the two sub-controls creates two parallel relationship paths. , Based on this, two types of structural relation matrices are constructed respectively: in, Used to describe a consistent relationship in the same direction. Used to describe complementary interaction relationships; to achieve adaptive modeling of the two types of relationships, a learnable gating matrix is ​​introduced. Perform a weighted fusion of the two branches: in, It is an activation function. This represents element-wise multiplication; therefore, the attention output is: in, This involves L1 normalization; finally, the output of the real part structure enhancement module is obtained through residual connections and a feedforward network. : in, It is layer normalization. It is a feedforward network. It is a projection matrix. It is an activation function. It is a bias term that changes linearly.

4. The long-time series prediction method based on frequency domain crosstalk enhancement according to claim 1, characterized in that, The specific steps of step S5 are as follows: For the imaginary part sequence Flatten and linearly map to unify into the hidden space: Obtain the query matrix through the projection matrix. Key matrix Sum matrix : in, Both are projection matrices; standard attention similarity is calculated and exponential activation is performed: in, Represents a symbolic function. For learnable parameters, ; As a scaling factor, the query matrix Bond matrix Mapping each feature to one of the two sub-controls creates two parallel relationship paths. , Based on this, two types of structural relation matrices are constructed respectively: To achieve adaptive modeling of the two types of relationships, a learnable gating matrix is ​​introduced. Perform a weighted fusion of the two branches: in, It is an activation function. This represents element-wise multiplication; therefore, the attention output is: in, This involves L1 normalization; finally, the output of the imaginary part structure enhancement module is obtained through residual connections and a feedforward network. : in, It is layer normalization. It is a feedforward network. It is a projection matrix. It is an activation function. It is a bias term that changes linearly.

5. The long-time series prediction method based on frequency domain crosstalk enhancement according to claim 1, characterized in that, The specific steps of step S6 are as follows: The bidirectional cross-attention module consists of two symmetrical substructures, namely, the cross-attention of the imaginary part guiding the real part and the cross-attention of the real part guiding the imaginary part; in the branch of the real part guided by the imaginary part, the imaginary part features are used as queries, and the real part features are used as keys and values: in, Both are projection matrices; calculate cross-attention. The interaction result is consistent with the original representation. After addition and layer normalization, the imaginary part table after cross-enhancement is obtained. Show : In the branch of the imaginary part guided by the real part, the real part feature is used as the query, and the imaginary part feature is used as the key and value: in, Both are projection matrices; calculate cross-attention. The interaction result is consistent with the original representation. After addition and layer normalization, the real part table after cross-enhancement is obtained. Show : Output Reshape into d dimensions The subarray blocks form the intermediate tensor. Then through a linear projection layer Each sub-matrix block undergoes a dimensionality transformation, mapping it to the initial dimensional feature space. The final output of the bidirectional cross-attention module is as follows: 。 6. The long-time series prediction method based on frequency domain crosstalk enhancement according to claim 1, characterized in that, The specific steps of step S7 are as follows: Step S7-1: Reconstruct the complex spectrum to obtain: in, Output the real part. Output the imaginary part, where j is the imaginary unit; By using the inverse discrete Fourier transform to restore the complex spectrum to the time domain, we obtain the reconstructed time-domain feature representation: in, This represents the inverse discrete Fourier transform, where N is the number of channels, T is the sequence length, and d is the extended dimension. Step S7-2: Introduce residual connections to reconstruct the temporal features. Input features expanded with front-end dimensions The fusion process yields the final temporal fusion representation: 。 7. The long-time series prediction method based on frequency domain crosstalk enhancement according to claim 1, characterized in that, The specific steps of step S8 are as follows: For the temporal fusion representation... Perform a flattening operation to convert the high-dimensional tensor into a two-dimensional prediction representation: in, Indicates the flattening operation. N is the number of channels, T is the sequence length, and d is the expanded dimension; subsequently, it is projected onto the target prediction length H through a linear layer to obtain the prediction result in the normalized space: Since the original sequence is standardized using reversible instance normalization during the model input stage, the saved mean needs to be utilized during the output stage. and standard deviation The prediction results are inversely normalized to restore the original numerical space; specifically, the following form is used: in, This is the final predicted output.

Images