A long time series prediction method based on time domain decoupling enhancement

Abstract

The present invention belongs to the technical field of time series analysis, and discloses a long time series prediction method based on time domain decoupling enhancement. The method is implemented through a prediction model, and the model mainly includes a time network module, an augmented network module, a sampling module, and a fusion enhancement module. The sampling module converts a long sequence into a time subsequence to obtain more refined feature information. The time network module is mainly used to perform coarse prediction processing on the overall time series and provide feature information for the fusion enhancement module. The augmented network module performs fine processing on the time subsequence and transmits its own feature information to the fusion enhancement module. The fusion enhancement module receives the refined features from the subsequence and the feature information of the original sequence for fusion enhancement, and serves as the auxiliary decoding input of the time network and augmented network modules. Finally, the output of the time network and the augmented network are combined as the prediction output, thereby realizing the alignment of spatiotemporal information and achieving more efficient and accurate prediction capabilities.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series analysis, and in particular relates to a long time series prediction method based on time domain decoupling enhancement. Background Art

[0002] Long-term time series forecasting (LTSF) involves building models based on historical time series data to predict future values ​​or trends. It has important applications in finance, meteorology, stock markets, traffic flow, energy consumption, and other fields. Long-term time series forecasting is also a long-standing problem in time series analysis. Accurately predicting the future of time series may require understanding their true relationships. LTSF requires obtaining as much information as possible from historical series, which can provide valuable insights into the systems represented by the time series.

[0003] Currently, there are several main methods for time series forecasting: statistical methods, machine learning methods, neural networks, and deep learning methods. Traditional statistical methods, such as autoregressive and moving average models, autoregressive moving averages, and spectral analysis techniques, are used for time series forecasting. However, these traditional statistical methods require many prior assumptions for time series forecasting, such as stability, normal distribution, linear correlation, and independence. However, in many practical situations, time series data exhibit non-stationarity. These assumptions limit the effectiveness of these traditional methods in practical applications. Machine learning methods generally calculate data metrics, such as minimum, maximum, mean, and variance, within a sliding window as new features for prediction. These models address the problem of predicting multivariate heteroskedastic time series with nonlinear relationships to a certain extent. However, their generalization capabilities are limited, resulting in limited forecasting accuracy. Neural networks and their variants are widely used models for processing sequential data. These models use the gradient descent algorithm to sequentially process batches of data to optimize the model parameters at each location. However, due to the sequential processing of input data and backpropagation over time, they are subject to limitations. In particular, when processing datasets with long-term dependencies, the training process can suffer from vanishing and exploding gradients. Deep learning methods have greatly improved the nonlinear modeling capabilities of LTSF. These models, built using neural network architectures, possess powerful nonlinear modeling capabilities and can automatically learn complex patterns and feature representations in time series. They are widely used due to their high flexibility, ability to capture long-term dependencies, and ability to process large amounts of data. Among deep learning models, the Transformer has achieved tremendous success in various application areas, including natural language processing (NLP), computer vision (CV), speech, and, more recently, time series. Benefiting from its attention mechanism, it can automatically learn the connections between elements in a sequence, making it an ideal choice for sequential modeling tasks. The Informer, Autoformer, and FEDformer are the best variants of the Transformer model that have been successfully applied to time series data. However, despite their excellent design, these models only predict the entire time series and ignore the fact that the lookback window of historical time steps is crucial for generating accurate predictions. Furthermore, when predicting only a portion of a time series, the changing interdependencies between sequences are often overlooked. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a long time series prediction method based on time domain decoupling enhancement, which can effectively capture long-term dependencies, while performing feature extraction and enhancement on the original signal and the sampled signal, and utilizing the fusion of time and space information to achieve time and space information alignment, thereby achieving more efficient and accurate prediction capabilities.

[0005] To achieve the above-mentioned purpose, the present invention is implemented through a technical solution.

[0006] The present invention is a long time series prediction method based on time domain decoupling enhancement, which includes the following steps:

[0007] Step 1: Get a dataset containing time series and divide it into training and test sets;

[0008] Step 2: Divide the network into two parts: the original time network module and the augmented network module. Sample the original signal as the input of the augmented network module, and the original sequence as the input of the original time network module.

[0009] Step 3: Decompose the original signal and the time series signal obtained after sampling to obtain the seasonal characteristics and trend characteristics of the time network module and the augmented time network module respectively;

[0010] Step 4: Process the decomposed trend features using the thread layer, and obtain the predicted values ​​of the trend features of the time network module and the augmented time network module;

[0011] Step 5: The seasonal features of the decomposed time network module are first embedded by position and then encoded by the Transformer. The fused enhanced feature information is used as the Transformer auxiliary decoding input to obtain the required seasonal prediction component of the time network module, which is then added to the previous trend prediction component to obtain the prediction value of the time network module.

[0012] Step 6: Input the decomposed seasonal features of the augmented network module, the seasonal features and trend features of the time network module into the enhancement module. Use the seasonal information of the time network and augmented network as the input of this module to obtain the fused enhanced seasonal features, which are used as auxiliary decoding input of the augmented network module Transformer.

[0013] Step 7: The seasonal features of the augmented network module obtained by decomposition are first positionally embedded and then encoded by the Transformer. The fused enhanced feature information is used as the Transformer auxiliary decoding input to obtain the seasonal prediction component of the augmented network module. The seasonal component and trend component in the augmented network module prediction information are summarized to obtain the prediction value of the augmented network module.

[0014] Step 8: Integrate the prediction of the time network module obtained in step 5 with the prediction of the augmented network obtained in step 7 to obtain the final time series prediction.

[0015] The long time series prediction method of the present invention is implemented by a long time series prediction network model, which includes a time network module, an augmented network module, a sampling module, a signal decomposition module, a fusion enhancement module, a Transformer encoding and decoding block, and a linear module.

[0016] The time network module is used for overall prediction of time series;

[0017] The augmented network module is used to extract predictions for parts of the sequence;

[0018] The sampling module cuts the original sequence into partial subsequences;

[0019] The signal decomposition module decomposes the time series to obtain the required seasonal and trend characteristics;

[0020] The fusion enhancement module uses self-supervision to fuse and extract the seasonal features and trend features of the time network module and the augmented time network module, enhance the feature information, and use it as the input for seasonal feature decoding of the time network module and the augmented network module to enhance the output of the fusion feature information prediction;

[0021] The Transformer encoding and decoding block is used to extract and predict seasonal features in time series;

[0022] The linear module is used for trend prediction. Finally, the prediction information obtained by the time network module is combined and enhanced with the information obtained by the augmented network module to output the final prediction value.

[0023] A further improvement of the present invention is that: in the step 2, the network is divided into two parts: the original time network module and the augmented network module, the original signal is sampled as the input of the augmented network module, and the original sequence is used as the input of the original time network module. Specifically, the network is divided into two parts: the time network module and the augmented network module, and the multivariate time series is sampled. After downsampling, it is cut into multiple segments sequence, where represents a subsequence of a multivariate time series, N represents the count of the signal from the sensor, T represents the length of the sequence, T s It is expressed as the time series length after sampling, D represents the dimension of the signal input, and the subsequence after sampling is As the input of the augmented network module, the original sequence X is used as the input of the temporal network module.

[0024] A further improvement of the present invention is that in step 3, the original signal and the time series signal obtained after sampling are decomposed to obtain the seasonality and trend characteristics of the time network module and the augmented network module respectively, specifically:

[0025] For trend characteristics, average counts are used to smooth repeated fluctuations and reveal potential long-term trends, calculated by the following formula:

[0026] X t =AvgPool(Padding(X))

[0027] X s =XX t

[0028]

[0029]

[0030] where X t , X s Respectively represent the trend characteristics and seasonal characteristics of the original time series, They represent the trend characteristics and seasonal characteristics of the subsequence after sampling, respectively. AvgPool(·) is the moving average. At the same time, the missing values ​​are padded with zeros to ensure that the length of the original sequence remains unchanged.

[0031] A further improvement of the present invention is that: in step 4, the trend features obtained by decomposition are processed using a thread layer, and the predicted amount of the trend features of the time network module and the augmented network module is obtained, and the formula is as follows:

[0032]

[0033]

[0034] in represents the predicted quantity of the trend characteristics of the time network module, represents the predicted quantity of the trend characteristics of the augmented network module, X t Represents the trend characteristics of the original time series, Indicates the trend characteristics of the subsequence after sampling.

[0035] A further improvement of the present invention is that: in step 5, the seasonal features of the decomposed time network module are first subjected to position embedding, then subjected to Transformer encoding processing, and the fusion enhanced feature information is used as Transformer auxiliary decoding input to obtain the required seasonal prediction component of the time network module, which is then added to the previous trend prediction component to obtain the predicted amount of the time network module, specifically including the following steps:

[0036] Step 5.1: For the time network module, the decomposed seasonal features are preprocessed, the original time series is embedded into the feature map and the original time series is split into patches. The formula is as follows:

[0037] X emb =Conv(X s )

[0038] X pat =Patch(X emb ,X0,p)

[0039] The time series with seasonal features is input into c feature maps embedded by a convolutional layer with a kernel size of 3×1. The patching process divides the original time series into Non-overlapping subsequences of size p, where X0 is zero-filled when the input sequence length L is not divisible by p1, and the resulting sequence is Where D represents the dimension of the signal input, N represents the count of the signal from the sensor, and finally the feature map dimension and the block size dimension are combined to form the input of the Transformer Using the attention mechanism in Transformer, as shown below:

[0040]

[0041]

[0042] in is a subset of keys selected based on the characteristics of the dataset, Represents the weight coefficient to prevent the gradient from disappearing. Following the channel-independent design, the final attention mechanism is transformed into the following:

[0043] Q h ,K h ,V h =Linear(Q,K,V) h

[0044]

[0045]

[0046] Among them, O h The encoding input from the fusion enhancement module is derived, and the information of the augmented network module and the temporal network module is fused and enhanced as the decoded V value. Represents Transformer output, N O Indicates the number of patches for the decoder;

[0047] Step 5.2: Separate the feature dimension and the patch size dimension to obtain

[0048] Step 5.3. Reduce the number of feature maps from c to 1 using a convolutional layer with a kernel size of 1×1 as follows:

[0049]

[0050] in It is the prediction of seasonal sub-features. Finally, the seasonal and trend sub-features are summed according to the following formula to generate the prediction of the time network module:

[0051]

[0052] in, represents the prediction of seasonal characteristics of the time network module, Represents the prediction of trend characteristics of the temporal network module.

[0053] A further improvement of the present invention is that: the seasonality and trend characteristics X of the original time series obtained by decomposition in step 6 and the subsequence after sampling are s , X t , They are input into the fusion enhancement module respectively. The fusion enhancement module includes a fusion attention mechanism and a feedforward network, and specifically includes the following steps:

[0054] Step 6.1: The fusion enhancement module obtains the seasonal characteristics and trend characteristics obtained by signal decomposition of the original time series and the sampled series as input, and uses the attention mechanism to

[0055] Q,K,V=Linear(X s , X t )

[0056]

[0057] in, represents the query matrix of the sampled sequence, represents the key matrix of the sampled sequence, Represents the value matrix of the sampled sequence, X t , X s Respectively represent the trend characteristics and seasonal characteristics of the original time series, Respectively represent the trend characteristics and seasonal characteristics of the subsequence after sampling;

[0058] Step 6.2: First, perform pooling to obtain the required fused token, and use it as Q to spread the augmented network information to each query token, and use the softmax attention mechanism to obtain:

[0059]

[0060] Subsequently, this part of information is used as the value of V, and the previously fused token is used as the value of K. The two exchange information and fuse the information of time and augmented network.

[0061] Step 6.3: Use the exchanged fusion information to integrate and use it as the final softmax attention V value. The supervision equation of the entire fusion enhancement module is:

[0062] O P =Attn S (Q,P,Attn S (P,K,V))

[0063] Among them Attn S (·) represents softmax attention.

[0064] A further improvement of the present invention is that: in step 7, the seasonal features of the decomposed augmented network module are first subjected to position embedding, then subjected to Transformer encoding processing, and the fused enhanced feature information is used as Transformer auxiliary decoding input to obtain the required seasonal prediction component of the augmented network module, which is then added to the previous trend prediction component to obtain the predicted value of the augmented network module, specifically comprising the following steps:

[0065] Step 7.1: For the augmented network module, preprocess the decomposed seasonal features, embed the sampled original time subsequence into the feature map, and split the sampled original time subsequence into patches. The formula is as follows:

[0066]

[0067]

[0068] The time series with seasonal features is input into c feature maps embedded by a convolutional layer with a kernel size of 3×1. The patching process divides the sampled original time subsequence into Non-overlapping subsequences of size p2, where for the input subsequence length When it is not divisible by p2, is zero-filled, and the resulting sequence is Where D represents the dimension of the signal input, N represents the count of the signal from the sensor, and finally the feature map dimension and the block size dimension are combined to form the input of the Transformer Using the attention mechanism in Transformer, as shown below:

[0069]

[0070]

[0071] in is a subset of keys selected based on the characteristics of the dataset, Represents the weight coefficient to prevent the gradient from disappearing. Following the channel-independent design, the final attention mechanism is transformed into the following:

[0072] Q h ,K h ,V h =Linear(Q,K,V) h

[0073]

[0074]

[0075] Among them, Oh is derived from the encoding input of the fusion enhancement module, and the information of the augmented network module and the temporal network module is fused and enhanced as the decoded V value. Represents Transformer output, N O Indicates the number of patches for the decoder;

[0076] Step 7.2: Separate the feature dimension and the patch size dimension to obtain

[0077] Step 7.3. Reduce the number of feature maps from c to 1 using a convolutional layer with a kernel size of 1×1 as follows:

[0078]

[0079] in It is the prediction of seasonal features. Finally, the seasonal features and trend features are element-wise summed according to the following formula to generate the prediction of the augmented network module:

[0080]

[0081] in represents the prediction of seasonal characteristics of the augmented network module, Represents the prediction of trend features of the augmented network module.

[0082] A further improvement of the present invention is that the output of the time network module and the output of the augmented network module are integrated to obtain the final time series prediction value, which is as follows:

[0083]

[0084] The beneficial effects of the present invention are:

[0085] This paper studies multivariate time series data from the perspective of complementary enhancement by utilizing the complementary relationship between time and augmentation, and proposes a new enhancement module to learn the hidden spatial dependencies between variables.

[0086] This paper introduces a fusion-enhanced attention mechanism to establish a good inductive bias when learning potential spatiotemporal relationships;

[0087] This paper proposes a joint framework to model multivariate time series. This framework is more general than existing deep learning networks and can process multivariate time series with or without predefined structures. While ensuring that the network can be quickly trained and tested, it enhances temporal information and uses the enhanced information to improve the accuracy and stability of long time series predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 It is the overall framework diagram of the long time series prediction network model constructed by the present invention.

[0089] Figure 2 It is a structural diagram of the fusion enhancement module of the present invention.

[0090] Figure 3 It is a structural diagram of the time network module of the present invention.

[0091] Figure 4 It is a structural diagram of the augmented network module of the present invention. DETAILED DESCRIPTION

[0092] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them.

[0093] Figure 1This is an overall framework diagram for the implementation of the long time series prediction method based on time domain decoupling enhancement of the present invention. As shown in 1-4, the long time series prediction network model includes a time network module, an augmented network module, a sampling module, a signal decomposition module, a fusion enhancement module, a Transformer codec block and a linear module. Among them, the augmented network module is similar to the time network module, using a signal decomposition module, a Transformer codec module and a linear module. The time network module is used as the overall prediction of the time series. The augmented network module is used to extract the prediction of part of the sequence. The sampling module cuts the original sequence into partial subsequences. The signal decomposition module decomposes the time series to obtain the required seasonal features and trend features. The fusion enhancement module adopts a self-supervision function to fuse and extract the seasonal features and trend features of the time network module and the augmented time network module, enhance the feature information, and serve as the input of the seasonal feature decoding of the time network module and the augmented network module to enhance the output of the fusion feature information prediction. The Transformer codec block is used to extract and predict seasonal features in the time series. The linear module is used for trend prediction. Finally, the prediction information obtained by the time network module is combined and enhanced with the information obtained by the augmented network module to output the final prediction value.

[0094] A long time series prediction method based on time domain decoupling enhancement based on the long time series prediction model includes the following steps:

[0095] Step 1: Get a dataset containing time series and divide it into training and test sets;

[0096] Step 2: Divide the network into two parts: the original time network module and the augmented network module. Sample the original signal and use it as the input of the augmented network module, and the original sequence as the input of the original time network module. In step 2, divide the network into two parts: the original time network module and the augmented network module. Sample the original signal and use it as the input of the augmented network module, and the original sequence as the input of the original time network module. Specifically: divide the network into two parts: the time network module and the augmented network module, and use the multivariate time series After downsampling, it is cut into multiple segments sequence, where represents a subsequence of a multivariate time series, N represents the count of the signal from the sensor, T represents the length of the sequence, T s It is expressed as the time series length after sampling, D represents the dimension of the signal input, usually D = 1. As the input of the augmented network module, the original sequence X is used as the input of the temporal network module.

[0097] Step 3: Decompose the original signal and the time series signal obtained after sampling to obtain the seasonal characteristics and trend characteristics of the time network module and the augmented time network module respectively. Specifically:

[0098] For trend characteristics, average counts are used to smooth repeated fluctuations and reveal potential long-term trends, calculated by the following formula:

[0099] X t =AvgPool(Padding(X))

[0100] X s =XX t

[0101]

[0102]

[0103] where X t , X s Respectively represent the trend characteristics and seasonal characteristics of the original time series, They represent the trend characteristics and seasonal characteristics of the subsequence after sampling, respectively. AvgPool(·) is the moving average. At the same time, the missing values ​​are padded with zeros to ensure that the length of the original sequence remains unchanged.

[0104] Step 4: Process the decomposed trend features using the thread layer and obtain the predicted trend features of the time network module and the augmented time network module. The formula is as follows:

[0105]

[0106]

[0107] in represents the predicted quantity of the trend characteristics of the time network module, represents the predicted quantity of the trend characteristics of the augmented network module, X t Represents the trend characteristics of the original time series, Indicates the trend characteristics of the subsequence after sampling.

[0108] Step 5: The seasonal features of the decomposed time network module are first embedded in position and then encoded by the Transformer. The fused enhanced feature information is used as the Transformer auxiliary decoding input to obtain the required seasonal prediction component of the time network module. This is then added to the previous trend prediction component to obtain the prediction value of the time network module. The specific steps include the following:

[0109] Step 5.1: For the time network module, the decomposed seasonal features are preprocessed, the original time series is embedded into the feature map and the original time series is split into patches. The formula is as follows:

[0110] X emb =Conv(X s )

[0111] X pat =Patch(X emb ,X0,p)

[0112] The time series with seasonal features is input into c feature maps embedded by a convolutional layer with a kernel size of 3×1. The patching process divides the original time series into Non-overlapping subsequences of size p, where X0 is zero-filled when the input sequence length L is not divisible by p1, and the resulting sequence is Where D represents the dimension of the signal input, N represents the count of the signal from the sensor, and finally the feature map dimension and the block size dimension are combined to form the input of the Transformer Using the attention mechanism in Transformer, as shown below:

[0113]

[0114]

[0115] in is a subset of keys selected based on the characteristics of the dataset, Represents the weight coefficient to prevent the gradient from disappearing. Following the channel-independent design, the final attention mechanism is transformed into the following:

[0116] Q h ,K h ,V h =Linear(Q,K,V) h

[0117]

[0118]

[0119] Among them, multi-attention differs in terms of cross attention, Q in the temporal network is derived from the input of the decoder, O h The encoding input from the fusion enhancement module is derived, and the information of the augmented network module and the temporal network module is fused and enhanced as the decoded V value. Represents Transformer output, N O Indicates the number of patches for the decoder;

[0120] Step 5.2: Separate the feature dimension and the patch size dimension to obtain

[0121] Step 5.3. Reduce the number of feature maps from c to 1 using a convolutional layer with a kernel size of 1×1 as follows:

[0122]

[0123] in It is the prediction of seasonal sub-features. Finally, the seasonal and trend sub-features are summed according to the following formula to generate the prediction of the time network module:

[0124]

[0125] in, represents the prediction of seasonal characteristics of the time network module, Represents the prediction of trend characteristics of the temporal network module.

[0126] Step 6: The seasonality and trend characteristics X of the original time series obtained by decomposition and sampling of the subsequence are s , X t , are input into the fusion enhancement module respectively, such as Figure 2 As shown, the fusion enhancement module includes a fusion attention mechanism and a feedforward network, specifically including the following steps:

[0127] Step 6.1: The fusion enhancement module obtains the seasonal characteristics and trend characteristics obtained by signal decomposition of the original time series and the sampled series as input, and uses the attention mechanism to

[0128] Q,K,V=Linear(X s , X t )

[0129]

[0130] in, represents the query matrix of the sampled sequence, represents the key matrix of the sampled sequence, Represents the value matrix of the sampled sequence, X t , X s Respectively represent the trend characteristics and seasonal characteristics of the original time series, Respectively represent the trend characteristics and seasonal characteristics of the subsequence after sampling;

[0131] Step 6.2: First, perform pooling to obtain the required fused token, and use it as Q to spread the augmented network information to each query token, and use the softmax attention mechanism to obtain:

[0132]

[0133] Subsequently, this part of information is used as the value of V, and the previously fused token is used as the value of K. The two exchange information and fuse the information of time and augmented network.

[0134] Step 6.3: Use the exchanged fusion information to integrate and use it as the final softmax attention V value. The supervision equation of the entire fusion enhancement module is:

[0135] O P =Attn S (Q,P,Attn S (P,K,V))

[0136] Among them Attn S (·) represents softmax attention. More specifically, the present invention has made a generalized expansion in the attention triple (Q, K, V), using the mapping relationship of linear attention, adding an additional vector P, and redefining a new four-element attention mechanism (Q, P, K, V). In order to reduce the time complexity and complete the original information interaction, we first pool the original Q, obtain the vector P from the original sequence, and integrate it with the information in KV, and then pass the integrated information to Q. Overall, the fusion attention mechanism contains two softmax attentions, which can be approximately equivalent to the natural fusion of generalized linear attention and softmax attention. Specifically, we use the initial Q pooled as the vector P and the sampled sequence KV to perform attention calculations to obtain the fusion feature V p , then use P as the key in the attention calculation, V p As the value and query matrix Q, the temporal information of the fused features is diffused to each query token and the final output O is obtained. P In this way, through the form of fusion, the information exchange between each K and V is retained, and the integration of the time and augmentation network information is obtained. Finally, it is used as the auxiliary decoding input of the time and augmentation network for the prediction output of the time and augmentation network.

[0137] Step 7: The seasonal features of the augmented network module obtained by decomposition are first embedded in the position and then encoded by the Transformer. The fused enhanced feature information is used as the Transformer auxiliary decoding input to obtain the seasonal prediction component of the augmented network module. The seasonal component and trend component in the augmented network module prediction information are summarized to obtain the prediction value of the augmented network module. The specific steps include the following:

[0138] Step 7.1: For the augmented network module, preprocess the decomposed seasonal features, embed the sampled original time subsequence into the feature map, and split the sampled original time subsequence into patches. The formula is as follows:

[0139]

[0140]

[0141] The time series with seasonal features is input into c feature maps embedded by a convolutional layer with a kernel size of 3×1. The patching process divides the sampled original time subsequence into Non-overlapping subsequences of size p2, where for the input subsequence length When it is not divisible by p2, is zero-filled, and the resulting sequence is Where D represents the dimension of the signal input, N represents the count of the signal from the sensor, and finally the feature map dimension and the block size dimension are combined to form the input of the Transformer Using the attention mechanism in Transformer, as shown below:

[0142]

[0143]

[0144] in is a subset of keys selected based on the characteristics of the dataset, Represents the weight coefficient to prevent the gradient from disappearing. Following the channel-independent design, the final attention mechanism is transformed into the following:

[0145] Q h ,K h ,V h =Linear(Q,K,V) h

[0146]

[0147]

[0148] Among them, Oh is derived from the encoding input of the fusion enhancement module, and the information of the augmented network module and the temporal network module is fused and enhanced as the decoded V value. Represents Transformer output, N O Indicates the number of patches for the decoder;

[0149] Step 7.2: Separate the feature dimension and the patch size dimension to obtain

[0150] Step 7.3. Reduce the number of feature maps from c to 1 using a convolutional layer with a kernel size of 1×1 as follows:

[0151]

[0152] in It is the prediction of seasonal features. Finally, the seasonal features and trend features are element-wise summed according to the following formula to generate the prediction of the augmented network module:

[0153]

[0154] in represents the prediction of seasonal characteristics of the augmented network module, Represents the prediction of trend features of the augmented network module.

[0155] Step 8: Integrate the prediction of the time network module obtained in step 5 with the prediction of the augmented network obtained in step 7 to obtain the final time series prediction. The formula is as follows:

[0156]

[0157] Finally, the test set multivariate time series As input to the trained long-term time series prediction network model, the time series prediction results are obtained. The effectiveness of the final method is evaluated by measuring its accuracy using the mean square error (MSE) and average error (MAE) indicators. The formulas for the mean square error and average error are as follows:

[0158]

[0159]

[0160] In order to verify the effectiveness of the present invention, the current mainstream time series prediction models Triformer, Crossformer, Scaleformer, PatchTST, FiLM, and TSMixer are tested on the time series data sets ETTh1, ETTh2, ETTm1, ETTm2, Electricity, Traffic, Weather, Exchange, and ILI, and compared with the time series prediction results of the present invention. The comparison results are shown in Table 1 below.

[0161] Table 1

[0162]

[0163] The experimental simulation content of the long time series prediction method based on deep learning of the present invention is as follows:

[0164] The experimental platform uses a 64-bit Ubuntu system with a system version of 20.04.4, a GPU model of GeForce RTX3080Ti, Python 3.7.6 as the programming language, and PyCharm as the software development platform. The model is trained using the ADAM optimizer, starting from 10 -4 We started with a learning rate of 100 and halved it every two epochs. We used early stopping during training; if there was no improvement after 10 epochs, the process was stopped. The batch size for training was 8. All tests were performed using the PyTorch framework.

[0165] The above description is merely a preferred embodiment of the present invention and does not limit the present invention in any way. Any person skilled in the art who, without departing from the scope of the present invention, makes any equivalent substitution, modification, or other variation to the technical solution and technical contents disclosed in the present invention shall be deemed to be within the scope of the present invention and still fall within the scope of protection of the present invention.

Claims

1. A long time series prediction method based on time domain decoupling enhancement, characterized by: The long time series prediction method comprises the following steps: Step 1: Get a dataset containing time series and divide it into training and test sets; Step 2: Divide the network into two parts: the original time network module and the augmented network module. Sample the original signal as the input of the augmented network module, and the original sequence as the input of the original time network module. Step 3: Decompose the original signal and the time series signal obtained after sampling to obtain the seasonal characteristics and trend characteristics of the time network module and the augmented time network module respectively; Step 4: Process the decomposed trend features using the thread layer, and obtain the predicted values ​​of the trend features of the time network module and the augmented time network module; Step 5: The seasonal features of the decomposed time network module are first embedded by position and then encoded by the Transformer. The fused enhanced feature information is used as the Transformer auxiliary decoding input to obtain the required seasonal prediction component of the time network module, which is then added to the previous trend prediction component to obtain the prediction value of the time network module. Step 6: Input the decomposed seasonal features of the augmented network module, the seasonal features and trend features of the time network module into the enhancement module. Use the seasonal information of the time network and augmented network as the input of this module to obtain the fused enhanced seasonal features, which are used as auxiliary decoding input of the augmented network module Transformer. Step 7: The seasonal features of the augmented network module obtained by decomposition are first positionally embedded and then encoded by the Transformer. The fused enhanced feature information is used as the Transformer auxiliary decoding input to obtain the seasonal prediction component of the augmented network module. The seasonal component and trend component in the augmented network module prediction information are summarized to obtain the prediction value of the augmented network module. Step 8: Integrate the prediction of the time network module obtained in step 5 with the prediction of the augmented network obtained in step 7 to obtain the final time series prediction; wherein, The step 5 specifically includes the following steps: Step 5.1: For the time network module, the decomposed seasonal features are preprocessed, the original time series is embedded into the feature map and the original time series is split into patches. The formula is as follows: ， ， Among them, the time series with seasonal features is input into c feature maps embedded by the convolution layer with a kernel size of 3×1. The patching process divides the original time series into Size Non-overlapping subsequences of , where for the input sequence length Cannot be When divisible, is zero-filled, and the resulting sequence is ,in represents the dimension of the signal input, Represents the count of the signal from the sensor, and finally combines the feature map dimension and the block size dimension to form the input of the Transformer , using the attention mechanism in Transformer, as shown below: ， ， in is a subset of keys selected based on the characteristics of the dataset, Represents the weight coefficient to prevent the gradient from disappearing. Following the channel-independent design, the final attention mechanism is transformed into the following: ， ， ， in, The encoding input from the fusion enhancement module is derived, and the information of the augmented network module and the temporal network module is fused and enhanced as the decoded V value. Represents the Transformer output, Indicates the number of patches for the decoder; Step 5.2: Separate the feature dimension and the patch size dimension to obtain ; Step 5.

3. Reduce the number of feature maps from c to 1 using a convolutional layer with a kernel size of 1×1 as follows: ， in It is the prediction of seasonal sub-features. Finally, the seasonal and trend sub-features are summed according to the following formula to generate the prediction of the time network module: ， in, represents the prediction of seasonal characteristics of the time network module, Represents the prediction of trend characteristics of time network modules; The step 7 specifically includes the following steps: Step 7.1: For the augmented network module, preprocess the decomposed seasonal features, embed the sampled original time subsequence into the feature map, and split the sampled original time subsequence into patches. The formula is as follows: ， ， The time series with seasonal features is input into c feature maps embedded by a convolutional layer with a kernel size of 3×1. The patching process divides the sampled original time subsequence into Size Non-overlapping subsequences of , where for the input subsequence length Cannot be When divisible, is zero-filled, and the resulting sequence is ,in represents the dimension of the signal input, Represents the count of the signal from the sensor, and finally combines the feature map dimension and the block size dimension to form the input of the Transformer , using the attention mechanism in Transformer, as shown below: ， ， in is a subset of keys selected based on the characteristics of the dataset, Represents the weight coefficient to prevent the gradient from disappearing. Following the channel-independent design, the final attention mechanism is transformed into the following: ， ， ， in, The encoding input from the fusion enhancement module is derived, and the information of the augmented network module and the temporal network module is fused and enhanced as the decoded V value. Represents the Transformer output, Indicates the number of patches for the decoder; Step 7.2: Separate the feature dimension and the patch size dimension to obtain ; Step 7.

3. Reduce the number of feature maps from c to 1 using a convolutional layer with a kernel size of 1×1 as follows: ， in It is the prediction of seasonal features. Finally, the seasonal features and trend features are element-wise summed according to the following formula to generate the prediction of the augmented network module: ， in represents the prediction of seasonal characteristics of the augmented network module, Represents the prediction of trend features of the augmented network module.

2. The long time series prediction method based on time domain decoupling enhancement according to claim 1, characterized in that: The long time series prediction method is implemented by a long time series prediction network model, which includes a time network module, an augmented network module, a sampling module, a signal decomposition module, a fusion enhancement module, a Transformer encoding and decoding block, and a linear module. The time network module is used for overall prediction of time series; The augmented network module is used to extract predictions for parts of the sequence; The sampling module cuts the original sequence into partial subsequences; The signal decomposition module decomposes the time series to obtain the required seasonal and trend characteristics; The fusion enhancement module uses self-supervision to fuse and extract the seasonal features and trend features of the time network module and the augmented time network module, enhance the feature information, and use it as the input for seasonal feature decoding of the time network module and the augmented network module to enhance the output of the fusion feature information prediction; The Transformer encoding and decoding block is used to extract and predict seasonal features in time series; The linear module is used for trend prediction. Finally, the prediction information obtained by the time network module is combined and enhanced with the information obtained by the augmented network module to output the final prediction value.

3. The long time series prediction method based on time domain decoupling enhancement according to claim 1, characterized in that: In step 2, the network is divided into two parts: the original time network module and the augmented network module. The original signal is sampled as the input of the augmented network module, and the original sequence is used as the input of the original time network module. Specifically, the network is divided into two parts: the time network module and the augmented network module, and the multivariate time series is sampled. After downsampling, it is cut into multiple segments sequence, where represents a subsequence of a multivariate time series, represents the count of the signal from the sensor, Indicates the length of the sequence, It is expressed as the time series length after sampling, Represents the dimension of the signal input, and the sampled subsequence As the input of the augmented network module, the original sequence As input to the temporal network module.

4. The long time series prediction method based on time domain decoupling enhancement according to claim 1 is characterized in that: In step 3, the original signal and the time series signal obtained after sampling are decomposed to obtain the seasonality and trend characteristics of the time network module and the augmented network module respectively, specifically: For trend characteristics, average counts are used to smooth repeated fluctuations and reveal potential long-term trends, calculated by the following formula: ， ， ， ， in , Respectively represent the trend characteristics and seasonal characteristics of the original time series, , Respectively represent the trend characteristics and seasonal characteristics of the subsequence after sampling, It is a moving average, and the missing values ​​are padded with zeros to ensure that the length of the original sequence remains unchanged.

5. The long time series prediction method based on time domain decoupling enhancement according to claim 1, characterized in that: In step 4, the decomposed trend features are processed using a thread layer to obtain the predicted amount of trend features of the time network module and the augmented network module. The formula is as follows: ， ， in represents the predicted quantity of the trend characteristics of the time network module, represents the predicted quantity of the trend characteristics of the augmented network module, Represents the trend characteristics of the original time series, Indicates the trend characteristics of the subsequence after sampling.

6. The long time series prediction method based on time domain decoupling enhancement according to claim 1, characterized in that: Step 6 decomposes the original time series obtained, and the seasonality and trend characteristics of the subsequence after sampling , , , They are input into the fusion enhancement module respectively. The fusion enhancement module includes a fusion attention mechanism and a feedforward network, and specifically includes the following steps: Step 6.1: The fusion enhancement module obtains the seasonal characteristics and trend characteristics obtained by signal decomposition of the original time series and the sampled series as input, and uses the attention mechanism to , ， in, represents the query matrix of the sampled sequence, represents the key matrix of the sampled sequence, Represents the value matrix of the sampled sequence, , Respectively represent the trend characteristics and seasonal characteristics of the original time series, , Respectively represent the trend characteristics and seasonal characteristics of the subsequence after sampling; Step 6.2: First, perform pooling to obtain the required fused token, and use it as Q to spread the augmented network information to each query token, and use the softmax attention mechanism to obtain: ， Subsequently, this part of information is used as the value of V, and the previously fused token is used as the value of K. The two exchange information and fuse the information of time and augmented network. Step 6.3: Use the exchanged fusion information to integrate and use it as the final softmax attention V value. The supervision equation of the entire fusion enhancement module is: ， in stands for softmax attention.

7. The long time series prediction method based on time domain decoupling enhancement according to claim 1, characterized in that: The output of the time network module is integrated with the output of the augmented network module to obtain the final time series prediction value. The formula is as follows: 。

Images