Time series data prediction method based on multilayer noise stripping
Through multi-layer noise stripping technology, including RevIN reversible normalization, moving average decomposition, wavelet decomposition and threshold quantization, the prediction performance problem of the time series data prediction model under noise interference is solved, and higher prediction accuracy is achieved.
Patent Information
- Application Number
- CN202411609858.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-02-18
AI Technical Summary
Existing time series data prediction models do not work well when dealing with noise interference, resulting in a degradation in prediction performance.
Using a multi-layer noise peeling method, the noise components in the time series data are gradually extracted through steps such as RevIN reversible normalization, moving average decomposition, wavelet decomposition and threshold quantization to achieve accurate prediction.
Through multi-layer noise stripping technology, noise interference is significantly reduced, and the prediction accuracy of time series data is improved, especially in univariate and multivariate prediction tasks.
Smart Images
Figure CN119474752A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of time series data prediction, and in particular to a time series data prediction method based on multi-layer noise stripping. Background Art
[0002] In recent years, long-term multivariate time series data forecasting has attracted widespread attention due to its outstanding performance in various fields such as finance, transportation, energy, and weather forecasting. In order to capture the internal features of time series data, a large number of forecasting models based on RNN, GNN, and Transformers have designed complex neural network architectures. However, due to their high computational overhead, such models face severe challenges in terms of computational volume and efficiency when applied to large-scale real-world datasets. Studies have shown that a set of simple single-layer linear models perform better than complex neural network models. In addition, existing models mostly focus on capturing the temporal and dimensional correlation of time series data, while ignoring the impact of distribution offset and noise in time series data on forecasting performance.
[0003] The existing technical solutions for time series data prediction can be mainly divided into methods such as Transformer-based models, MLP-based time series models, and statistical time series models.
[0004] (1) The Transformer-based model is a revolutionary neural network architecture that is mainly used to process sequence-to-sequence tasks such as machine translation and language modeling. It was proposed by researchers at Google in 2017 and has attracted widespread attention and application in the field of NLP. Unlike traditional recurrent neural networks (RNNs) and convolutional neural networks (CNNs), the Transformer model is completely based on the self-attention mechanism, which enables it to process all positions of the input sequence in parallel regardless of the length of the sequence. This feature gives the Transformer a great advantage in processing long sequences and can better capture long-distance dependencies. However, their large number of parameters and complex network structure will lead to high computational costs.
[0005] (2) MLP-based time series models usually refer to models that use MLP to model and predict time series data. MLP is a basic feedforward neural network structure consisting of multiple fully connected layers, with nonlinear activation functions between each layer. In time series data, MLP can be used to solve regression or classification problems, where the input is a time series sequence and the output is a prediction of future values or a classification of the sequence. However, due to the limitations of the MLP structure itself, it may perform poorly when dealing with long-term dependencies; in time series data, the value at a certain time point is usually affected by multiple previous time points, and MLP may not be able to fully capture this long-term dependency.
[0006] (3) Statistical time series model is a method of modeling and predicting time series data using statistical methods. These models are usually based on analyzing the statistical characteristics of the data and using these characteristics to make predictions. However, when dealing with complex and nonlinear time series data, these statistical prediction methods are not very effective.
[0007] Therefore, the linear model can extract the seasonal and trend characteristics within the sequence, but it ignores the fact that seasonality and trend will also be affected by random fluctuations or interference (noise) in the time series data. Noise will make the data more complex and difficult to predict, thus affecting their prediction performance.
[0008] Therefore, it is necessary to provide a time series data prediction method based on multi-layer noise stripping to solve the above technical problems. Summary of the invention
[0009] The purpose of the present invention is to provide a time series data prediction method based on multi-layer noise stripping to solve the problem that the time series data prediction model is interfered by noise. The key to the model of the present invention is to extract multiple noises inside the time series data, and to achieve accurate prediction of the time series data by accurately stripping the noise components in the time series data.
[0010] The time series data prediction method based on multi-layer noise stripping provided by the present invention comprises the following steps: S1, using RevIN to reversibly normalize the original time series data;
[0011] S2. Decompose trend and seasonal terms from time series data using moving average method;
[0012] S3, using wavelet decomposition to perform multi-layer decomposition on the seasonal term and trend term separated in the previous step;
[0013] S4, performing threshold quantization on each layer of data decomposed in the previous step to realize noise removal;
[0014] S5. Linearly map and denormalize the seasonal terms and trend terms of the time series data processed by wavelet threshold.
[0015] Preferably, the step S1 is reversible normalization, that is, using a learnable affine parameter vector to normalize the input instance according to a formula to reduce non-stationary information and improve prediction performance.
[0016] Preferably, the seasonal trend item decomposition in step S2 is used to split the time series data into three parts: trend, seasonality and residual, so as to make it easier to understand the structure and characteristics of the data, thereby performing more accurate prediction and analysis.
[0017] Preferably, step S2 specifically includes: using an average pooling operation to obtain the long-term stable trend within the sequence, and performing data padding at both ends of the sequence to ensure that the decomposed sequence is equal to the length of the original sequence, and finally using the original sequence to remove the trend item sequence to obtain the seasonal item sequence.
[0018] Preferably, step S3 specifically comprises: performing wavelet decomposition on the seasonal terms and trend terms separated in step S2 using Symlet basis functions, obtaining approximate coefficients (low-frequency coefficients) and detail coefficients (high-frequency coefficients) of the time series data through a formula, and being able to perform multiple decompositions for time series data of different lengths.
[0019] Preferably, in step S4, a soft threshold quantization method is used for threshold quantization of each layer of data. For the detail coefficients after soft threshold quantization, the approximate coefficients obtained by decomposition of the last layer need to be added to the tail to form a reconstructed sequence; and if the total number of input data is an odd number, the last element is discarded after reconstruction to maintain the original input length.
[0020] Preferably, the step S5 specifically includes: performing linear mapping operations on the two parts of the seasonal item and the trend item after wavelet threshold processing respectively to retain the original characteristics of the data, and then summing up the results of each part to obtain the final prediction result.
[0021] Compared with the related art, the time series data prediction method based on multi-layer noise stripping provided by the present invention has the following beneficial effects:
[0022] The key to the model of the present invention is to extract multiple noises from the time series data. Previous studies have shown that using moving average to decouple the seasonal term and trend term of the sequence can reduce noise interference and improve the accuracy of prediction. However, the reduction of noise interference and the improvement of prediction accuracy are far from enough. The present invention continues to perform wavelet filtering on the seasonal term and trend term to further remove the noise. Through double denoising, the model of the present invention achieves the best performance in single variable and multivariate prediction tasks.
[0023] Furthermore, the present invention combines reversible normalization and linear mapping to eliminate the distribution shift problem in the sequence and extract internal features of the sequence, so the model of the present invention is very effective in long-term sequence prediction for four real-world data sets. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 An overall flow chart of a time series data prediction method based on multi-layer noise stripping provided by the present invention;
[0025] Figure 2 This is a flow chart of the wavelet decomposition in the present invention. DETAILED DESCRIPTION
[0026] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.
[0027] The present invention proposes a time series data prediction method based on multi-layer noise removal, which comprises the following steps:
[0028] The first step is to use RevIN to reversibly normalize the original time series data:
[0029] Reversible normalization refers to the process of restoring the normalized data to the original data through some inverse operation after normalization. In many machine learning and statistical modeling applications, normalizing data can make the data within the same scale range, which is conducive to model training and convergence. However, in some cases, it is necessary to restore the normalized data to the scale of the original data, in which case reversible normalization technology is needed.
[0030] Distribution shift is one of the main challenges that hinder accurate time series data prediction. The main reason for distribution shift is that statistical properties such as mean and variance usually change over time in time series data. To solve this problem, the present invention uses RevIN to normalize the input time series data. For each input instance X (i) ,Use formula (1) to normalize the input instance.
[0031]
[0032] in, is the learnable affine parameter vector.
[0033] The normalized series can have a more consistent mean and variance, which reduces non-stationary information and improves forecasting performance.
[0034] The second step is to use the moving average method to decompose the trend term and seasonal term from the time series data:
[0035] Seasonal trend decomposition is a commonly used time series data analysis method, which is used to split time series data into three parts: trend, seasonality, and residual. This decomposition can help us better understand the structure and characteristics of the data, thereby making more accurate predictions and analysis.
[0036] In order to separate the time pattern within the time series data, an average pooling operation is used to obtain the long-term stable trend X within the sequence. t , and fill in data at both ends of the sequence to ensure that the length of the decomposed sequence is equal to that of the original sequence. Finally, the original sequence is used to remove the trend term sequence to obtain the seasonal term sequence X s If X is the input sequence, its decomposition formula is:
[0037] X t =AvgPool(Padding(X)) (2)
[0038] X s =XX t
[0039] Where X t ,X s Represent the trend term and seasonal term of sequence X respectively.
[0040] Although the moving average can smooth the time series data and remove the noise and sudden fluctuations in the data, it cannot separate the deeper noise of the data. Therefore, the present invention decides to add the wavelet decomposition method to continue to remove the noise of the seasonal trend item.
[0041] The third step is to use wavelet decomposition to perform multi-layer decomposition on the seasonal terms and trend terms separated in the previous step:
[0042] Wavelet (Torrence & Compo, 1998) is the key to multi-scale analysis. Its advantage is that features that cannot be found at one scale can be easily found at another scale. The significance of wavelet decomposition (Soltani, 2002) is that it can decompose signals at different scales, and the choice of different scales can be determined according to different goals.
[0043] Wavelet is the key to multi-scale analysis. Its advantage is that features that cannot be found at a certain scale will be easily found at another scale. The significance of wavelet decomposition is that it can decompose signals at different scales, and the choice of different scales can be determined according to different goals. Before decomposition, the wavelet basis function must be determined first. The wavelet basis function is a set of functions of a specific shape that can be used to analyze signals in different frequency ranges. Common wavelet basis functions include Haar wavelet, Daubechies wavelet, Symlet wavelet, Morlet wavelet, etc. They have their own advantages and applicability in different application scenarios. Among them, the Symlet basis function is a wavelet function that is approximately symmetrical. It also has good regularity and can reduce the phase distortion when analyzing and reconstructing the signal to a certain extent.
[0044] The present invention selects Symlet basis function SymN (N=2, 3, ..., 8), and the formula of wavelet decomposition is as follows:
[0045] W jk =∫X(t)ψ*[(tb) / a]dt (3)
[0046] Among them, W jk represents the wavelet coefficient, x(t) represents the original signal, ψ(t) represents the wavelet basis function, a and b represent the scale factor and translation factor respectively.
[0047] The approximate coefficients (low-frequency coefficients) and detail coefficients (high-frequency coefficients) of time series data are obtained through wavelet decomposition. Among them, the approximate coefficients are used to describe the overall trend of the data, and the detail coefficients are used to describe the detailed characteristics of the data. Noise usually does not interfere with the overall trend of the data, so noise is commonly present in the detail coefficients.
[0048] When dealing with time series data of different lengths, multiple decompositions are required, that is, the approximate coefficients of each layer are further decomposed. For example, after the data X is decomposed into 3 layers, X dec ={cD1, cD2, cD3, cA3}.
[0049] Step 4: Threshold quantization is performed on each layer of data decomposed in the previous step to achieve noise removal:
[0050] Threshold processing is a common signal processing and data processing technique used to process or filter values in data that are less than or greater than a certain threshold. This method is usually used to remove noise, identify specific patterns, or perform data compression.
[0051] For the decomposed data X dec={cD1, cD2, cD3, cA3}, it needs to be thresholded to achieve the effect of denoising. The threshold confirmation formula is:
[0052]
[0053] Among them, CD1 is the detail coefficient of the first layer decomposition, and N is the sequence length.
[0054] The current quantization operation is mainly divided into soft threshold quantization and hard threshold quantization. Hard threshold quantization sets the coefficient to zero when the absolute value is less than the threshold, and remains unchanged when it is greater than or equal to the threshold. This method can achieve relatively strong noise suppression, but may cause signal distortion. Soft threshold quantization sets the coefficient to zero when the absolute value is less than the threshold, and softly compresses it when it is greater than the threshold, that is, subtracts the threshold. Soft threshold quantization can better retain the characteristics of the signal and suppress noise to a certain extent.
[0055] The present invention adopts a soft threshold quantization method, and the formula is:
[0056]
[0057] Among them, W jk is the wavelet coefficient, and T is the selected threshold.
[0058] Detail coefficient after soft threshold quantization The approximate coefficient cA3 obtained by the last layer of decomposition needs to be added to its tail to form a reconstructed sequence The formula for wavelet reconstruction is:
[0059]
[0060] Among them, t is the original data, is the processed wavelet coefficient, ψ jk is the wavelet basis function during decomposition.
[0061] It is worth noting that if the total number of input data is an odd number, the total number will become an even number after reconstruction, which is caused by the tree decomposition strategy of wavelet. Generally speaking, the last element after reconstruction can be removed to maintain the original input length.
[0062] Step 5: Linearly map and denormalize the seasonal and trend terms of the time series data processed by wavelet threshold:
[0063] In order to retain the original characteristics of the data, linear mapping operations are performed on the two parts of the seasonal term and the trend term after wavelet threshold processing, and then the results of each part are summed up to obtain the final prediction result. The formula is:
[0064]
[0065] The following table shows the long-term prediction results of the model of the present invention in four real-world data sets. The best result is represented by a bold line, and the second best is underlined. We can see that in the ETT data set, the model of the present invention achieved good performance. In the prediction experiments of the four data sets, we obtained 12 optimal results, and the remaining 4 were all second best. The evaluation indicators MSE and MAE were reduced by 13.17% and 9.31% overall. Compared with DLinear, which is also a linear model, the prediction effect of the model of the present invention is significantly ahead in the four data sets.
[0066]
[0067]
[0068] Therefore, compared with the prior art, the present invention has obvious advantages: the key to the model of the present invention is to extract multiple noises from the time series data. Previous studies have shown that using moving average to decouple the seasonal term and trend term of the sequence can reduce the interference of noise and improve the accuracy of prediction, but we believe that this is not enough. So we continue to perform wavelet filtering on the seasonal term and trend term to further remove the noise. Through double denoising, the model of the present invention achieves the best performance in single variable and multivariate prediction tasks. In addition, since we combine reversible normalization and linear mapping to eliminate the distribution shift problem in the sequence and extract the internal features of the sequence, our model is very effective in long-term series prediction for four real-world data sets.
[0069] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A time series data prediction method based on multi-layer noise stripping, characterized in that: The steps include: S1. Use RevIN to reversibly normalize the original time series data. S2. Decompose trend and seasonal terms from time series data using moving average method; S3, using wavelet decomposition to perform multi-layer decomposition on the seasonal term and trend term separated in the previous step; S4, performing threshold quantization on each layer of data decomposed in the previous step to realize noise removal; S5. Linearly map and denormalize the seasonal terms and trend terms of the time series data processed by wavelet threshold.
2. The time series data prediction method based on multi-layer noise stripping according to claim 1 is characterized in that: The step S1 is reversible normalization, that is, using a learnable affine parameter vector to normalize the input instance according to a formula to reduce non-stationary information and improve prediction performance.
3. The time series data prediction method based on multi-layer noise stripping according to claim 1 is characterized in that: The seasonal trend item decomposition in step S2 is used to split the time series data into three parts: trend, seasonality and residual, so as to make it easier to understand the structure and characteristics of the data, thereby performing more accurate prediction and analysis.
4. The time series data prediction method based on multi-layer noise stripping according to claim 1 is characterized in that: The step S2 is specifically as follows: using an average pooling operation to obtain the long-term stable trend within the sequence, and performing data padding at both ends of the sequence to ensure that the length of the decomposed sequence is equal to that of the original sequence, and finally using the original sequence to remove the trend item sequence to obtain the seasonal item sequence.
5. The time series data prediction method based on multi-layer noise stripping according to claim 1, characterized in that: The step S3 specifically includes: using the Symlet basis function to perform wavelet decomposition on the seasonal term and trend term separated in step S2, obtaining the approximate coefficient and detail coefficient of the time series data through the formula, and being able to perform multiple decompositions for time series data of different lengths.
6. The time series data prediction method based on multi-layer noise stripping according to claim 1 is characterized in that: In step S4, the soft threshold quantization method is used to perform threshold quantization on each layer of data. For the detail coefficients after soft threshold quantization, the approximate coefficients obtained by decomposing the last layer need to be added to the tail to form a reconstructed sequence; and if the total number of input data is an odd number, the last element is removed after reconstruction to maintain the original input length.
7. The time series data prediction method based on multi-layer noise stripping according to claim 1 is characterized in that: The step S5 is specifically as follows: linear mapping operations are performed on the two parts of the seasonal item and the trend item after wavelet threshold processing to retain the original characteristics of the data, and then the results of each part are summed up to obtain the final prediction result.