Residual random-based long-time sequence prediction method
By adopting a loss function based on the Hilbert-Schmidt independence criterion in long time series prediction and combining it with L2 loss, the problems of noise fitting and neglect of internal rules in the existing technology are solved, and efficient and accurate long time series prediction is achieved.
Patent Information
- Application Number
- CN202510894146.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-09-26
AI Technical Summary
Existing long time series prediction methods are prone to fitting noise when using mean square error (MSE) as the loss function, ignoring the inherent laws of time series, resulting in a decline in model performance in long time series prediction.
A loss function based on the Hilbert-Schmidt independence criterion is used in combination with L2 loss to perform long time series prediction through a linear model, reduce noise interference, and improve the model's ability to capture the inherent laws of time series.
It significantly improves the accuracy and performance of long time series prediction, reduces the number of network parameters and inference time of the model, and reduces the risk of overfitting.
Smart Images
Figure CN120706574A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of data sequence prediction, and in particular relates to a long time series prediction method based on residual randomness. Background Art
[0002] A time series refers to ordered data recorded in chronological order. In time series forecasting, historical data is analyzed to reveal underlying patterns and trends, thereby predicting future behavior. With the continuous advancement of technology, time series forecasting has been widely used in various fields, including finance, industry, disease control, and meteorology. Specifically, by analyzing and modeling existing data, it is possible to predict the changing trends of time series over longer time spans, thus achieving long time series forecasting.
[0003] Real-world time series data is often highly complex, exhibiting characteristics such as nonlinearity, nonstationarity, long-term dependencies, and noise. Deep learning models, with their powerful nonlinear modeling capabilities, can effectively handle these complex features, thereby more accurately capturing temporal variations in time series. In recent years, Transformer- and MLP-based models have been widely used in time series analysis. However, these methods generally use mean squared error (MSE) as a loss function, which has certain limitations, especially in long time series forecasting.
[0004] In long time series prediction tasks, using MSE as a loss function has two major drawbacks: (1) MSE causes the model to fit the inherent noise in the time series, thereby affecting the overall prediction performance of the model; (2) MSE fails to fully consider the intrinsic characteristics of the time series and is prone to missing some valuable information during the backpropagation process, thereby damaging the model's ability to capture long-term dependencies. In addition, as a point-level loss function, MSE only focuses on the prediction error of each individual time point, often ignoring the inherent temporal consistency and pattern of time series data. Summary of the Invention
[0005] In view of the problems that existing long time series prediction methods cannot fully learn the relationship between sequences, have fitting noise and ignore the inherent laws of time series, the present invention provides a long time series prediction method based on residual randomness, which uses a linear model for long time series prediction and gives full play to its advantage of efficient reasoning speed. On this basis, a loss function based on the Hilbert-Schmidt independence criterion is designed to overcome the shortcomings of traditional MSE in time series prediction. This loss function can more accurately reflect the long-term dependencies in time series and reduce the impact of noise interference on model performance. In this way, not only the number of network parameters and reasoning time are reduced, but also the prediction accuracy is significantly improved. The loss function based on the Hilbert-Schmidt independence criterion can better capture the inherent laws of time series, thereby achieving efficient and accurate long time series prediction.
[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:
[0007] A long time series prediction method based on residual randomness includes the following steps:
[0008] Step 1: Obtain a time series dataset and perform z-standardization on the time series in the time series dataset;
[0009] Step 2: Input the z-normalized time series into a pre-trained long time series prediction model and output a prediction result; the long time series prediction model includes an input, a backbone network, and an output; the training of the long time series prediction model uses L2 loss combined with the correlation between the residual and random noise as a loss function;
[0010] Step 3: Denormalize the prediction results output by the long time series prediction model to obtain the final prediction sequence.
[0011] Furthermore, the specific operation of z normalization in step 1 is:
[0012] The time series is standardized using the mean and standard deviation of the time series dataset. The calculation formula is as follows:
[0013]
[0014] in, represents the z-normalized time series, represents the original time series, , T represents the length of the input time series, N represents the number of time series, represents the mean of the time series dataset, Represents the standard deviation of the time series dataset.
[0015] Furthermore, the specific operations of the backbone network in step 2 are:
[0016] Perform one-dimensional average pooling on the input time series to decompose the time series into trend parts and seasonal sections , the trend part and seasonal sections Put them into the feature extractor Linear respectively to get and , the output result is the sum of the two, and the calculation formula is as follows:
[0017]
[0018]
[0019]
[0020]
[0021]
[0022] in, For input, For output, is a linear neural network.
[0023] Furthermore, the loss function in step 2 is:
[0024]
[0025]
[0026] Among them, the first term is L2 loss, and the HSIC in the second term is the Hilbert-Schmidt independence criterion, which is used to measure the correlation between the residual and the random noise. The more correlated the residual is with the random noise, the more sufficient the information learned by the model is. is a hyperparameter, is a random noise sequence, is the residual, represents the trace function, represents the Gaussian kernel function, The value of is the number of time series N, express The identity matrix of order.
[0027] Furthermore, the specific operation of denormalization in step 3 is:
[0028] The time series is denormalized using the mean and standard deviation of the time series dataset. The calculation formula is as follows:
[0029]
[0030] in, represents the time series after inverse normalization, represents the output of the model, , T represents the length of the input time series, N represents the number of time series, represents the mean of the time series dataset, Represents the standard deviation of the time series dataset.
[0031] Compared with the prior art, the present invention has the following advantages:
[0032] (1) Compared with the prediction method based on Transformer, the long time series prediction method based on residual randomness proposed in this paper has fewer network parameters and inference time, which can reduce the risk of model overfitting and significantly improve the prediction performance.
[0033] (2) Compared with the simple linear model, the long time series prediction method based on the Hilbert-Schmidt independence criterion proposed in this invention adopts the Hilbert-Schmidt independence criterion with better information extraction ability as the loss function, which can more accurately capture the long-term dependence in the time series, reduce noise interference, and improve the prediction accuracy, thereby improving the performance of the model.
[0034] The method of the present invention is easy to implement, and its application value is mainly reflected in the following aspects:
[0035] (1) For complex time series features such as nonlinearity, non-stationarity, long-term dependence and noise interference, efficient and accurate long-term series prediction can be achieved.
[0036] (2) Provide loss function options with better information extraction capabilities for long time series prediction tasks to improve prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is the overall flow chart of the present invention;
[0038] Figure 2 It is a network framework diagram of the present invention;
[0039] Figure 3 This is an example diagram of some original data and predicted data obtained from the etth1 dataset;
[0040] Figure 4 This is an example diagram of some original data and predicted data obtained from the etth2 dataset;
[0041] Figure 5 This is an example diagram of some original data and predicted data obtained from the ettm1 dataset;
[0042] Figure 6 This is an example diagram of some original data and predicted data obtained from the ettm2 dataset. DETAILED DESCRIPTION
[0043] In order to further illustrate the technical solution of the present invention, the present invention is further described below through examples.
[0044] Example 1
[0045] A long time series prediction method based on residual randomness in this embodiment includes the following steps:
[0046] Step 1: Get a time series dataset and perform z-standardization on the time series using the mean and standard deviation of the time series dataset. The calculation formula is as follows:
[0047]
[0048] in, represents the z-normalized time series, represents the original time series, , T represents the length of the input time series, N represents the number of time series, represents the mean of the time series dataset, Represents the standard deviation of the time series dataset.
[0049] In this embodiment, the time series data set is divided into a training set and a validation set in a ratio of 8:2. The training set is used for model training, and the validation set is used to constrain the degree of model training.
[0050] Step 2: Input the z-normalized time series into a pre-trained long time series prediction model and output the prediction result; the long time series prediction model is as follows: Figure 2 As shown, it includes input, backbone network and output, where the specific operations of the backbone network are:
[0051] Perform one-dimensional average pooling on the input time series to decompose the time series into trend parts and seasonal sections , the trend part and seasonal sections Put them into the linear neural network Linear to get and , the output result is the sum of the two, and the calculation formula is as follows:
[0052]
[0053]
[0054]
[0055]
[0056]
[0057] in, For input, For output, the in_features and out_features of the linear neural network Linear are the length of the historical sequence and the length of the predicted sequence respectively.
[0058] This embodiment uses a training set to train the time series prediction model, uses L2 loss combined with the correlation between the residual and random noise as the loss function, continuously optimizes the model weights, and uses the loss of the validation set to constrain the degree of model training. If the loss of the validation set does not decrease for three consecutive times, training is stopped.
[0059] The loss function is calculated as follows:
[0060]
[0061]
[0062] Among them, the first term is L2 loss, and the second term HSIC is Hilbert-Schmidt independence criterion. is a hyperparameter ( =10), is a random noise sequence, The amplitude of is (0, 1), is the residual, represents the trace function, represents the Gaussian kernel function, The value of is the number of time series N, express The identity matrix of order.
[0063] Step 3: Denormalize the prediction results output by the long time series prediction model to obtain the final prediction sequence. The calculation formula is as follows:
[0064]
[0065] in, represents the time series after inverse normalization, represents the output of the model, , T represents the length of the input time series, N represents the number of time series, represents the mean of the time series dataset, Represents the standard deviation of the time series dataset.
[0066] This example conducts experiments on four datasets: etth1, etth2, ettm1, and ettm2. The epoch of the experiment is 10, the batch size is 32, the observation length is 336, the prediction length of etth1, etth2, and ettm2 is 192, and the prediction length of ettm1 is 336. The results are shown in Figure 2. Figures 3 to 6 , it can be seen that the present invention can achieve accurate prediction effect.
[0067] The foregoing shows and describes the principal features and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present invention is defined by the appended claims, not the foregoing description, and all variations that come within the meaning and range of equivalents of the claims are intended to be embraced therein.
[0068] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.
Claims
1. A long time series prediction method based on residual randomness, characterized in that: The following steps are involved: Step 1: Obtain a time series dataset and perform z-standardization on the time series in the time series dataset; Step 2: Input the z-normalized time series into a pre-trained long time series prediction model and output a prediction result; the long time series prediction model includes an input, a backbone network, and an output; the training of the long time series prediction model uses the Hilbert-Schmidt independence criterion combined with the mean of the residuals as a loss function; Step 3: Denormalize the prediction results output by the long time series prediction model to obtain the final prediction sequence.
2. A long time series prediction method based on residual randomness according to claim 1, characterized in that: The specific operation of z standardization in step 1 is: The time series is z-standardized using the mean and standard deviation of the time series dataset. The calculation formula is as follows: in, represents the z-normalized time series, represents the original time series, , T represents the length of the input time series, N represents the number of time series, represents the mean of the time series dataset, Represents the standard deviation of the time series dataset.
3. The long time series prediction method based on residual randomness according to claim 1, characterized in that: The specific operations of the backbone network in step 2 are: Perform one-dimensional average pooling on the input time series to decompose the time series into trend parts and seasonal sections , the trend part and seasonal sections Put them into the feature extractor Linear respectively to get and , the output result is the sum of the two, and the calculation formula is as follows: in, For input, For output, is a linear neural network.
4. The long time series prediction method based on residual randomness according to claim 1, characterized in that: The loss function in step 2 is: Among them, the first term is L2 loss, and the HSIC in the second term is the Hilbert-Schmidt independence criterion, which is used to measure the correlation between the residual and the random noise. The more correlated the residual is with the random noise, the more sufficient the information learned by the model is. is a hyperparameter, is a random sequence, is the residual, represents the trace function, represents the Gaussian kernel function, The value of is the number of time series N, express The identity matrix of order.
5. The long time series prediction method based on residual randomness according to claim 1, characterized in that: The specific operation of the denormalization in step 3 is: The time series is denormalized using the mean and standard deviation of the time series dataset. The calculation formula is as follows: in, represents the time series after inverse normalization, represents the output of the model, , T represents the length of the input time series, N represents the number of time series, represents the mean of the time series dataset, Represents the standard deviation of the time series dataset.