Time series prediction method, device, program product, storage medium and application thereof

Abstract

This invention discloses a time series forecasting method, apparatus, program product, storage medium, and its applications, relating to the field of time series analysis, to address the problems of high computational complexity and poor reliability of prediction results in time series forecasting tasks. This invention trains an inference model using time series data. This inference model extracts dependencies between variables based on the autocorrelation matrix of the input original time series; integrates these dependencies into the original time series to obtain key time-series features; learns long-term dependencies between variables from these key time-series features; and maps the time series for future time steps from these long-term dependencies. This invention achieves accurate time series forecasting with high reliability at a computationally low level.

Description

Technical Field

[0001] This invention relates to the field of time series analysis, and in particular to a time series forecasting method, apparatus, computer program product, computer-readable storage medium, and its application in forecasting weather time series or traffic flow time series. Background Technology

[0002] Time series (or dynamic series) refers to a sequence of statistical indicators arranged chronologically according to their occurrence. Time series forecasting is of great significance in the digital society and is widely used in fields such as weather forecasting, traffic flow prediction, and financial market analysis to help improve decision-making efficiency, optimize resource allocation, and address the challenges brought about by system changes. In recent years, with the development of deep learning, more and more researchers have used time series forecasting models based on deep neural networks to carry out forecasting tasks for future time steps, and have achieved certain results.

[0003] However, most existing methods use deep learning models with the Transformer architecture to perform time series prediction tasks, employing multi-layer self-attention mechanisms to capture cross-variable dependencies and learn features from historical data. This approach significantly increases computational resource overhead.

[0004] Additionally, there are solutions that use implicit modeling architectures such as Informer and Autoformer to build deep learning models. However, these models may introduce noise due to misaligned time markers, affecting their ability to capture real cross-variable interactions. Furthermore, cross-variable relationships in time series data are typically simpler and more stable than in text data. Existing attention mechanisms may be better suited for mining hidden features in text-based data, but they are prone to losing variable dependency features in time series prediction tasks, failing to fully uncover the potential patterns in time series data and impacting the reliability of prediction results.

[0005] In summary, most known studies utilize deep learning models based on attention mechanisms to capture cross-variable dependencies, which have high computational complexity and cannot fully uncover the potential patterns in time series, thus affecting the reliability of prediction results. Summary of the Invention

[0006] The purpose of this invention is to provide a time series forecasting method, apparatus, program product, storage medium, and its application, addressing all or part of the problems mentioned above, in order to solve the problems of high computational complexity and poor reliability of forecasting results in existing methods for time series forecasting tasks.

[0007] The technical solution adopted in this invention is as follows:

[0008] A time series forecasting method, comprising:

[0009] S1. Divide the time series dataset into a training set, a validation set, and a test set, wherein the time series includes multiple variables;

[0010] S2. Standardize the training set, validation set, and test set, and train the inference model using the training set and validation set with the goal of minimizing the loss. Test the trained inference model using the test set to obtain the prediction model.

[0011] S3. Use the prediction model to predict the target time series;

[0012] The inference model performs the following steps S4-S7 on the input data;

[0013] S4. Extract the dependencies between variables based on the autocorrelation matrix of the original input time series;

[0014] S5. Integrate the dependencies into the original time series to obtain key time series features;

[0015] S6. Learn the long-term dependencies between variables from the key temporal features;

[0016] S7. Map the time series of future time steps from the long-term dependencies.

[0017] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed, performs the aforementioned time series forecasting method.

[0018] To address the aforementioned problems, the present invention also provides a computer program product, including a computer program that, when run by a processor, can execute the aforementioned time series prediction method.

[0019] To address the aforementioned problems, the present invention also provides a time series forecasting apparatus, comprising a processor and a storage medium, wherein the storage medium stores computer instructions, and the processor executes the computer instructions to perform the aforementioned time series forecasting method.

[0020] On the other hand, the present invention also proposes the application of the above-mentioned time series forecasting method in weather time series or traffic flow time series forecasting.

[0021] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0022] This application presents a time series forecasting method based on time variable dependencies. It extracts the dependencies between variables using the autocorrelation matrix of the time series, effectively reducing computational complexity compared to traditional methods that employ attention mechanisms to capture these dependencies. Furthermore, the dependencies captured in this application are independent of attention mechanisms and are entirely based on the autocorrelation mechanism of the time series. This ensures that the dependency characteristics between variables are not ignored, fully revealing these dependencies and thus improving the accuracy of time series forecasts for future time steps. Attached Figure Description

[0023] The present invention will be described by way of example and with reference to the accompanying drawings, wherein:

[0024] Figure 1 This is a flowchart of the time series prediction method provided in the embodiments of this application.

[0025] Figure 2 This is a flowchart of the inference model's processing of input data provided in the embodiments of this application. Detailed Implementation

[0026] All features disclosed in this specification, or all steps in all disclosed methods or processes, may be combined in any way, except for mutually exclusive features and / or steps.

[0027] Any feature disclosed in this specification (including any appended claims and abstract) may be replaced by other equivalent or similar features, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0028] To address the issues of high computational complexity and poor prediction reliability in existing time series forecasting methods, this application provides a time series forecasting method, apparatus, program product, storage medium, and its applications, aiming to improve the reliability of time series forecasting with lower computational complexity and reduce the consumption of computing resources within computer systems.

[0029] This application provides a time series forecasting method, such as... Figure 1 As shown, the method includes the following steps:

[0030] S1. Divide the time series dataset into training set, validation set and test set.

[0031] Time series data includes multiple variables. These variables are the various indicators recorded in the time series. For example, in a weather time series, the variables might include temperature, humidity, wind speed, wind force, wind direction, rainfall / snowfall, etc. Each variable is recorded once at the same time, in chronological order. Over time, this results in an array of variables, which is the time series.

[0032] Recorded historical data is used to help predictive models uncover patterns in time series data, thereby predicting future time series. The recorded historical data is used as a time series dataset, which is then randomly divided into training, validation, and test sets according to a predetermined ratio, such as 7:2:1 or 6:2:2, with the training set having the highest proportion and ensuring it exceeds half of the total time series dataset.

[0033] For time series forecasting tasks, the inference model needs to be based on historical time series. To predict future time series Where N is the number of variables, and L and T represent the lengths of the input and output sequences, respectively, i.e., the number of future time steps. Since time series data consists of one or more data points recorded sequentially in time, rather than being independent like text samples, time series datasets are divided into fixed window sizes. Within these windows, the data is further divided into fixed input and output lengths according to a set ratio or length, thus adapting to the training of the inference model.

[0034] S2. Standardize the training set, validation set, and test set.

[0035] To ensure that the distribution of each variable has the same scale, so as to facilitate the discovery of dependencies between variables, the training set, validation set and test set are standardized in this embodiment to eliminate the influence of the dimensions of the variables.

[0036] As an optional implementation, the standardization process performed on the training, validation, and test sets in step S2 is layer normalization using a LayerNorm. Layer normalization is performed on a per-variable basis, ensuring that the time series of each variable has a standard distribution with a mean of 0 and a standard deviation of 1, thereby eliminating the influence of data dimensions and improving training stability. Assuming that the time series after standardization is... .

[0037] After standardization, the inference model has uniform input data, ensuring that the weight difference of each variable in the training process of the inference model will not be too large due to scale differences.

[0038] The inference model is trained using a training set and a validation set. Specifically, the inference model is trained using the training set, and during the training process, the inference model is tuned using the validation set. This process can be repeated multiple times.

[0039] The training process of the inference model aims to minimize the loss. As an optional implementation, this loss is the mean squared error (MSE) between the time series of future time steps output by the inference model and the actual time series of future time steps. The mean squared error is calculated as follows:

[0040] ,

[0041] In the formula, This represents the predicted value at time step i. This represents the actual value at time step i.

[0042] As an alternative implementation, the loss of the inference model can also be the sum of the mean squared error and the mean absolute error (MAE).

[0043] Training of the inference model ends after the loss of the inference model stabilizes or after a predetermined number of iterations. The performance of the inference model is validated using a test set. Once the performance meets the requirements, the trained inference model is used as the prediction model for future time series predictions.

[0044] S3. Use a prediction model to predict the target time series.

[0045] Once the prediction model is obtained, in the same scenario as the statistical time series dataset, the time series of the current time step is input into the prediction model, and the time series of future time steps (with a fixed length) can be predicted.

[0046] Inference models predict time series data by learning latent features from input data. For example... Figure 2 As shown in the embodiment of this application, the inference model performs steps S4-S7 on the input data. In each step, relevant calculations are performed through model parameters. The purpose of training is to adjust the model parameters to the optimal value so that the calculation results are more consistent with reality.

[0047] S4. Extract the dependencies between variables based on the autocorrelation matrix of the original input time series.

[0048] As an optional implementation, step S4 includes the following sub-steps:

[0049] S4-1: Calculate the autocorrelation matrix of the original time series.

[0050] Taking the original time series of the training set as an example, the standardized time series is: ,Will Its transpose Multiply them, calculate the inner product, and obtain the time series. The inner product matrix is ​​a form of autocorrelation matrix that represents the correlation between each variable and other variables at different time steps in a time series. This inner product operation measures the similarity between each variable and can reflect the dependencies between variables to some extent.

[0051] Time series The inner product matrix may exhibit significant variations. Greater variations can lead to the neglect of subtle features during feature extraction, hindering the full exploration of potential dependencies between variables. To address this, this embodiment of the application performs a normalization operation on the inner product matrix, converting it into an autocorrelation matrix representing similarity. This normalizes the element values, making the matrix smoother and more consistent in scale.

[0052] In some feasible implementations, the click matrix is ​​normalized by applying a Softmax operation to obtain the autocorrelation matrix. This is represented as:

[0053] .

[0054] The softmax operation is used to proportionally adjust the correlation between each variable, ensuring the accuracy of the autocorrelation matrix. The values ​​of all elements are within a certain range, which not only reflects the dependencies between variables but also provides a foundation for subsequent feature extraction.

[0055] S4-2: On the autocorrelation matrix Perform linear transformations and normalization operations to obtain the dependencies between variables.

[0056] In step S4-1, the autocorrelation matrix is ​​calculated. The dependencies between variables can be initially extracted; here, the autocorrelation matrix is ​​used. Perform fine-grained modeling to more accurately capture the dependencies between variables.

[0057] by To represent the dependency relationship between variables, in this embodiment, the linear operation is used to represent the linear transformation, and the autocorrelation matrix after the linear transformation is... Normalization is achieved by performing a Softmax operation. Then:

[0058] .

[0059] S5. Integrate the dependencies into the original time series to obtain key time series features.

[0060] As an optional implementation, step S5 includes the following sub-steps:

[0061] S5-1: Use dependencies to weight the original time series to obtain the first time series.

[0062] The dependencies between variables are During the training phase, the original time series is the historical time series X, and the dependency relationship will be used here. The original time series X is weighted, and the weighted time series is called the first time series. It is represented as:

[0063] .

[0064] This weighting operation can capture dependencies. Integrate it into the original time series X.

[0065] S5-2, First time series Connect it to the original time series X.

[0066] In some feasible implementations, the sequence connection operation is performed by designing an embedding layer within the inference model. This operation integrates the captured dependencies into the input sequence of the inference model, allowing the model to retain the characteristics of the original time series while effectively introducing cross-variable dependencies during further processing. This operation enables the inference model to better understand and learn the complex relationships between variables, improving inference accuracy.

[0067] Let Concatenate represent the join operation. The join operation described above is represented as follows:

[0068] .

[0069] S5-3. Perform a linear transformation on the concatenated time series to obtain key time series features.

[0070] by To represent key time-series features, we have:

[0071] .

[0072] in, D represents the feature dimension.

[0073] S6. From key time-series characteristics Learn the long-term dependencies between variables.

[0074] As an alternative implementation, a feedforward neural network (FFN) is used to learn key temporal features. Long-term dependencies between variables.

[0075] Step S6, in some feasible implementations, includes the following sub-steps:

[0076] S6-1: Key time-series features Perform layer normalization.

[0077] Layer normalization operation is represented as Layer normalization can unify feature scale.

[0078] S6-2: Use at least one level of feedforward neural network to learn the long-term dependencies between variables from key temporal features after layer normalization.

[0079] As an optional implementation, the feedforward neural network sequentially performs linear transformation, activation, and another linear transformation on the input time series, then establishes skip connections with the input time series, followed by layer normalization. This can be represented as:

[0080] ,

[0081] In the formula, FNN represents the output of the feedforward neural network, F represents the input of the feedforward neural network, and Activation represents the activation operation.

[0082] Multilevel feedforward neural networks are represented by FNNs, which are constructed by nesting multiple FNNs.

[0083] by To represent the learned long-term dependencies, we have:

[0084] .

[0085] S7, Due to long-term dependency It maps the time series of future time steps.

[0086] The predicted time series for the future time step has a length of T. In step S7, a projection layer is designed in the inference model. The extracted long-term dependencies are mapped to a time series of length T through this projection layer using a linear transformation. This is the time series for the future time step. This can be represented as:

[0087] .

[0088] Based on the ideas in this application, this application also provides a computer-readable storage medium storing a computer program, which can be used to execute the time series prediction method of the above embodiments.

[0089] In another aspect, embodiments of this application also provide a computer program product, including a computer program that, when run by a processor, can execute the time series prediction method described above.

[0090] Furthermore, embodiments of this application also provide a time series forecasting apparatus, including a processor and a storage medium, wherein the storage medium stores computer instructions, and the processor executes the computer instructions to perform the aforementioned time series forecasting method.

[0091] In addition, this application also discloses the application of the above-mentioned time series forecasting method in weather time series forecasting, thus obtaining a weather time series forecasting method.

[0092] The weather time series dataset was collected by the Max Planck Institute for Biogeochemistry. This dataset consists of 21 meteorological indicators collected every ten minutes in 2020, including various meteorological indicators such as air humidity and temperature, which also serve as variables in a multivariate time series.

[0093] Given an input sequence with a backoff window of 960 min (meaning the length of the input sequence is 96), the time series to be predicted has a length of 720 and 21 variables.

[0094] Based on the above scenario, in some feasible implementations, the weather time series forecasting method includes:

[0095] S1. Determine the training batch size, select GPU training, set the fallback window length to 96, the prediction window length to 720, the number of variables to 21, and set the training set: validation set: test set = 7:1:2.

[0096] S2. Standardize the training, validation, and test sets. Train the inference model using the training and validation sets, setting appropriate learning rates and training epochs until the inference model converges. Test the trained inference model using the test set to obtain the prediction model.

[0097] S3. Use a forecasting model to predict the weather time series for future time steps, with a forecast length of 720.

[0098] In addition, this application also discloses the application of the above-mentioned time series prediction method in traffic flow time series prediction, thus obtaining a traffic flow time series prediction method.

[0099] Traffic flow datasets consist of traffic monitoring values ​​from sensors at each intersection. A sampling frequency of one hour can be set, with each hourly sensor value representing a multivariate value at a single point in time. For example, a publicly available dataset contains hourly road occupancy measurements from 862 sensors on highways in the San Francisco Bay Area, collected from January 2015 to December 2016, thus creating a traffic flow dataset.

[0100] In this scenario, some feasible implementations of the traffic flow time series prediction method include:

[0101] S1. Determine the training batch size, select GPU training, set the fallback window length to 96, set the prediction window length to 96 (i.e., predict traffic flow in the next 96 hours), set the number of variables to 862, and set the training set: validation set: test set = 7:1:2 (other ratios are also acceptable).

[0102] S2. Standardize the training, validation, and test sets. Train the inference model using the training and validation sets, setting appropriate learning rates and training epochs until the inference model converges. Test the trained inference model using the test set to obtain the prediction model.

[0103] S3. Use a prediction model to predict the traffic flow time series for future time steps, with a prediction length of 96.

[0104] This invention is not limited to the specific embodiments described above. The invention extends to any new feature or combination disclosed in this specification, as well as any new method or process step or combination disclosed herein.

Claims

1. A time series prediction method for weather time series prediction, characterized in that, The method comprises the following steps: S1, determining a rollback window, a prediction window length and a variable number of an input time series; dividing a time series dataset into a training set, a validation set and a test set, wherein the time series comprises a plurality of variables including temperature, humidity, wind speed, wind size, wind direction and rainfall / snowfall, each variable is recorded once at the same time according to time sequence, and an array of a group of variables, i.e. a time series, is obtained according to the time sequence; S2, performing standardization processing on the training set, the validation set and the test set, training an inference model by using the training set and the validation set as an objective to minimize loss, and testing the trained inference model by using the test set to obtain a prediction model; S3, predicting a weather time series of a future time step by using the prediction model; The inference model performs the following steps S4-S7 on input data: S4, extracting a dependency relationship between variables based on an autocorrelation matrix of an input original time series, comprising: S4-1, calculating the autocorrelation matrix of the original time series, comprising: multiplying the standardized original time series with its transpose matrix to obtain the autocorrelation matrix; S4-2, performing linear transformation and normalization operation on the autocorrelation matrix to obtain the dependency relationship between variables; S5, integrating the dependency relationship into the original time series to obtain a key time sequence feature, comprising: S5-1, weighting the original time series by using the dependency relationship to obtain a first time sequence; S5-2, connecting the first time sequence with the original time sequence; S5-3, performing linear transformation on the connected time sequence to obtain the key time sequence feature; S6, learning a long-term dependency relationship between variables from the key time sequence feature, comprising: S6-1, performing layer normalization operation on the key time sequence feature; S6-2, learning the long-term dependency relationship between variables from the layer-normalized key time sequence feature by using at least one level feedforward neural network; wherein the feedforward neural network sequentially performs linear transformation, activation, linear transformation on the input time sequence, then performs skip connection with the input time sequence, and then performs layer normalization operation; S7, mapping a time sequence of a future time step from the long-term dependency relationship.

2. The time series prediction method of claim 1, wherein, The loss is a mean square error between a time sequence of a future time step output by the inference model and an actual time sequence of the future time step.

3. The time series prediction method of claim 1, wherein, The standardization processing in step S2 is layer normalization processing.

4. A computer readable storage medium storing a computer program, characterized in that, Running the computer program can execute the time series prediction method according to any one of claims 1-3.

5. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to execute the time series prediction method according to any one of claims 1-3.

6. A time series prediction apparatus comprising a processor and a storage medium having stored therein computer instructions, wherein, The processor executes the computer instructions to execute the time series prediction method according to any one of claims 1-3.

Images