Time sequence prediction method based on connection function

Abstract

The invention discloses a time sequence prediction method based on a connection function. The method comprises the following steps: firstly, generating a backtest data set by using GhonTS and dividing a training set, a verification set and a timestamp; and then, through PyTorchTS, a prediction framework with a Mangbar model as a skeleton is constructed, the prediction framework comprises a connection function encoder, a marginal cumulative distribution encoder, a depth Sigmoid stream and a connection function decoder, and finally, a prediction sequence is obtained by using a connection function. According to the method, the joint distribution modeling capability and the time sequence prediction precision of the complex time sequence are remarkably improved through a double-path coding structure.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning time series prediction, and particularly relates to a time series prediction method based on a connection function and a Mamba model. BACKGROUND

[0002] Time series prediction refers to a process of predicting future values by analyzing historical time series data. This process requires analyzing the inherent temporal patterns in the data (such as trends and seasonality) and estimating future values based on historical observations. The goal is to support decision-making by providing information about uncertain futures, such as predicting future demand changes or price fluctuations. Time series prediction can be achieved using various methods, including statistical methods, machine learning, and deep learning methods, which focus on capturing the characteristics of data evolution over time. Multivariate time series prediction. Multivariate prediction refers to predicting by considering multiple variables simultaneously. For example, in weather forecasting, predicting the next day's temperature by integrating multiple variables such as temperature, humidity, and wind speed is a typical application of multivariate prediction. This method can capture complex relationships by integrating the interactions and correlations between multiple variables, thereby providing higher prediction accuracy. However, such models are usually more complex, require more data support, and are more difficult to process, and may face higher risk of overfitting due to increased model complexity.

[0003] Time series data is a collection of ordered data points collected at fixed time intervals, representing observations of phenomena that change over time. Its time continuity allows understanding and analyzing the evolution of things based on the time sequence. Each data point represents the state or value at a specific time, and through observation of these data points, long-term trends, seasonality, periodicity, and irregular fluctuations can be identified. These patterns provide valuable information support for predicting future values or detecting state changes at key points. By extracting and studying meaningful information provided by time series data, it can be applied to practical scenarios to help solve many challenges in various disciplines. Time series data exists in a wide range of fields, each exhibiting unique characteristics. This diversity is due to the nature of time series data, which addresses real-world problems in various fields and can exhibit highly different patterns depending on time and environmental factors. For example, stock price data usually shows a long-term upward or downward trend. Due to complex influences such as economic conditions, company performance, and policy changes, stock market fluctuations tend to develop in a certain direction over a long period of time. In contrast, electricity consumption data exhibits a clear periodicity throughout the day, with higher consumption during the day and lower consumption at night. There is also a seasonal pattern, with different consumption rates in summer and winter. Therefore, an accurate time series prediction method is urgently needed. SUMMARY

[0004] The application aims to provide a time series prediction method based on a connection function, which improves the time series prediction accuracy.

[0005] In order to achieve the object of the application, the application provides a time series prediction method based on a connection function, characterized in that it comprises the following steps:

[0006] Step 1: Obtain time series data by using the GluonTS generated backtest dataset function, and divide the data into several groups of training data, validation data and data timestamps;

[0007] Step 2: Use the PyTorchTS package to build a training and inference framework of a time series prediction model based on a Mamba model; the time series prediction model comprises a subnetwork: a connection function encoder, a marginal cumulative distribution encoder, a deep S-shaped flow and a connection function decoder;

[0008] Step 3: After encoding the training data, obtain the connection function encoding information and the marginal cumulative function encoding information of the time series by using the connection function encoder and the marginal cumulative function encoder respectively;

[0009] Step 4: Estimate the marginal cumulative distribution of each variable of the training data by processing the marginal cumulative function encoding information through the deep S-shaped flow;

[0010] Step 5: Obtain the connection distribution of the time series by using the time-variable double attention decoder on the marginal cumulative distribution of each variable and the connection function encoding information;

[0011] Step 6: Obtain the prediction result of the time series by sampling according to the connection distribution.

[0012] Compared with the prior art, the application has the following advantages: the application makes full use of the advantages of the connection function method in separately modeling the variable distribution and the connection distribution of the time series, and the advantages of the Mamba model in having an approximate linear complexity for long sequence models, effectively improving the problem that neural networks are difficult to model long time series and have low prediction accuracy.

[0013] To make the function and structure of the application clearer, the following further describes the application in combination with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the application, and form a part of the application. The illustrative embodiments of the application and their descriptions serve to explain the application, and do not constitute an improper limitation on the application. In the drawings:

[0015] Figure 1 is a flow structure block diagram of the application. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0017] The time series prediction method based on a connection function of the present application combines Figure 1 , and comprises the following steps:

[0018] Step 1, acquiring time series data by using a generated backtest dataset function of GluonTS (GluonTS is a Python library focusing on probabilistic time series modeling), and dividing the time series data into several groups of training data, validation data and timestamps of data;

[0019] Step 2, constructing a training and inference framework of a time series prediction model with a Mamba model as a skeleton by using a PyTorchTS (PyTorchTS is an open source time series prediction framework based on PyTorch) package; the time series prediction model comprises a subnetwork: a connection function encoder, a marginal cumulative distribution encoder, a deep S-shaped flow and a connection function decoder; the connection function encoder and the marginal cumulative distribution encoder take the Mamba model as a skeleton network, and respectively capture features of the time series from the time and variable directions; the deep S-shaped flow (deep Sigmoid flow) is to map data from different scales to a uniform distribution on the interval; the connection function decoder is to summarize data in the network and generate a connection distribution of the time series.

[0020] Step 3, obtaining connection function encoding information and marginal cumulative function encoding information of the time series by respectively processing the training data through the connection function encoder and the marginal cumulative function encoder after encoding processing of the training data;

[0021] Step 4, estimating the marginal cumulative distribution of each variable of the training data by processing the marginal cumulative function encoding information through the deep S-shaped flow;

[0022] Step 5, obtaining the connection distribution of the time series by the time-variable dual attention decoder through the marginal cumulative distribution of each variable and the connection function encoding information;

[0023] Step 6, obtaining the prediction result of the time series by sampling according to the connection distribution.

[0024] ​The step 3 comprises the following steps:

[0025] Step 3-1, the training data is processed by data encoding (embedding layer) to obtain vector embedding;

[0026] The data encoding process generates a vector embedding for each element (i.e. label) in each time series;

[0027] Step 3-2, the timestamp of the data is added to the vector embedding by position encoding to obtain the final embedding:

[0028] Step 3-3, the final embedding is respectively input into the connection function encoder and the marginal cumulative function encoder to obtain the connection function encoding information and the marginal cumulative function encoding information of the time series; the marginal cumulative function encoding information is used as a parameter in the subsequent process in the deep S-shaped flow; the Mamba model of the marginal cumulative distribution encoder is trained along the time direction, avoiding interference between variables, thereby obtaining a more accurate marginal cumulative distribution; the connection function encoder acts on the processing of the interaction relationship between variables, and at this time the Mamba model of the connection function encoder is trained along the variable direction, avoiding interference between variables, thereby obtaining a more accurate marginal cumulative distribution.

[0029] The step 3-1 is specifically as follows:

[0030] ;

[0031] Wherein, is the vector embedding, is the dimension of the neural network embedding layer, is the position index of the input data, is the real number field; is the parameter of the neural network, is the training data, is the relevant covariate, is the mask of the training data, which is composed of a sequence of 0 and 1, and the training data and the mask are multiplied element by element to mask the part that needs to be predicted in the prediction network, so that it is invisible to the network.

[0032] The step 3-2 is specifically as follows:

[0033] ;

[0034] Wherein, is the position encoding, is the timestamp of the data, is the final embedding.

[0035] The step 4 is specifically as shown in the following formula:

[0036]

[0037] wherein, is the marginal cumulative distribution of each variable, is the marginal cumulative function encoding information, is a deep S-shaped flow network taking as the parameter.

[0038] The step 5 is specifically as shown in the following formula:

[0039]

[0040] wherein, is the connection distribution (the output is the cumulative probability density function of the sequence, which represents the probability of the sequence element being a specific value, and the distribution is a reversible mapping, which can obtain the probability of the value from a value, and can also obtain the value from the size of the probability), is the connection function encoding information, is a time-variable dual attention decoder network.

[0041] The step 6 specifically comprises the following steps:

[0042] Step 6-1, sampling from a uniform distribution to obtain the probability of each value of the predicted sequence wherein represents the length of the predicted value;

[0043] Step 6-2, transforming each sampled value (probability size) according to the marginal cumulative distribution to obtain the predicted value of the time sequence.

[0044] The step 6-2 is specifically as shown in the following formula:

[0045] Y;

[0046] wherein, is the inverse function of the marginal cumulative distribution function, which represents the process of obtaining a specific value from a probability value, and Y is the predicted value of the time sequence.

[0047] Table 1 is the continuous ranking probability score obtained by using 6-fold cross-validation on five open source data sets (electricity, fred-md, kdd-cup, solar-10min, traffic) by the present application, and is compared with eight models of Auto-ARIMA, ETS, TempFlow, SPD, TimeGrad, GPVar, TACTiS, TACTiS-2, as shown below:​​

[0048]

[0049] The above table uses the sum of mean continuous ranking probability scores to represent the prediction accuracy of the model, which can measure the accuracy of the model prediction distribution at the same time. The lower the value, the better the fitting effect of the model on the multivariate joint distribution (the best result is highlighted). The results show that the model (Ours) ranks first in 3 of the 5 data sets (Electricity, Fred-md, Solar), with a comprehensive ranking of 1.6, proving that the model not only predicts accurately, but also estimates risk and uncertainty (joint distribution) more realistically and reliably.

[0050] It should be noted that, in this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0051] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A time series prediction method based on a connection function, characterized by, The method comprises the following steps: Step 1, obtaining time series data by using the generating backtest dataset function of GluonTS, and dividing the time series data into training data, validation data and data timestamps; Step 2, constructing a training and inference framework of a time series prediction model based on a Mamba model by using a PyTorchTS package; the time series prediction model comprises a subnetwork: a connection function encoder, a marginal cumulative distribution encoder, a deep S-shaped flow and a connection function decoder; Step 3, after the training data are encoded, the connection function encoding information and the marginal cumulative function encoding information of the time series are obtained by using the connection function encoder and the marginal cumulative function encoder respectively; Step 4, the marginal cumulative distribution of each variable of the training data is estimated by using the deep S-shaped flow based on the marginal cumulative function encoding information; Step 5, the connection distribution of the time series is obtained by using a time-variable dual attention decoder based on the marginal cumulative distribution of each variable and the connection function encoding information; Step 6, the prediction result of the time series is obtained by sampling based on the connection distribution. 2.The time series prediction method based on connection function according to claim 1, characterized in that, The step 3 comprises the following steps: Step 3-1, obtaining a vector embedding by data encoding processing on the training data; The data encoding processing generates a vector embedding for each element in each time series; Step 3-2, adding the timestamp of the data to the vector embedding by position encoding to obtain a final embedding: Step 3-3, inputting the final embedding into the connection function encoder and the marginal cumulative function encoder respectively to obtain the connection function encoding information and the marginal cumulative function encoding information of the time series. 3.The time series prediction method based on connection function according to claim 2, characterized in that, The step 3-1 is specifically as follows: ； wherein, is a vector embedding, is a neural network embedding layer dimension, is a position index of the input data, is a real number field; is a parameter of the neural network, is training data, is a relevant covariate, is a mask of the training data.

4. The time series forecasting method based on connectionist function according to claim 3, characterized in that, The step 3-2 is specifically as follows: ； wherein, is a position encoding, is a timestamp of the data, is the final embedding.

5. The time series forecasting method based on a connection function according to claim 4, characterized in that, The step 4 is specifically as follows: ； wherein, is the marginal cumulative distribution of each variable, is the marginal cumulative function encoding information, is a deep sigmoidal flow network with as parameters.

6. The time series forecasting method based on a connection function according to claim 5, characterized in that, The step 5 is specifically as follows: ； wherein, is a connection distribution, is a connection function encoding information, is a time-variable dual attention decoder network.

7. The time series forecasting method based on a connection function according to claim 6, characterized in that, The step 6 specifically comprises the following steps: Step 6 - 1, Obtain the probability on each value of the prediction sequence from the uniform distribution where denotes the length of the prediction value;​ Step 6-2, transforming each sampling value based on the marginal cumulative distribution to obtain the prediction value of the time series.

8. The time series prediction method based on a connection function according to claim 7, characterized in that, The step 6-2 is specifically as follows: Y； wherein, is the inverse function of the marginal cumulative distribution function, representing the process of obtaining a specific value from a probability value, Y is the predicted value of the time series.

Images