Multivariable time series data prediction method based on depth sub-feature decomposition

Abstract

The invention discloses a multivariable time series data prediction method based on depth sub-feature decomposition. The method comprises the following steps: firstly, recording and collecting time series data; and secondly, constructing a multivariable time series data prediction model, inputting the collected time series data, and carrying out characteristic decomposition to obtain a prediction result. And finally, training and testing the multivariable time series data prediction model. According to the method, the significance of the key time variable on the data level is improved, decoupling of complex mixed time sequence data is achieved, the time variable capturing difficulty is reduced, and the prediction precision and robustness of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the field of time series data prediction, and specifically relates to a multivariate time series data prediction method based on deep sub-feature decomposition. Background Art

[0002] In the digital age, the widespread deployment of IoT sensors has led to an exponential growth in the amount of time series data. These data are composed of specific indicators recorded in chronological order, and are the external manifestation of the complex internal mechanisms of the observed object that change over time. The main goal of the time series forecasting method is to use this data to capture its internal complex mechanisms and make predictions for the future. That is, to use limited historical data to predict performance in the future. The importance of time series forecasting is self-evident, and it plays a key role in many important fields. The following are some possible application scenarios:

[0003] Retail sales forecasting: By analyzing past merchandise sales data, companies can predict future sales trends and optimize inventory management and marketing strategy formulation. Traffic flow forecasting: Using historical traffic data analysis to predict future traffic volume helps improve traffic planning and reduce congestion. Natural disaster early warning system: Monitor and interpret environmental indicators to predict the occurrence of natural disasters, reduce the extent of damage, and strengthen emergency response mechanisms.

[0004] The application of time series forecasting methods is not limited to the above fields; it also plays an irreplaceable and important role in many other industries including medicine, service industry, manufacturing industry and agriculture. However, due to the complexity and non-stationarity of the real world or system, the observed time series usually presents a complex coupling pattern, in which the changes are often highly mixed and intertwined. This characteristic poses a huge challenge to the forecasting task, because traditional methods have difficulty in effectively separating and capturing these deep mixed patterns.

[0005] In the practice of fine-grained prediction models, in order to solve complex time patterns, many deep prediction models usually involve some special designs. These widely recognized design paradigms mainly include decomposition, mixing, and multi-periodicity analysis. As a classic time series analysis technique, decomposition is usually embedded in the model as a basic module to decompose complex time patterns into more predictable parts, such as cycles and trends, which is beneficial to the prediction process. However, existing methods such as Autoformer and Pyraformer have not achieved sufficient decomposition of deeply coupled time variables, and still face severe challenges in dealing with deeply coupled time variables, because deeply coupled time variables may involve more complex time dependencies, which may be difficult to fully capture and utilize in traditional decomposition. Summary of the invention

[0006] The present invention aims to improve the defect that the existing prediction methods are difficult to fully decompose the deeply coupled time variables, resulting in the inability to capture effective dependencies. From the perspectives of sub-features, multi-scales and variable modeling, a multivariate time series data prediction method based on deep sub-feature decomposition is proposed, which includes the following steps:

[0007] Step 1: Record and collect time series data.

[0008] Use various methods including sensors, observation equipment, etc. to regularly or continuously record and collect time series data Where T represents the size of the sequence in the time dimension, and D represents the size of the sequence in the variable dimension; the time series data includes multiple data items, and for each data item, its data value includes a corresponding unified timestamp and an observation value. When the timestamp is t, the corresponding observation value of the time series data is X t ={v1,v2,...,v D}.

[0009] Step 2: Build a multivariate time series data prediction model, input the collected time series data, perform feature decomposition, and obtain the prediction results.

[0010] Step 2.1: Model initialization

[0011] Based on the characteristics of the prediction task, initialize the hidden variable space dimension F, and select K groups of different pooling layers and L groups of time scales l∈{1,2,...,L} is used for feature extraction and scale decomposition; the time scale is a ratio, which indicates the degree of change in the length of the time axis before and after the time series data is scaled; based on the selected time scale, a two-dimensional convolutional backbone network is used to construct L independent sub-prediction modules.

[0012] Step 2.2: Sub-feature extraction

[0013] Use the K groups of pooling layers constructed in step 2.1 to classify the time series Perform a preliminary decomposition and obtain K groups of sub-feature sequences χ={x1,x2,...,x k},in, k∈{1,2,...,K}.

[0014] Step 2.3: Multi-scale decomposition

[0015] By adjusting the size and step size of the pooling layer in the time dimension, the sub-feature sequence obtained in step 2.2 is further decomposed in the time scale. k , decomposed into L groups of sub-scale sequences There are K×L groups in total. l∈{1,2,...,L},k∈{1,2,...,K}.

[0016] Step 2.4: Subsequence encoding

[0017] For the K×L groups of subsequences obtained in the previous steps Encode the input data from D to the hidden variable space dimension F selected in step 2.1, and obtain the K×L group of encoded multi-scale sub-feature time series.

[0018] Step 2.5: Decomposition of cycle term and trend term

[0019] For the K×L groups of time series obtained in step 2.4 Perform fast Fourier transform respectively to obtain the corresponding frequency domain representation in, represents fast Fourier transform; let f s is the center frequency of the Fourier transform result, N is the total frequency of the transform result, then the high frequency area It can be expressed as In the case of f>f s For the part, the high-frequency region in the frequency space is selected for inverse Fourier transform, and the corresponding time domain representation is obtained as the periodic term From the original sequence Subtract periodic terms Get trend items

[0020] Step 2.6: Prediction

[0021] For the periodic term, Fourier transform is used to convert it to the frequency domain and then extended to the prediction window, where the ratio of the prediction window size to the original prediction window size is equal to the scaling scale; for the trend term, the sub-prediction module constructed in step 2.1 is extended to the prediction window, and similarly, the ratio of the prediction window size to the original prediction window size is equal to the scaling scale; the prediction windows of the periodic term and the prediction term are intercepted; the prediction results at different time scales are mapped to the original prediction window using transposed convolution; finally, the prediction results obtained by adding different multi-scale sub-feature sequences element by element to obtain the final output.

[0022] Step 3: Train and test the multivariate time series data prediction model.

[0023] Compared with the existing prediction methods, the advantages of the present invention are:

[0024] The present invention deeply decomposes the time series from the aspects of sub-features, multi-scale and sequence modeling, improves the significance of key time variables at the data level, realizes the decoupling of complex mixed time series data, reduces the difficulty of capturing time variables, and improves the prediction accuracy and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0026] The following is a further description of a multivariate time series data prediction method based on interactive two-dimensional convolution proposed by the present invention in conjunction with the accompanying drawings.

[0027] like Figure 1 The figure shows the overall flow chart of the method of the present invention. For the input time series data, the maximum pooling layer, the minimum pooling layer, etc. are used to perform sub-feature decomposition; the average pooling layer is used to perform multi-scale decomposition on the obtained sub-feature sequence by adjusting its size and step size in the time dimension; the obtained multi-scale sub-feature sequence is encoded and predicted. During prediction, the periodic term and trend term are decomposed again using Fourier transform; the periodic term and trend term of the subsequence are predicted independently and then scaled back to the original prediction window to obtain the prediction result of the current subsequence, and the prediction results of each subsequence are mixed to obtain the final prediction result.

[0028] Taking the Electricity dataset as an example, we build a time series data prediction model. The dataset describes the electricity consumption of 321 households. The specific steps are as follows:

[0029] Step 1: According to the data set and prediction task, set the maximum pooling layer and the minimum pooling layer as the sub-feature decomposition module; set the average pooling layer with multi-scale scaling ratios of 1 / 1, 1 / 2 and 1 / 4 as the multi-scale decomposition module; use Fourier transform as the periodic trend decomposition module and the periodic item prediction module; set the hidden feature space dimension F to 256; set 6 ResNet18 convolutional backbone networks as sub-prediction modules to independently predict the trend items obtained by decomposing the multi-scale sub-feature sequence; set the input window length to 96 and the prediction window to 96, that is, use the data of the past 96 time steps to predict the situation of the next 96 time steps.

[0030] Step 2: For the given Input: 96×321, use the maximum pooling layer and the minimum pooling layer to process the time series respectively to obtain the maximum sub-feature Max: 96×321 and the minimum sub-feature Min: 96×321.

[0031] Step 3: Use average pooling layers with different scaling ratios to perform multi-scale decomposition on the maximum sub-feature Max and the minimum sub-feature Min, and obtain MaxL and MinL: 96×321, MaxM and MinM: 48×321, MaxS and MinS: 24×321.

[0032] Step 4: Use value coding, position coding and time coding to map each subsequence obtained in step 3, and the sequence size changes from T×321 to T×256; the value coding is to perform a one-dimensional convolution with a step size of 1 on the original data along the time axis; the position coding is to construct an increasing matrix from 0 to T to represent the data position; the time coding is to generate the corresponding mixed time embedding vector (year, month, week, day, hour, minute) according to the data timestamp.

[0033] Step 5: Use Fourier transform to further process the multi-scale sub-feature sequences to obtain their frequency domain representation. Cut the first half of the high-frequency region and perform inverse Fourier transform to obtain the time domain representation as the periodic term. Subtract the periodic term from the subsequence and use the result as the trend term.

[0034] Step 6: Based on the time scale of the subsequence, make predictions for the corresponding prediction window. The periodic term of the subsequence is extrapolated to the prediction window of the corresponding scale based on its frequency domain representation and then inverse Fourier transform is performed. The trend term is added with 0 to expand the prediction window to obtain the sequence: (192|96|48)×196, and then extrapolated to the prediction window of the corresponding scale using the sub-prediction modules set in step 1. The periodic term and trend term that belong to the same subsequence are merged, and the scale is scaled using transposed convolution to change the time scale size from the scaled time scale size to the original size: 192×196. The prediction results of each subsequence are merged.

[0035] Step 7: Use the fully connected layer to map the variable dimension size of the prediction result from F back to its original size of 321; take the data of the last 96 time steps on the time axis, i.e. the prediction window, to obtain the final prediction result. The experimental results are shown in Tables 1 and 2.

[0036] Table 1 shows the results of comparative experiments with other existing advanced models based on data sets such as Electricity and Weather. Table 2 shows the effectiveness experiment of the deep sub-feature decomposition used in the present invention.

[0037] Table 1. Comparative experimental results

[0038]

[0039] Table 2. Validity verification

[0040]

Claims

1. A multivariate time series data prediction method based on deep sub-feature decomposition, characterized in that: The following steps are involved: Step 1: Record and collect time series data; Step 2: Build a multivariate time series data prediction model, input time series data, perform feature decomposition, and obtain prediction results; Step 3: Train and test the multivariate time series data prediction model.

2. The multivariate time series data prediction method based on deep sub-feature decomposition according to claim 1 is characterized in that: The time series data described in step 1 is expressed as T represents the size of the sequence in the time dimension, and D represents the size of the sequence in the variable dimension; the time series data includes multiple data items. When the timestamp is t, the corresponding observation value of the time series data is X t ={v1,v2,...,v D }.

3. The multivariate time series data prediction method based on deep sub-feature decomposition according to claim 1 is characterized in that: The multivariate time series data prediction model is specifically implemented as follows: Step 2.1: Initialize the hidden variable space dimension F and select K groups of different pooling layers and L groups of time scales Perform feature extraction and scale decomposition; based on the time scale, use two-dimensional convolution to construct L independent sub-prediction modules; Step 2.2: Use K groups of pooling layers to pool the time series Perform a preliminary decomposition and obtain K groups of sub-feature sequences χ={x1,x2,...,x k },in, Step 2.3: Use the average pooling layer to adjust the size and step size of the pooling layer in the time dimension to perform multi-scale decomposition on the sub-feature sequence obtained in step 2.2 on the time scale. For each sub-feature sequence x k , decomposed into L groups of sub-scale sequences There are K×L groups, among which Step 2.4: For K×L groups of subsequences Encode the input data from D to the hidden variable space dimension F selected in step 2.1, and obtain the K×L group of encoded multi-scale sub-feature time series. Step 2.5: For the K×L groups of time series obtained in step 2.4 Perform fast Fourier transform respectively to obtain the corresponding frequency domain representation in, represents fast Fourier transform; let f s is the center frequency of the Fourier transform result, N is the total frequency of the transform result, then the high frequency area Expressed as In the case of f>f s For the part, the high-frequency region in the frequency space is selected for inverse Fourier transform, and the corresponding time domain representation is obtained as the periodic term from Subtract periodic terms Get trend items Step 2.6: For the periodic term, use Fourier transform to convert to the frequency domain and then extend it to the prediction window, where the ratio of the prediction window size to the original prediction window size is equal to the scaling scale; for the trend term, use the sub-prediction module constructed in step 2.1 to extend it to the prediction window, where the ratio of the prediction window size to the original prediction window size is equal to the scaling scale; intercept the prediction window of the periodic term and the prediction term; use transposed convolution to map the prediction results at different time scales to the original prediction window; finally, add the prediction results obtained by different multi-scale sub-feature sequences element by element to get the final output.

4. The multivariate time series data prediction method based on deep sub-feature decomposition according to claim 3 is characterized in that: The time scale is a ratio, which indicates the degree of change in the length of the time axis of the time series data before and after scaling.

5. The multivariate time series data prediction method based on deep sub-feature decomposition according to claim 3 is characterized in that: The specific implementation process of step 2.4 is as follows: For the K×L groups of subsequences obtained before Encode the input data by using value encoding, position encoding, and timestamp encoding, respectively, to map the variable dimension size D of the input data to the hidden variable space dimension size F selected in step 2.1; among them, value encoding uses one-dimensional convolution for convolution; position encoding generates a position sequence of [0,1,...,T); time encoding generates a mixed time embedding vector based on the timestamp; add the above encoding results element by element to obtain the K×L group of encoded multi-scale sub-feature time series

Images