Multivariable time series data prediction method based on sequence features and time convolution

Abstract

The invention discloses a multivariable time series data prediction method based on sequence features and time convolution. The multivariable time series data prediction method is specifically implemented according to the following steps: step 1, data preprocessing; step 2, core representation generation; step 3, multi-scale representation generation; step 4, fusing core representation and multi-scale representation; and step 5, outputting a prediction result. According to the method, channels are established for time sequence data according to features, aggregation of the features of all the channels is achieved through a perceptron and random pooling operation, then dependency of multi-scale time step lengths is extracted through a set of convolution kernels which are gradually increased, and the multi-scale time step lengths are obtained. The prediction accuracy of the multivariable time sequence prediction method is improved by using the correlation between the sequence features and the dependency between the time steps, and the calculation complexity of the prediction method is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction methods, and specifically relates to a multivariate time series data prediction method based on sequence features and time convolution. Background Art

[0002] Time series data is usually the result of observing a potential process at a given sampling rate within an equally spaced time period, so there is a close connection between data values ​​at different times. Time series forecasting uses historical time series data to predict data for a period of time in the future, which can provide data support for formulating corresponding strategies. It has wide applications in economics, finance, meteorology and other fields. According to the different variable dimensions, it can be divided into univariate time series forecasting and multivariate time series forecasting. Univariate time series forecasting assumes that the variable value is only related to its own historical value; while multivariate time series forecasting assumes that each variable is not only affected by its own historical value, but also closely related to other variables. Deep learning methods usually establish channels based on the variable dimensions in the sequence to capture the correlation between different features.

[0003] Multivariate time series prediction has always been a research hotspot at home and abroad. In recent years, models built by using attention mechanisms to capture channel dependencies have played an important role in multivariate time series prediction tasks. For example, Vaswani et al. proposed the Transformer model, in which the self-attention mechanism used can capture the dependencies of long sequence features, and is therefore applied to the field of multivariate prediction. Zhou et al. proposed a frequency-enhanced Transformer model with a hybrid expert mechanism of periodic term trend term decomposition, which can be used to capture the global view of multivariate time series prediction. Zhou et al. proposed the Informer model, which improved the calculation method of the self-attention mechanism, improved the computational efficiency, and effectively extracted the most important features in the sequence. Guo et al. proposed the Conformer model, which combined the self-attention mechanism of the Transformer with the local receptive field advantage of CNN to improve the efficiency of the model on long sequences and maintain high performance. Although the time series prediction model based on the attention mechanism has achieved good prediction results in practical applications, it has the problems of complex calculation and low efficiency. Therefore, it is crucial to effectively integrate the independence of each channel and utilize the correlation between channels in a simpler and more efficient way to build a better time series prediction model. Summary of the invention

[0004] The purpose of the present invention is to provide a multivariate time series data prediction method based on sequence features and time convolution, which solves the problem of complex calculation of time series prediction models in the prior art.

[0005] The technical solution adopted by the present invention is a multivariate time series data prediction method based on sequence features and time convolution, which is specifically implemented according to the following steps:

[0006] Step 1: Data preprocessing

[0007] Normalize the input time series data and embed the encoding information;

[0008] Step 2: Core representation generation

[0009] The preprocessed sequence data is subjected to multi-layer perceptron and random pooling operations to aggregate sequence features and generate the core representation of the sequence;

[0010] Step 3: Multi-scale representation generation

[0011] Through a set of gradually increasing convolution kernels, the sequence data is subjected to the extraction of multi-scale sequence dependencies and the establishment of a multi-scale time representation sequence.

[0012] Step 4: Fusion of core representation and multi-scale representation

[0013] Interact the core representation sequence with the multi-scale identification sequence and multiply the two element by element;

[0014] Step 5: Output prediction results

[0015] Pass the sequence obtained in step 4 into the fully connected linear layer to get the final prediction output

[0016] The present invention is also characterized in that

[0017] Step 1 is as follows: data normalization. Given a time series data X, whose structure is L×C, L represents the length of the time series, and C represents the number of channels, the data is mapped to the specified range through linear changes. The specific calculation method is:

[0018] normal_value=(value-min) / (max-min) (1)

[0019] For each channel, find the maximum and minimum values ​​of the feature data and use the above method to map the data to between [0,1].

[0020] The embedded coding in step 1 aims to map the time series data from the original feature space to a lower-dimensional latent feature space. Specifically, linear embedding is used: each feature channel is mapped to a latent feature space of fixed dimension using linear transformation, and multiple channel sequence representations {S1, S2, S3…S C}, where S C ∈RL Represents the sequence data corresponding to the Cth channel.

[0021] Step 2 is as follows: multiple channel sequences {S1, S2, S3…S C}, where S C ∈R L Represents the sequence corresponding to the Cth channel, and the sequence S is transformed into i ∈R L Map to Then, the C sequences are aggregated through random pooling operation to obtain the core representation

[0022] O=Stoch_pool(MLP(S1,S2,S3...S C )) (2)

[0023] Among them, MLP is a mapping layer with GELU activation function, and Stoch_pool is a random pooling operation.

[0024] The random pooling calculation method is: perform softmax normalization on each element in the pooling window to obtain the probability p of each element being selected (j) :

[0025]

[0026] Among them, a (j) is the jth element in the pooling window, n is the number of elements in the pooling window, and random sampling is performed according to the probability of each element, and an element is selected as the output of the pooling window. During the training process, random sampling is performed for each forward propagation;

[0027] Through the above method, we can get a global core representation sequence Each element in the sequence is the result of aggregating the most important features in the C channels at the corresponding time step.

[0028] Step 3 is as follows: obtain the correlation between time steps through a set of dilated convolution layers with gradually larger convolution kernels {v1, v2, v3} to extract multi-scale temporal dependencies:

[0029]

[0030] Among them, X i represents the sequence obtained by using the i-th convolution kernel, X i[...,-(L-V3+1):] means that only the values ​​of (L-V3+1) time steps are retained in the time dimension. Finally, the three multi-scale feature values ​​are concatenated on the channel and restored to the same time dimension as the input through the fully connected layer to obtain the multi-scale time representation sequence Y:

[0031]

[0032] Y∈R L×C =MLP(X L ) (9)

[0033] Among them, Concat represents concatenation operation, and MLP represents multi-layer perceptron.

[0034] In step 4, steps 2 and 3 are repeated N times to obtain the output sequence representation S of the N layer. No With S Ny , multiply the two element by element, and restore the dimension through the linear layer to realize the interaction between the time step and the characteristics of each channel, specifically:

[0035]

[0036] Among them, * represents element-by-element multiplication, and Linear represents a linear transformation, which usually consists of a weight matrix W and a bias term b.

[0037] In step 5, for each channel and each time step, the final sequence representation is multiplied by the weight matrix W and the bias vector b is added to obtain the prediction result

[0038]

[0039] in, is the value of the prediction result at the Cth channel and the Lth time step, W (C:) is the Cth row of the weight matrix W, b (c) is the cth element of the bias vector b, combining the prediction results of all channels and all time steps to obtain the final prediction output

[0040] The beneficial effects of the present invention are:

[0041] The present invention is a multivariate time series data prediction method based on sequence features and time convolution. Channels are established for time series data according to features, and the features of each channel are aggregated using perceptron and random pooling operations. Then, a set of gradually increasing convolution kernels is used to extract the dependency of multi-scale time steps. The correlation between sequence features and the dependency between time steps are used to improve the prediction accuracy of the multivariate time series prediction method and reduce the computational complexity of the prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the multivariate time series data prediction method based on sequence features and time convolution of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] The present invention is based on a multivariate time series data prediction method based on sequence features and time convolution, such as Figure 1 As shown, the specific implementation steps are as follows:

[0046] Step 1: Data preprocessing

[0047] Normalize the input time series data and embed the encoding information;

[0048] Step 2: Core representation generation

[0049] The sequence data preprocessed in step 1 is aggregated through a multi-layer perceptron and random pooling operation to generate the core representation of the sequence;

[0050] Step 3: Multi-scale representation generation

[0051] Through a set of gradually increasing convolution kernels, the sequence data is subjected to the extraction of multi-scale sequence dependencies and the establishment of a multi-scale time representation sequence.

[0052] Step 4: Fusion of core representation and multi-scale representation

[0053] The core representation sequence obtained in step 2 and the multi-scale identification sequence obtained in step 3 are interacted, and the two are multiplied element by element. The sequence dimension is restored through the multi-layer perceptron to realize the interaction between the global feature information and the features of each channel, extract the dependency relationship between channels, and improve the prediction accuracy of the model;

[0054] Step 5: Output prediction results

[0055] Pass the sequence obtained in step 4 into the fully connected linear layer to get the final prediction output

[0056] Example 2

[0057] The present invention is a multivariate time series data prediction method based on sequence features and time convolution, wherein step 1 is specifically as follows:

[0058] Data normalization: Given a time series data X, whose structure is L×C, L represents the length of the time series, and C represents the number of channels, the data is mapped to a specified range through linear changes. The common method is to map the data to between [0,1]. The specific calculation method is:

[0059] normal_value=(value-min) / (max-min) (1)

[0060] For each channel, find the maximum and minimum values ​​of the feature data and use the above method to map the data to between [0,1].

[0061] The purpose of embedding coding is to map time series data from the original feature space to a lower-dimensional latent feature space so that the model can more easily extract features and make predictions. Specifically, linear embedding is used: each feature channel is mapped to a latent feature space of fixed dimension using linear transformation. After normalization and embedding coding operations, multiple channel sequence representations {S1, S2, S3…S C}, where S C ∈R L Represents the sequence data corresponding to the Cth channel.

[0062] Step 2 is as follows:

[0063] After the preprocessing in step 1, multiple channel sequences {S1, S2, S3…S C}, where S C ∈R L Represents the sequence corresponding to the Cth channel, and the sequence S is transformed into i ∈R L Map to Then, the C sequences are aggregated through random pooling operation to obtain the core representation

[0064] O=Stoch_pool(MLP(S1,S2,S3...S C )) (2)

[0065] Among them, MLP is a mapping layer with GELU activation function, and Stoch_pool is a random pooling operation;

[0066] The random pooling calculation method is: perform softmax normalization on each element in the pooling window to obtain the probability p of each element being selected (j) :

[0067]

[0068] Among them, a (j)is the jth element in the pooling window, n is the number of elements in the pooling window, and random sampling is performed according to the probability of each element. An element is selected as the output of the pooling window. During the training process, random sampling is performed for each forward propagation.

[0069] The most important information in each channel sequence is extracted through random pooling operation and combined into a new sequence Each element in the sequence O is the result of the aggregation of features in the C channels at the corresponding time step. Through the core representation, the model can capture the interrelationships and dependencies between different channels, so as to better understand the overall structure and trend of the data; compress the information of multiple channels into a lower dimension, thereby reducing the number of parameters that the model needs to learn and reducing the computational cost; it has stronger robustness to the noise and outliers of a single channel. Even if an abnormality occurs in a channel, the core representation can still remain relatively stable, thereby improving the prediction accuracy of the model.

[0070] Example 3

[0071] The present invention is a multivariate time series data prediction method based on sequence features and time convolution, wherein step 3 is specifically: obtaining the correlation between time steps through a set of dilated convolution layers with gradually increasing convolution kernels {v1, v2, v3}, extracting multi-scale time dependencies, and establishing a multi-scale time representation sequence:

[0072]

[0073] Among them, X i represents the sequence obtained by using the i-th convolution kernel, X i [...,-(L-V3+1):] means that only the values ​​of (L-V3+1) time steps are retained in the time dimension. Finally, the three multi-scale feature values ​​are concatenated on the channel and restored to the same time dimension as the input through the fully connected layer to obtain the multi-scale time representation sequence Y:

[0074]

[0075] Y∈R L×C =MLP(X L ) (9)

[0076] Among them, Concat represents concatenation operation, and MLP represents multi-layer perceptron.

[0077] In step 4, steps 2 and 3 are repeated N times to obtain the output sequence representation S of the N layer. No With S Ny, multiply the two element by element, and restore the dimension through the linear layer to realize the interaction between the time step and the characteristics of each channel, and realize the fusion of multi-scale time representation and each channel representation, specifically:

[0078]

[0079] Among them, * represents element-by-element multiplication, and Linear represents a linear transformation, which usually consists of a weight matrix W and a bias term b.

[0080] In step 5, for each channel and each time step, the final sequence representation is multiplied by the weight matrix W and the bias vector b is added to obtain the prediction result

[0081]

[0082] in, is the value of the prediction result at the Cth channel and the Lth time step, W (C:) is the Cth row of the weight matrix W, b (c) is the cth element of the bias vector b, combining the prediction results of all channels and all time steps to obtain the final prediction output

[0083] Compared with complex neural network layers, linear layers have lower computational complexity because they only involve simple matrix multiplication operations, which can significantly reduce the computational burden of the model and speed up training and reasoning. Moreover, because they do not have complex nonlinear characteristics, they can reduce the possibility of model overfitting, which helps the model maintain its generalization ability during training and avoid poor performance on test data.

[0084] The multivariate time series data prediction method based on sequence features and time convolution in the present invention is different from the traditional method of managing channel interactions through a distributed structure (such as the attention mechanism). It adopts a centralized strategy to extract the correlation of each channel in the sequence. Its core is to aggregate all sequence information to form a global core representation; a set of gradually increasing convolution kernels are used to extract the dependencies between time steps to form a multi-scale sequence representation, which is then multiplied element by element with the global core representation to achieve the fusion of sequence features and steps, so as to make full use of the information of time series data and improve the accuracy of multivariate time series prediction.

[0085] The present invention can make full use of the correlation between sequence data features and the dependence between time steps, provide a solution for multivariate time series prediction, and improve the prediction accuracy of the model by aggregating the feature information of the sequence and using convolution kernels of different sizes to generate multi-scale sequence time dimensions. At the same time, it can provide solutions and method support for sequence tasks such as text translation, speech recognition, and time series classification.

Claims

1. A multivariate time series data prediction method based on sequence features and time convolution, characterized in that: Follow the steps below to implement it: Step 1: Data preprocessing Normalize the input time series data and embed the encoding information; Step 2: Core representation generation The preprocessed sequence data is subjected to multi-layer perceptron and random pooling operations to aggregate sequence features and generate the core representation of the sequence; Step 3: Multi-scale representation generation Through a set of gradually increasing convolution kernels, the sequence data is subjected to the extraction of multi-scale sequence dependencies and the establishment of a multi-scale time representation sequence. Step 4: Fusion of core representation and multi-scale representation Interact the core representation sequence with the multi-scale identification sequence and multiply the two element by element; Step 5: Output prediction results Pass the sequence obtained in step 4 into the fully connected linear layer to get the final prediction output 2. The multivariate time series data prediction method based on sequence features and time convolution according to claim 1 is characterized in that: The step 1 is specifically as follows: data normalization. Given a time series data X, whose structure is L×C, L represents the length of the time series, and C represents the number of channels, the data is mapped to a specified range through linear changes. The specific calculation method is: normal_value=(value-min) / (max-min) (1) For each channel, find the maximum and minimum values ​​of the feature data and use the above method to map the data to between [0,1].

3. The multivariate time series data prediction method based on sequence features and time convolution according to claim 1 is characterized in that: The embedded coding in step 1 aims to map the time series data from the original feature space to a potential feature space of lower dimension, specifically by using linear embedding: each feature channel is mapped to a potential feature space of fixed dimension using linear transformation, and multiple channel sequence representations {S1, S2, S3…S C }, where S C ∈R L Represents the sequence data corresponding to the Cth channel.

4. The multivariate time series data prediction method based on sequence features and time convolution according to claim 1 is characterized in that: The step 2 specifically includes: performing preprocessing on the multiple channel sequences {S1, S2, S3...S C }, where S C ∈R L Represents the sequence corresponding to the Cth channel, and the sequence S is transformed by a multi-layer perceptron i ∈R L Map to Then, the C sequences are aggregated through random pooling operation to obtain the core representation O=Stoch_pool(MLP(S1,S2,S3…S C )) (2) Among them, MLP is a mapping layer with GELU activation function, and Stoch_pool is a random pooling operation.

5. The multivariate time series data prediction method based on sequence features and time convolution according to claim 4 is characterized in that: The random pooling calculation method is: perform softmax normalization on each element in the pooling window to obtain the probability p of each element being selected (j) : Among them, a (j) is the jth element in the pooling window, n is the number of elements in the pooling window, and random sampling is performed according to the probability of each element, and an element is selected as the output of the pooling window. During the training process, random sampling is performed for each forward propagation; Through the above method, we can get a global core representation sequence Each element in the sequence is the result of aggregating the most important features in the C channels at the corresponding time step.

6. The multivariate time series data prediction method based on sequence features and time convolution according to claim 1 is characterized in that: The step 3 is specifically as follows: obtaining the correlation between time steps through a set of dilated convolution layers with gradually increasing convolution kernels {v1, v2, v3}, and extracting multi-scale time dependencies: Among them, X i represents the sequence obtained using the i-th convolution kernel, X i [...,-(L-V3+1):] means that only the values ​​of (L-V3+1) time steps are retained in the time dimension. Finally, the three multi-scale feature values ​​are concatenated on the channel and restored to the same time dimension as the input through the fully connected layer to obtain the multi-scale time representation sequence Y: Y∈R L×C =MLP(X L ) (9) Among them, Concat represents concatenation operation, and MLP represents multi-layer perceptron.

7. The multivariate time series data prediction method based on sequence features and time convolution according to claim 1 is characterized in that: In step 4, steps 2 and 3 are repeated N times to obtain an output sequence representation S of the N layers. No With S Ny , multiply the two element by element, and restore the dimension through the linear layer to realize the interaction between the time step and the characteristics of each channel, specifically: Among them, * represents element-by-element multiplication, and Linear represents a linear transformation, which usually consists of a weight matrix W and a bias term b.

8. The multivariate time series data prediction method based on sequence features and time convolution according to claim 1 is characterized in that: In step 5, for each channel and each time step, the final sequence representation is multiplied by the weight matrix W and the bias vector b is added to obtain the prediction result in, is the value of the prediction result at the Cth channel and the Lth time step, W (C:) is the Cth row of the weight matrix W, b (c) is the cth element of the bias vector b, combining the prediction results of all channels and all time steps to obtain the final prediction output

Images