Multivariate time sequence prediction method and system based on sequence decomposition

Abstract

The invention relates to a multivariate time sequence prediction method and system based on sequence decomposition, and the method comprises the steps: carrying out the fusion of local time features and global sequence features extracted from multivariate time sequence data, and obtaining a fusion feature sequence; initializing a node embedding matrix, and performing inner product calculation on the node embedding matrix to obtain a static graph matrix; decomposing the fusion feature sequence by using the static graph matrix to obtain a first long-term component and a first short-term component; performing graph convolution on the first long-term component by utilizing the static graph matrix and performing graph convolution on the first short-term component by utilizing the dynamic graph matrix to obtain long-term and short-term convolution quantities; wherein the dynamic graph matrix is obtained by utilizing a static graph matrix and introducing differential data; obtaining a total component according to the long-term and short-term convolution quantities, and fusing and splicing the total component with the global sequence features to obtain a spliced sequence; and fusing the fused feature sequence and the spliced sequence to obtain a prediction result. According to the method, the uniqueness of dependency relationship decomposition among the sequences is ensured, and the uncertainty of performance expression is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of multivariate time series prediction, and in particular to a multivariate time series prediction method and system based on sequence decomposition. Background Art

[0002] Research on multivariate time series forecasting shows that using graph structures to build models to capture and model complex dependencies between multivariate time series has shown excellent performance. This method can not only effectively analyze the dynamic interactions between variables, but also significantly improve the accuracy and reliability of forecasts.

[0003] In traditional graph neural network-based methods, most of them use predefined graph structures or learn fixed graph structures to describe the dependencies between sequences. However, in the face of the dynamic changes between sequence dependencies over time, fixed graph structure methods cannot effectively capture and process this dynamic nature. To solve this problem, the prior art proposes a strategy that combines static graph structures and dynamic graph structures, which can effectively describe the dependencies between sequences that change over time. Among them, the static graph structure focuses on the dependencies of long-term trends between sequences, while the dynamic graph structure describes the dependencies of short-term fluctuations between sequences. However, this method directly learns static graph structures and dynamic graph structures from the original sequence, and it may happen that the same set of data learns multiple combinations of static graph structures and dynamic graph structures, resulting in non-unique decomposition of dependencies between sequences, thereby exacerbating the uncertainty of performance. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art that a plurality of static graph structure and dynamic graph structure combinations are learned from the same set of data, resulting in non-unique decomposition of dependencies between sequences and uncertainty in performance. A multivariate time series prediction method and system based on sequence decomposition are provided, which ensure the uniqueness of the decomposition of dependencies between sequences through a static-dynamic graph learning network, thereby reducing the uncertainty of performance.

[0005] In a first aspect, in order to solve the above technical problems, the present invention provides a multivariate time series prediction method based on sequence decomposition, comprising:

[0006] Extracting local time features and global sequence features of multivariate time series data, fusing the local time features and the global sequence features to obtain a fused feature sequence; initializing a node embedding matrix, and obtaining a static graph matrix by performing inner product calculation on the node embedding matrix;

[0007] Decomposing the fused feature sequence by using the static image matrix into a first long-term component and a first short-term component; wherein the first short-term component is the difference between the fused feature sequence and the first long-term component;

[0008] The first long-term component is convolved by using the static graph matrix to obtain a long-term convolution component; the first short-term component is convolved by using the dynamic graph matrix to obtain a short-term convolution component; wherein the method for obtaining the dynamic graph matrix is: decomposing the multivariate time series data by using the static graph matrix to decompose a second long-term component and a second short-term component; the second short-term component is the difference between the multivariate time series data and the second long-term component; the dynamic graph matrix is ​​obtained by using the second short-term component and the static graph matrix and introducing differential data;

[0009] Obtaining a total component according to the long-term convolution component and the short-term convolution component;

[0010] Performing fusion and splicing processing on the total component and the global sequence feature to obtain a spliced ​​sequence;

[0011] The fusion feature sequence and the splicing sequence are fused to obtain a prediction result.

[0012] In one embodiment of the present invention, by performing inner product calculation on the node embedding matrix, the calculation formula for obtaining the static graph matrix is:

[0013]

[0014] in, is a non-normalized static graph structure, A l is a static graph matrix, E is a randomly initialized node embedding matrix, ε is a very small number, and I N is the identity matrix, for The diagonal matrix of .

[0015] In one embodiment of the present invention, the calculation formulas of the second long-term component and the second short-term component are:

[0016] X l =(A l ) n X;

[0017] X s =XX l ;

[0018] Among them, X l is the second long-term component, X s is the second short-term component, X is the multivariate time series data, Al is the static graph matrix, and n is the number of smoothing iterations.

[0019] In one embodiment of the present invention, the calculation formula of the dynamic graph matrix is:

[0020]

[0021]

[0022] in, is the pth segment of the second short-term component, W s ,W d ,W l is the learnable weight, h p is the node representation of the pth segment in the second short-term component, is the non-normalized dynamic graph matrix corresponding to the pth fragment, for The diagonal matrix of is the second short-term component X s The dynamic graph matrix corresponding to the pth segment in , ΔX p is the differential data corresponding to the pth fragment, ε is a very small number, I N is the identity matrix, E is a randomly initialized node embedding matrix, T is the total duration, L is the length of each segment, and ⊙ is the Hadamard product operator.

[0023] In one embodiment of the present invention, the mathematical expression of the differential data is:

[0024]

[0025] Among them, t is the time, T is the total duration, i∈{1,2,…,N}, x i (t) is the data of the i-th sequence at time t, x i (t-1) is the data of the i-th sequence at time t-1.

[0026] In one embodiment of the present invention, the local time feature and the global sequence feature are fused to obtain a fused feature sequence by splicing the local time feature and the global sequence feature in the channel dimension, and the weight of the spliced ​​vector is calculated by two fully connected layers, and the weight calculation formula is:

[0027]

[0028] Among them, r is the weight, is the local time feature, H mlpis the global sequence feature; based on r, the local time feature and the global sequence feature are fused, and the fusion formula is:

[0029]

[0030] Wherein, H is the fusion feature sequence, and ⊙ is the Hadamard product operator.

[0031] In one embodiment of the present invention, a method for obtaining a total component according to the long-term convolution component and the short-term convolution component is:

[0032]

[0033] in, is the long-term convolution component of the lth layer, is the short-term convolution component of the p-th segment in the l-th layer, is the total component, A l is the static graph matrix, is the second short-term component X s The dynamic graph matrix corresponding to the pth fragment in , W (l) is a learnable parameter, It is formed by concatenating the fragments of all the short-term convolution components in the lth layer.

[0034] In one embodiment of the present invention, obtaining the prediction result includes calculating the prediction loss value, and the calculation formula is:

[0035]

[0036] Among them, τ is the step size of the future time step that we want to predict, N is the total dimension of the multivariate time series, and T is the total duration. is the actual value, is the predicted value.

[0037] In a second aspect, in order to solve the above technical problems, the present invention provides a multivariate time series prediction system based on sequence decomposition, comprising:

[0038] A graph structure learning module is used to initialize a node embedding matrix, and obtain a static graph matrix by performing inner product calculation on the node embedding matrix; decompose the multivariate time series data using the static graph matrix to decompose a second long-term component and a second short-term component; the second short-term component is the difference between the multivariate time series data and the second long-term component; obtain a dynamic graph matrix using the second short-term component and the static graph matrix and introducing differential data;

[0039] A first fusion multi-layer perceptron module is used to fuse local time features and global sequence features to obtain a fusion feature sequence; wherein the local time features and the global sequence features are respectively extracted from the multivariate time series data by a time convolution module and a feature extraction module;

[0040] A time series decomposition graph convolution module is used to decompose the fused feature sequence using the static graph matrix into a first long-term component and a first short-term component; wherein the first short-term component is the difference between the fused feature sequence and the first long-term component; perform graph convolution on the first long-term component using the static graph matrix to obtain a long-term convolution component; perform graph convolution on the first short-term component using a dynamic graph matrix to obtain a short-term convolution component; and obtain a total component based on the long-term convolution component and the short-term convolution component;

[0041] A second fusion multi-layer perceptron module is used to fuse and splice the total component with the global sequence feature to obtain a spliced ​​sequence;

[0042] The output module is used to fuse the fusion feature sequence and the splicing sequence to obtain a prediction result.

[0043] In one embodiment of the present invention, the temporal convolution module, the first fusion multi-layer perceptron module, the time series decomposition graph convolution module and the second fusion multi-layer perceptron module constitute a space-time layer, the system includes K space-time layers, when K>1, the output of the kth space-time layer is used as the input of the k+1th space-time layer, the output module receives and fuses the fused feature sequences of the K space-time layers and the output of the Kth space-time layer to obtain a prediction result; wherein K and k are integers and K≥1, 1≤k≤K.

[0044] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0045] The method and system for predicting multivariate time series based on sequence decomposition described in the present invention adopts a static-dynamic graph fusion learning network to achieve efficient prediction of multivariate time series. By calculating the static graph matrix (static graph matrix can also be called static graph structure), the original multivariate time series is decomposed using this static graph matrix to obtain the second short-term component, and combining this component with the introduced differential data to further derive the dynamic graph matrix (dynamic graph matrix can also be called dynamic graph structure). This process dynamically captures the changing trend in the time series. The method described in the present invention is significantly different from the traditional method. It avoids directly learning static and dynamic graph structures from the original multivariate time series, thereby eliminating the uncertainty of multiple combinations of static graphs and dynamic graph structures caused by different data interpretation angles. By clearly defining the dependency decomposition path between sequences, the uniqueness and accuracy of this process are ensured, thereby greatly improving the stability of the prediction model and the certainty of performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0047] Figure 1 It is a flowchart of a multivariate time series prediction method based on sequence decomposition in a preferred embodiment of the present invention;

[0048] Figure 2 A flowchart of obtaining a static image matrix in a preferred embodiment of the present invention;

[0049] Figure 3 A flowchart of obtaining a dynamic graph matrix in a preferred embodiment of the present invention;

[0050] Figure 4 It is a structural diagram of the time series decomposition graph convolution in the preferred embodiment of the present invention;

[0051] Figure 5 It is a structural diagram of a multivariate time series prediction system based on sequence decomposition in a preferred embodiment of the present invention;

[0052] Figure 6 It is a structural diagram of the feature extraction module in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0054] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Embodiment 1

[0056] Reference Figure 1 As shown, a multivariate time series prediction method based on sequence decomposition provided by an embodiment of the present invention includes:

[0057] Extract local time features and global sequence features of multivariate time series data, fuse local time features and global sequence features to obtain a fused feature sequence; initialize the node embedding matrix, and obtain a static graph matrix by performing inner product calculation on the node embedding matrix;

[0058] Decomposing the fused feature sequence into a first long-term component and a first short-term component by using a static graph matrix; wherein the first short-term component is the difference between the fused feature sequence and the first long-term component;

[0059] The first long-term component is convolved with a static graph matrix to obtain a long-term convolution component; the first short-term component is convolved with a dynamic graph matrix to obtain a short-term convolution component; wherein the dynamic graph matrix is ​​obtained by: decomposing the multivariate time series data with a static graph matrix to decompose a second long-term component and a second short-term component; the second short-term component is the difference between the multivariate time series data and the second long-term component; the dynamic graph matrix is ​​obtained by using the second short-term component and the static graph matrix and introducing differential data;

[0060] According to the long-term volume component and the short-term volume component, the total component is obtained;

[0061] The total component and the global sequence features are fused and spliced ​​to obtain a spliced ​​sequence;

[0062] The fusion feature sequence and the splicing sequence are fused to obtain the prediction result.

[0063] The embodiment of the present invention provides a multivariate time series prediction method based on sequence decomposition, which uses a static-dynamic graph learning network to achieve multivariate time series prediction. By calculating the static graph matrix (the static graph matrix can also be called the static graph structure), the multivariate time series data is decomposed using the static graph matrix to obtain the second short-term component, and then the dynamic graph matrix (the dynamic graph matrix can also be called the dynamic graph structure) is obtained based on the second short-term component and the introduced differential data. This method avoids learning the static graph structure and the dynamic graph structure directly from the original multivariate time series, and effectively eliminates the uncertainty of learning multiple static and dynamic graph structure combinations for the same data due to direct processing. It ensures the uniqueness of the decomposition of the dependency relationship between sequences, thereby increasing the certainty of performance.

[0064] In order to explain the embodiments of the present invention in detail, the definitions and terms involved in the embodiments of the present invention are explained. The definitions and terms involved in the embodiments of the present invention are applicable to the following explanations.

[0065] Model multivariate time series (also called multivariate time series) data as a graph to capture the relationship between different variables. Represented as a graph, where is the node set, ε is the edge set, for The adjacency matrix represents the connections between nodes. is an N-dimensional real number space.

[0066] Definition 1 (node) Each node Represents a variable in a multivariate time series. Node Set The set corresponding to all variables, that is Where N is the total number of variables.

[0067] Definition 2: The edge set ε captures the dependencies and interactions between these variables. When there is a relationship between variables i and j, the edge (v i ,v j )∈ε exists. The strength of these relationships is encoded by the adjacency matrix A, where A ij represents the weight of the edge between node i and node j. In particular, A ij >0 means there is a connection, while A ij =0 means no direct connection.

[0068] Definition 3 (Node Embedding) Node embedding is expressed as Where d<<N is the embedding dimension. The embedding vector of the i-th node is represented as

[0069] set up represents the value of N variables at time step t, where Represents the value of the i-th variable at time step t. Given the historical observations of the past T time steps The goal of multivariate time series forecasting is to predict the value of a variable at a future time based on historical observations. In a single-step forecast, the forecast target is the value of the next τ time step. In multi-step prediction, the prediction target is the value of the next τ time steps. Right now

[0070]

[0071] Where f1 and f2 are the mapping functions for single-step prediction and multi-step prediction tasks respectively.

[0072] The complex inter-sequence dependencies in multivariate time series data can be fully captured by integrating static and dynamic graph structures. Static graph structures focus on revealing long-term trends and stable patterns in data, while dynamic graph structures keenly capture the ever-changing short-term fluctuations and interactive patterns. The two complement each other.

[0073] Specifically, the static graph structure can be represented by the static graph matrix Representation, constructed through learnable node embedding. Figure 2 , first randomly initialize a node embedding matrix The i-th row of E represents the node embedding corresponding to the i-th variable. The inner product of the node embedding matrix is ​​calculated. Specifically, the edges between nodes are calculated by the inner product between node embeddings to obtain the static graph matrix. The calculation formula is:

[0074]

[0075]

[0076] in, is a non-normalized static graph structure, A l is the static graph matrix, E is a randomly initialized node embedding matrix, ε is a very small number to prevent 0 from being used as a divisor during normalization, and I N is the identity matrix, for The angle matrix of . Formula (3) is Perform asymmetric normalization.

[0077] Specifically, the time series decomposition method based on graph smoothing enables the method described in the embodiment of the present invention to use the learned static graph matrix to decompose the original time series into a second long-term component and a second short-term component. The second long-term component is inverted from the original sequence through the static graph structure, so the second long-term component contains a long-term, stable pattern. The remaining part after the second long-term component is stripped from the original sequence is the second short-term component. The second short-term component contains a short-term, dynamic pattern, which is the basis for determining the dynamic graph structure later. Therefore, based on the time series decomposition method based on graph smoothing, the calculation formula for the second long-term component and the second short-term component is:

[0078] X l =(A l ) n X; (4)

[0079] X s =XX l ; (5)

[0080] Among them, X l is the second long-term component, X s is the second short-term component, X is the multivariate time series data, and n is the number of smoothing iterations. Formula (4) shows that the second long-term component X l is to transform the static graph matrix A l Multiply the original time series n times. This operation can be interpreted as a smoothing operation between series across node features in a graph neural network (GNN). The second short-term component X s is the original sequence X and the long-term component X l The difference implies the dependency of short-term fluctuations between sequences and can be used to learn the dependency of short-term fluctuations between sequences, that is, the dynamic graph structure.

[0081] Specifically, short-term patterns in multivariate time series are mainly driven by dynamic and sudden events, but are also constrained by static graph structures that contain long-term patterns. Therefore, after decomposing the original series X, the second short-term component X s Combined with the static graph matrix A l Infer the dynamic graph matrix A s , where the dynamic graph structure can be called a dynamic graph matrix. Differential data is introduced to control the influence of the static graph structure on the dynamic graph structure. Differential data represents the difference between the value at the current time and the value at the previous time, and its mathematical expression is:

[0082]

[0083] Among them, t is the time, T is the total duration, i∈{1,2,…,N}, x i (t) is the data of the i-th sequence at time t, x i(t-1) is the data of the i-th sequence at time t-1.

[0084] The difference data reflects the severity of the sequence change. When the sequence changes drastically, it indicates that the short-term fluctuation is obvious. In this case, the dynamic graph structure is dominated by the second short-term component. On the contrary, when the sequence changes slowly, the second short-term component X s The effect of is no longer obvious, and the dynamic graph structure tends to be more closely consistent with the static graph structure, which is mainly affected by the node embedding matrix E. Compared with the second long-term component, the second short-term component of the sequence shows more complex patterns over a longer time span. Therefore, it becomes more challenging to capture the dependencies between short-term components. However, in a shorter time span, the pattern of the second short-term component is simpler, resulting in a more obvious dependency between them. Therefore, the second short-term component X s Divide into non-overlapping segments along the time dimension to compute the dynamic graph structure. Figure 3 , assuming that the length of each segment is L, X s Can be divided into The pth fragment is represented by If T is not divisible by L, zeros are padded along the time dimension to make it divisible by L. Similarly, the difference data ΔX is segmented to obtain The pth fragment is represented by The calculation formula of the dynamic graph matrix is:

[0085]

[0086]

[0087]

[0088] in, is the pth segment of the second short-term component, W s ,W d ,W l is the learnable weight, h p is the node representation of the pth segment in the second short-term component, is the non-normalized dynamic graph matrix corresponding to the pth fragment, for The diagonal matrix of is the second short-term component X s The dynamic graph matrix corresponding to the pth segment in , ΔX p is the differential data corresponding to the pth segment, E is a randomly initialized node embedding matrix, T is the total duration, L is the length of each segment, and ⊙ is the Hadamard product operator. Formula (7) shows the use of differential data to control the node embedding matrix E to the second short-term component Xs information flow.

[0089] Specifically, in the process of obtaining local temporal features, a set of standard dilated one-dimensional CNNs are used to extract local temporal features from multivariate time series data. The dilated convolutional network achieves an exponential receptive field by increasing the layer depth, thereby effectively processing long sequences. Formally, given an input sequence and a kernel size of 1D convolution filter The definition of dilated convolution is:

[0090]

[0091] Among them, q is the expansion factor.

[0092] Furthermore, in order to capture multiple temporal patterns in time series data, multiple filters of different sizes, i.e., DIL layers, can be used. Specifically, four filters of sizes 1×2, 1×3, 1×6, and 1×7 are preferred. The output calculation formula of the DIL layer is:

[0093] x i ′=DIL(x i )= concat(x i *f 1×2 ,x i *f 1×3 ,x i *f 1×6 ,x i *f 1×7 ) (11)

[0094] The outputs of these filters are truncated in the time dimension to the same length as the largest filter and concatenated along the channel dimension. Subsequently, a gating mechanism is applied to control the flow of information to the next processing module. i First, it passes through two DIL layers, and then two different activation functions. In this example, the two different activation functions are preferably tanh function and sigmoid function, where the tanh function acts as a filter and the sigmoid function is used as a gate to control the amount of information that the filter can pass to the next processing module. Finally, element-by-element multiplication is performed, and the calculation formula is:

[0095]

[0096] in, It is a local time feature.

[0097] Specifically, three learnable embedding matrices are defined, namely: and Among them, S encodes the spatial information of each sequence, and T td and T tw Different timestamp information is encoded. d Indicates the number of samples per day, N w =7 represents the seven days of the week, and D is the embedding dimension. These embedding matrices can be adjusted according to specific data sampling intervals and periodicity.

[0098] First, features are extracted from the input sequence through a fully connected layer (FC). in Then, the extracted features are concatenated with the spatial and temporal embeddings to form in Subsequently, multiple multi-layer perceptrons (MLPs) are applied and residual connections are used. The calculation process of the lth layer is:

[0099]

[0100] in, The final extracted sequence is denoted as H mlp , called global sequence features.

[0101] Next, the local time features and the global sequence feature H mlp Perform fusion to obtain the fusion feature sequence. The specific method is:

[0102] The local time features and global sequence features are concatenated in the channel dimension, and the concatenated vectors are weighted by two fully connected layers. The weight calculation formula is:

[0103]

[0104] Where r is the weight; based on r, the local time features and the global sequence features are fused, and the fusion formula is:

[0105]

[0106] Among them, H is the fusion feature sequence.

[0107] Specifically, when the static graph matrix A l When directly applied to multivariate time series, it enhances the similarity of representations of series with long-term correlations, but significantly weakens their short-term fluctuations. s This may cause errors to propagate throughout the entire sequence. To solve this problem, this embodiment provides a time series decomposition graph convolution (TSDGC). Figure 4 To more clearly illustrate the process of TSDGC, the input of TSDGC is recorded as Zin , and the input Z here in is the fusion feature sequence H. First, use A l Decompose the sequence representation into the first long-term component and the first short-term component respectively, and the calculation formula is:

[0108]

[0109]

[0110] in, is the first long-term component, is the first short-term component. It is worth noting that the decomposition here is applied to the hidden layer representation rather than to the input sequence as in dynamic graph structure learning.

[0111] Specifically, TSDGC is used for the first long-term component Use A l Perform graph convolution to obtain the long-term convolution component; for the first short-term component Use A s Perform graph convolution to obtain the short-term convolution component. According to the long-term convolution component and the short-term convolution component, the total component is calculated as follows:

[0112]

[0113]

[0114]

[0115] in, is the long-term convolution component of the lth layer, is the short-term convolution component of the p-th segment in the l-th layer, is the total weight, W (l) is a learnable parameter, It is formed by concatenating the fragments of all short-term convolution components in the lth layer. Equations (18) and (19) describe the graph convolution process of long-term and short-term representations in detail respectively. Since the dynamic graph structure is segmented, the short-term representation is also processed by segment. Equation (20) summarizes the long-term and short-term representations of all layers. This method prevents short-term fluctuations from being smoothed out and prevents errors originating from the dynamic graph from propagating throughout the sequence.

[0116] Furthermore, for the total With the global sequence feature H mlp The specific method for fusion splicing to obtain the spliced ​​sequence is as follows: With the global sequence feature H mlp The concatenated vectors are concatenated along the channel dimension and the weights are calculated through two fully connected layers. The weight calculation formula is:

[0117]

[0118] Based on r, the total component and the global sequence feature H mlp Perform fusion and splicing processing, and the fusion calculation formula is:

[0119]

[0120] Among them, Z out The splicing sequence.

[0121] Specifically, the obtained fusion feature sequence H and splicing sequence Z out After the prediction result is obtained, the prediction loss value is calculated. Specifically, the prediction loss is calculated by the mean absolute error between the predicted value and the actual value. The calculation formula is:

[0122]

[0123] Among them, τ is the step size of the future time step that we want to predict, N is the total dimension of the multivariate time series, and T is the total duration. is the actual value, is the predicted value.

[0124] Embodiment 2

[0125] Based on the same inventive concept, this embodiment provides a multivariate time series prediction system based on sequence decomposition, and its principle of solving the problem is similar to the multivariate time series prediction method based on sequence decomposition provided in Example 1, and the repeated parts will not be repeated.

[0126] Reference Figure 5 As shown, an embodiment of the present invention provides a multivariate time series prediction system based on sequence decomposition, comprising:

[0127] The graph structure learning module is used to initialize the node embedding matrix and obtain the static graph matrix by performing inner product calculation on the node embedding matrix; the static graph matrix is ​​used to decompose the multivariate time series data into the second long-term component and the second short-term component; the second short-term component is the difference between the multivariate time series data and the second long-term component; the second short-term component and the static graph matrix are used, and the differential data is introduced to obtain the dynamic graph matrix;

[0128] The first fusion multi-layer perceptron module is used to fuse the local time features and the global sequence features to obtain a fusion feature sequence; wherein the local time features and the global sequence features are respectively obtained by extracting the multivariate time series data by the time convolution module and the feature extraction module;

[0129] The time series decomposition graph convolution module is used to decompose the fused feature sequence using a static graph matrix into a first long-term component and a first short-term component; wherein the first short-term component is the difference between the fused feature sequence and the first long-term component; the first long-term component is graph convolved using a static graph matrix to obtain a long-term convolution component; the first short-term component is graph convolved using a dynamic graph matrix to obtain a short-term convolution component; and the total component is obtained based on the long-term convolution component and the short-term convolution component.

[0130] The second fusion multi-layer perceptron module is used to fuse and splice the total component and the global sequence features to obtain a spliced ​​sequence;

[0131] The output module is used to fuse the fusion feature sequence and the splicing sequence to obtain the prediction result.

[0132] In this embodiment, the temporal convolution (TC) module, the first fused multi-layer perceptron module, the time series decomposition graph convolution (TSDGC) module and the second fused multi-layer perceptron module constitute a spatiotemporal layer, and the system includes K spatiotemporal layers. When K>1, the output of the kth spatiotemporal layer is used as the input of the k+1th spatiotemporal layer, and the output module receives and fuses the fused feature sequence of the K spatiotemporal layers and the output of the Kth spatiotemporal layer to obtain the prediction result. Wherein, K and k are integers and K≥1, 1≤k≤K, and the output of the kth spatiotemporal layer is the splicing sequence generated by the second fused multi-layer perceptron module. In addition, the total number of spatiotemporal layers can be determined according to actual needs.

[0133] Specifically, the transmission mode from each spatiotemporal layer to the output module is preferably to transmit information by skip connection, and the output module is preferably a multi-layer perceptron for converting features into the required dimensions. In the output module, the calculation formula for obtaining the prediction result is:

[0134]

[0135] in, is the final prediction result, Z represents the output result of the Kth spatiotemporal layer, and H k It is the fused feature sequence obtained in the first fused multilayer perceptron module at the kth spatiotemporal layer.

[0136] The graph structure learning module learns static and dynamic graph structures from multivariate time series data X. The feature extraction (Identification) module captures the intrinsic self-evolution patterns of variables without involving the interactions between variables. The feature extraction process can be referred to Figure 6, this module can extract features from each sequence separately and enhance the spatiotemporal features extracted by the spatiotemporal layer. In addition, the global sequence features extracted by the feature extraction module are input into the first fusion multilayer perceptron module of each spatiotemporal layer. The temporal convolution module is used to extract the temporal dimension dependency of variables. The time series decomposition graph convolution module extracts long-term and short-term inter-variable dependencies based on static and dynamic graph structures, respectively. Two fusion multilayer perceptron (FusionMLP) modules, namely the first fusion multilayer perceptron module and the second fusion multilayer perceptron module, are used to fuse the information extracted by the identification module, the TC module, and the TSDGC module. In order to cope with the distribution changes in the time series, reversible instance normalization (RevIN) is applied before the first spatiotemporal layer. The output module projects the features extracted from the spatiotemporal layer to the required dimensions to obtain the final result.

[0137] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A multivariate time series prediction method based on sequence decomposition, characterized in that: include: Extracting local time features and global sequence features of multivariate time series data, fusing the local time features and the global sequence features to obtain a fused feature sequence; initializing a node embedding matrix, and obtaining a static graph matrix by performing inner product calculation on the node embedding matrix; Decomposing the fused feature sequence by using the static image matrix into a first long-term component and a first short-term component; wherein the first short-term component is the difference between the fused feature sequence and the first long-term component; The first long-term component is convolved with the static graph matrix to obtain a long-term convolution component; the first short-term component is convolved with the dynamic graph matrix to obtain a short-term convolution component; wherein the dynamic graph matrix is ​​obtained by: decomposing the multivariate time series data with the static graph matrix to decompose a second long-term component and a second short-term component; the second short-term component is the difference between the multivariate time series data and the second long-term component; the dynamic graph matrix is ​​obtained by using the second short-term component and the static graph matrix and introducing differential data; Obtaining a total component according to the long-term convolution component and the short-term convolution component; Performing fusion and splicing processing on the total component and the global sequence feature to obtain a spliced ​​sequence; The fusion feature sequence and the splicing sequence are fused to obtain a prediction result.

2. A multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: By performing inner product calculation on the node embedding matrix, the calculation formula for obtaining the static graph matrix is: in, is a non-normalized static graph structure, A l is a static graph matrix, E is a randomly initialized node embedding matrix, ε is a very small number, and I N is the identity matrix, for The diagonal matrix of .

3. The multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: The calculation formulas of the second long-term component and the second short-term component are: X l =(A l ) n X; X s =X-X l ; Among them, X l is the second long-term component, X s is the second short-term component, X is the multivariate time series data, A l is the static graph matrix, and n is the number of smoothing iterations.

4. The multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: The calculation formula of the dynamic graph matrix is: in, is the pth segment of the second short-term component, W s ,W d ,W l is the learnable weight, h p is the node representation of the pth segment in the second short-term component, is the non-normalized dynamic graph matrix corresponding to the pth fragment, for The diagonal matrix of is the second short-term component X s The dynamic graph matrix corresponding to the pth segment in , ΔX p is the differential data corresponding to the pth fragment, ε is a very small number, I N is the identity matrix, E is a randomly initialized node embedding matrix, T is the total duration, L is the length of each segment, and ⊙ is the Hadamard product operator.

5. The multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: The mathematical expression of the differential data is: Among them, t is the time, T is the total duration, i∈{1,2,…,N}, x i (t) is the data of the i-th sequence at time t, x i (t-1) is the data of the i-th sequence at time t-1.

6. The multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: The method for fusing the local time feature and the global sequence feature to obtain a fused feature sequence is as follows: the local time feature and the global sequence feature are spliced ​​in the channel dimension, and the weights of the spliced ​​vectors are calculated through two fully connected layers. The weight calculation formula is: Among them, r is the weight, is the local time feature, H mlp is the global sequence feature; based on r, the local time feature and the global sequence feature are fused, and the fusion formula is: Wherein, H is the fusion feature sequence, and ⊙ is the Hadamard product operator.

7. The multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: According to the long-term convolution component and the short-term convolution component, the method of obtaining the total component is: in, is the long-term convolution component of the lth layer, is the short-term convolution component of the p-th segment in the l-th layer, is the total component, A l is the static graph matrix, is the second short-term component X s The dynamic graph matrix corresponding to the pth fragment in , W (l) is a learnable parameter, It is formed by concatenating the fragments of all the short-term convolution components in the lth layer.

8. The multivariate time series prediction method based on sequence decomposition according to claim 1, characterized in that: The prediction results are obtained by calculating the prediction loss value, and the calculation formula is: Among them, τ is the step size of the future time step that we want to predict, N is the total dimension of the multivariate time series, and T is the total duration. is the actual value, is the predicted value.

9. A multivariate time series prediction system based on sequence decomposition, characterized in that: include: A graph structure learning module, used to initialize a node embedding matrix and obtain a static graph matrix by performing inner product calculation on the node embedding matrix; Decomposing the multivariate time series data using the static graph matrix to decompose a second long-term component and a second short-term component; The second short-term component is the difference between the multivariate time series data and the second long-term component; the dynamic graph matrix is ​​obtained by using the second short-term component and the static graph matrix and introducing differential data; A first fusion multi-layer perceptron module is used to fuse local time features and global sequence features to obtain a fusion feature sequence; wherein the local time features and the global sequence features are respectively extracted from the multivariate time series data by a time convolution module and a feature extraction module; A time series decomposition graph convolution module is used to decompose the fused feature sequence using the static graph matrix into a first long-term component and a first short-term component; wherein the first short-term component is the difference between the fused feature sequence and the first long-term component; perform graph convolution on the first long-term component using the static graph matrix to obtain a long-term convolution component; perform graph convolution on the first short-term component using a dynamic graph matrix to obtain a short-term convolution component; and obtain a total component based on the long-term convolution component and the short-term convolution component; A second fusion multi-layer perceptron module is used to fuse and splice the total component with the global sequence feature to obtain a spliced ​​sequence; The output module is used to fuse the fusion feature sequence and the splicing sequence to obtain a prediction result.

10. A multivariate time series prediction system based on sequence decomposition according to claim 9, characterized in that: The temporal convolution module, the first fusion multi-layer perceptron module, the time series decomposition graph convolution module and the second fusion multi-layer perceptron module constitute a space-time layer. The system includes K space-time layers. When K>1, the output of the kth space-time layer is used as the input of the k+1th space-time layer. The output module receives and fuses the fused feature sequences of the K space-time layers and the output of the Kth space-time layer to obtain a prediction result; wherein K and k are integers and K≥1, 1≤k≤K.

Images