A multidimensional time series prediction method and system based on learning dynamic dependency relationships between variables
Through one-dimensional expansion convolution and dynamic graph structure, combined with attention mechanism, the problem of difficulty in capturing long-term dependencies in the existing technology is solved, and a multi-dimensional time series prediction with higher accuracy and robustness is achieved, suitable for complex scenarios.
Patent Information
- Application Number
- CN202411009738.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-26
AI Technical Summary
The prior art is difficult to fully capture the long-term dependencies and complex interactions in multi-dimensional time series data, and the static graph structure cannot truly reflect dynamic dependencies, resulting in insufficient model correlation analysis and affecting robustness.
One-dimensional expanded convolution is used to extract parallel timing features, combine global average pooling and attention mechanisms, dynamically adjust connection weights, and learn dynamic dependencies between variables through dynamic graph structure and graph convolution layer to generate a dedicated dynamic adjacency matrix , optimize the characteristic representation of time sequence data.
It significantly improves the feature extraction and dynamic dependency learning ability of time series data, improves the accuracy and robustness of multi-dimensional time series prediction, can maintain good prediction performance under complex situations, and is suitable for a wider range of application scenarios.
Smart Images

Figure CN118551189B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data analysis, and in particular to a multidimensional time series prediction method and system based on learning dynamic dependency relationships between variables. Background Art
[0002] With the rapid development of science and technology, a large amount of multidimensional time series data has been generated in various fields. These data are not only large in scale, but also highly complex, and may contain various noises, seasonal changes, trends, and nonlinear relationships. This information can not only help us better understand and explain complex phenomena and relationships, but also be used to predict and optimize the performance of various systems, thereby providing more valuable information for decision-making. Dynamic relational learning is based on the interdependence between variables in actual data. It further mines time series data and can further improve the prediction ability of the model by learning the time-varying nature of variables, thereby providing more in-depth and valuable information for decision-making in different fields.
[0003] At present, the learning methods of dependencies between variables are mainly divided into two categories: one is the convolution-based method, and the other is the graph neural network (GNN)-based method. The convolution-based method is committed to using models such as CNN or RNN to capture local features in the data and short-term dependency patterns between variables, and combines the attention mechanism to focus on features that are more relevant to the predicted value. However, due to the complex and changeable interdependencies between variables in actual data, this convolution-based method may sometimes fail to fully capture long-term dependencies and complex interactions. Therefore, it is very difficult to extract the long-term dependency changes of variables in the data, and it is difficult to obtain reliable results. The core idea of the method based on graph neural network (GNN) is to capture the complex relationship between nodes through GNN, model the interdependencies between variables as a graph structure, and use structures such as graph convolutional neural network (GCN) and graph attention network (GAT) to learn node features and global characteristics of the graph. However, in the real world, the correlation between variables will change over time, and the static graph structure cannot truly reflect the dynamic dependencies in the actual complex system. While using convolution kernels to capture local patterns and introducing attention mechanisms to focus on the main features, ignoring the time-varying nature of variables will lead to deficiencies in the model in association analysis, which in turn affects the enhancement of robustness. Summary of the invention
[0004] In view of the shortcomings of the above existing methods, the purpose of the present invention is to propose a multi-dimensional time series prediction method and system based on learning dynamic dependency relationships between variables. The method extracts and utilizes the dynamic relationships between variables and multi-level hidden features of the original data.
[0005] In order to achieve the above object, the technical solution of the present invention is specifically as follows:
[0006] S1. Divide the original data into training sample set, validation sample set and test sample set in a ratio of 6:2:2.
[0007] S2: Extract and fuse features of the time series data of the training sample set in S1. The specific steps are as follows:
[0008] S2.1. Use one-dimensional dilated convolution to transform the training sample set into input features From the input features Extract information from the kernel and generate parallel and different levels of temporal features. The one-dimensional dilated convolution is of size 1×k j The specific expression of J groups of one-dimensional dilated convolutions (1≤j≤J) is as follows:
[0009]
[0010] Where: Represents input features The hidden state of the mth variable at time step t, i is the number of layers, m is the variable, and t is the time step. is the time series feature, Represented by a size of 1×k j The hidden state of the temporal features extracted by the dilated convolution of the mth variable of the i-th layer at time step t, q i is the convolution expansion rate of the i-th layer, Indicates that the size is 1×k j The dilated convolution kernel, Represents input features The mth variable in tq i The hidden state of τ time step, τ represents any time step in the whole process, k j represents the hidden state of the dilated convolution at time step t, k j -1 represents the hidden state of the dilated convolution at t-1.
[0011] S2.2. Because different dimensions can extract different features, the features containing global information are sorted from the two dimensions of variables and time series. (i) Perform global average pooling (GAP) to extract global features and generate weights corresponding to features at different levels Weight With timing characteristics Multiply and sum to get the multi-level time series feature fusion result The specific expression is as follows:
[0012]
[0013]
[0014] Where: Sel (i) is a feature containing global information, i is the number of layers, is the weight of the feature extracted by the i-th layer of dilated convolution, is the j feature descriptions generated by J sets of parallel convolutional layers, It is the result of multi-level feature fusion.
[0015] S3: Extract key information and global information from the time series data fused in S2 and dynamically adjust the connection weights to optimize the feature representation of the time series data. The specific steps are as follows:
[0016] S3.1. In the channel dimension, use a component consisting of a convolutional layer and a ReLU activation layer. The result of fusing multi-level features Convert to The transformed features Perform maximum pooling and average pooling operations to generate feature representations of the maximum pooling layer And the feature representation of the average pooling layer Two sets of features; these two sets of features are represented as compressed single channels, and the temporal attention weights T are generated through the Sigmoid activation function (i) , the specific expression is as follows:
[0017]
[0018] Where: It is the result of fusing multi-level features. After the convolution component The transformed features, It is a component composed of convolutional layer and ReLU activation layer. It is the result of multi-level feature fusion. is the feature representation of the maximum pooling of the temporal attention part, is the feature representation of the average pooling of the temporal attention part, T (i) is the temporal attention weight, Indicates the features after pooling and concatenation in the temporal attention part Perform convolution operation, Represents the features after temporal attention is partially pooled and concatenated.
[0019] S3.2. Transformed features Apply global maximum pooling and global average pooling layers to generate feature representations of the global maximum pooling layer respectively and the feature representation of the global average pooling layer Extract the key information of variable features in the channel dimension and capture the global timing information, and use the Sigmoid activation function to generate the channel attention weight C (i) :
[0020]
[0021] Where: It is the result of fusing multi-level features. After the convolution component The transformed features, is the feature representation of the maximum pooling of the channel attention part, is the feature representation of the average pooling of the channel attention part, C (i) is the channel attention weight, Component (i) (·) is a convolution group consisting of a convolution layer and a ReLU activation layer in the channel dimension. Represents the features after the channel attention parts are pooled and concatenated.
[0022] S3.3. The temporal attention weight T (i) and channel attention weight C (i) The transformed features Multiply and combine to get the output
[0023]
[0024] Where: represents the output, and ⊙ is the process of multiplication and combination.
[0025] S4. Information is transferred through node embedding of the dynamic graph structure and node feature updating of the dynamic graph convolution layer to further learn important patterns and changes in the data. The specific steps are as follows:
[0026] S4.1. In the initial state, the embedding matrix is represented as the initial feature of the node. By calculating the similarity score matrix S between the nodes, the score matrix S is normalized using the softmax function to obtain the adjacency matrix A between the nodes and the original adjacency matrix A in each dynamic graph convolution layer. (0) , the specific expression is as follows:
[0027] N 1 =tanh(α 1 E 1 )
[0028] N 2 =tan(α 2 E 2 )
[0029]
[0030] A=Soft max(S)c
[0031]
[0032] Where: N 1 is the feature representation of the first matrix randomly initialized for embedding, N 2 is the feature representation of the randomly initialized embedding second matrix, α 1 is the network parameter of the fully connected layer when randomly initializing the embedding of the first matrix, α 2 is the network parameter of the fully connected layer when randomly initializing the embedding second matrix, E 1 is the first matrix of randomly initialized embedding, E 2 is the second matrix of randomly initialized embedding, S is the similarity score matrix between nodes, The feature representation of the transposed matrix of the randomly initialized embedding second matrix, is the feature representation of the transposed matrix of the randomly initialized embedding first matrix, A is the adjacency matrix between nodes, c is the embedding dimension of the graph nodes, and A (0) is the original adjacency matrix in each dynamic graph convolution layer, β 1 is the first model parameter to be learned, β 2 is the second model parameter that needs to be learned.
[0033] S4.2. At time step t, the adjacency matrix A between the first node u and the second node v (0) The update method of the elements in is The specific expression is as follows:
[0034]
[0035] Where: is the adjacency matrix A between the first node u and the second node v at time step t (0) The updating method of the elements in , f(·) is an updating function used to calculate the new adjacency matrix elements according to the historical adjacency matrix and node feature information; Represents the adjacency matrix A between the first node u and the second node v at time step t-1 (0) How to update the elements in and Represent the feature vectors of the first node u and the second node v at time step t respectively.
[0036] Since when the variable node or edge changes, the adjacency matrix A (0)It needs to be adjusted accordingly to reflect the latest connection status; therefore, at each time step t, a time-dependent convolution kernel K is defined t , according to the update method of the dynamic adjacency matrix, calculate the dynamically updated adjacency matrix A at time step t (t) , and perform convolution operations to dynamically update the adjacency matrix A at time step t (t) The characteristic tensor Z t , the specific expression is as follows:
[0037]
[0038] Where: Z t Indicates the dynamic update of the adjacency matrix A at time step t (t) The feature tensor of is a diagonal matrix, is the adjacency matrix with self-loops added at time step t, X t is the feature vector at time step t, K t is a time-dependent convolution kernel.
[0039] S4.3. The feature tensor Z of all time steps t Concatenate on the channel dimension to get a new tensor Z, pool it, and reduce the dimension to a two-dimensional matrix Get the output feature matrix of each dynamic graph convolution layer Information transmission and feature fusion are performed layer by layer to obtain the final result of dynamic relationship learning. In particular, when the number of dynamic graph convolutional layers is 1, The result of learning the feature representation and dynamic relationship of each part The feature F is obtained by compression and skip connection d , through the output module, the feature F d Mapping to the final prediction result The specific expression is as follows:
[0040]
[0041]
[0042] Where: represents the output feature matrix of each dynamic graph convolution layer, G represents the number of dynamic graph convolution layers, V is the weight matrix, is the feature tensor Z of all time steps t The two-dimensional matrix obtained by splicing on the channel dimension, e is the bias vector, is the result of dynamic relationship learning. Indicates the pooled and concatenated features Perform convolution operation, represents the pooled and concatenated features, is the predicted value when the future time step is h, h is the predicted future time step, W out-1 is the weight of the first convolution kernel of the output module, W out-2 is the weight of the second convolution kernel of the output module, F d are the features after compression and skip connection.
[0043] S5. Evaluate the effectiveness of the model by calculating the square root error RRSE and the empirical correlation coefficient CORR. The specific steps are as follows:
[0044] S5.1. The accuracy of the prediction model is measured by calculating the RRSE, which is a measure of the degree of deviation between the predicted value and the actual value:
[0045]
[0046] Where: X t represents the actual value at the historical time step t, It indicates the predicted value when the future time step is h, mean(X t ) represents the mean of the actual values.
[0047] S5.2. The accuracy of the prediction model is measured by calculating the metric CORR that quantifies the correlation between the rolling prediction step sequence and the true value:
[0048]
[0049] Where: Represents the mean of the predicted values. The value range of CORR is (-1,1).
[0050] Another object of the present invention is to provide a multi-dimensional time series prediction system based on dynamic dependency relationship learning between variables, which mainly includes: a time series feature fusion module, a dynamic relationship learning module, and an attention mechanism module; the time series feature fusion module fusions the input multi-level time series feature results Perform feature extraction and fusion; the dynamic relationship learning module combines the hierarchical temporal features after feature extraction and fusion into a fusion result. Generate a dedicated dynamic adjacency matrix to learn and model the dynamic dependency relationship between variables; the attention mechanism module is used to analyze the node connections in the dynamic adjacency matrix generated by the dynamic relationship learning unit and their evolution over time, strengthen more important connections between variables through attention scores, focus on key dependencies, and effectively control the flow and integration of information;
[0051] The time series feature fusion module is used to extract and fuse the features of the input multi-dimensional time series data; the original data set is divided into a training sample set, a verification sample set and a test sample set in a ratio of 6:2:2, and the time series feature fusion module is used to extract and fuse the features of the input multi-dimensional time series data. t Processing, extracting useful information, generating four parallel and different levels of time series features, adaptively adjusting and selecting the weights of multi-level time series features Get the multi-level time series feature fusion result Effectively capture key patterns and structures in time series data;
[0052] The dynamic relationship learning module is used to generate a dedicated dynamic adjacency matrix from the extracted time series features, learn and model the dynamic dependency relationship between the variables; by establishing a dynamic relationship learning unit, a dynamic graph structure, calculating the similarity score matrix between nodes, introducing a dynamic graph convolution layer to automatically adjust the weights in the adjacency matrix and capture the complex dynamic relationship between variables in multi-dimensional time series data, further learning the important patterns and changes in the data;
[0053] The attention mechanism module is used to analyze the node connections in the dynamic adjacency matrix generated by the dynamic relationship learning unit and its evolution over time, strengthen the more important connections between variables through attention scores, focus on key dependencies, and effectively control the flow and integration of information; by establishing an attention mechanism module, the importance of the connection between each node at each time step is evaluated from the two dimensions of time and channel, and the connection weights are dynamically adjusted to optimize the feature representation of time series data.
[0054] Beneficial effects of the present invention
[0055] 1. The multidimensional time series prediction method of the present invention significantly improves the ability of time series data feature extraction, dynamic dependency learning and key dependency strengthening through innovative module design and algorithm optimization, and finally achieves a multidimensional time series prediction effect with higher accuracy and stronger robustness. These improvements not only solve many bottleneck problems in the prior art, but also provide a more effective solution for multidimensional time series prediction in practical applications.
[0056] 2. By comprehensively applying multi-level feature fusion, dynamic relationship learning and attention mechanism, the present invention greatly improves the accuracy and robustness of prediction, can maintain good prediction performance in a variety of complex situations, and is suitable for a wider range of application scenarios.
[0057] 3. Most existing technologies perform feature extraction at a single level, which makes it difficult to fully capture the complex patterns and structures in the data. However, the present invention effectively captures more comprehensive and deeper time series feature information through multi-level time series feature extraction and fusion, thereby improving the model's expressiveness and prediction effect.
[0058] 4. Traditional methods usually assume that the relationship between variables is static, ignoring the dynamic changes in the relationship between variables in time series data. The present invention uses a dynamic relationship learning unit to adjust and learn the dynamic dependency relationship between variables in real time, which more accurately reflects the actual situation in the data and improves the prediction performance of the model.
[0059] 5. Compared with ordinary time series prediction methods, the present invention introduces an attention mechanism module, which dynamically adjusts the connection weights between variables, strengthens important dependencies, optimizes the feature representation of time series data, and improves the effect of the model in processing complex time series data. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the technical solution implemented by the present invention, the various modules required in the embodiment description are briefly introduced below. Obviously, the accompanying drawings described below are only flow charts of the present invention. For ordinary technicians in this field, they can also expand based on this accompanying drawing and obtain other accompanying drawings without paying creative labor.
[0061] Figure 1 A schematic diagram of a method for learning dynamic dependency relationships between variables according to the present invention;
[0062] Figure 2 The experimental results of the simulation content of the present invention on the Solar-Energy data set;
[0063] Figure 3 The figure shows the experimental results of the simulation content of the present invention on the Electricity dataset. DETAILED DESCRIPTION
[0064] The present invention is further described in detail below in conjunction with specific embodiments, but the protection scope of the present invention is not limited to the described contents.
[0065] Example 1
[0066] like Figure 1 As shown, a multi-dimensional time series prediction method and system based on learning dynamic dependency relationships between variables, the specific steps are as follows:
[0067] S1. Divide the original data into training sample set, validation sample set and test sample set in a ratio of 6:2:2.
[0068] S2: Extract and fuse features of the time series data of the training sample set in S1. The specific steps are as follows:
[0069] S2.1. Use one-dimensional dilated convolution to transform the training sample set into input features From the input features Extract information from the kernel to generate parallel and different levels of temporal features. The one-dimensional dilated convolution is a 1×k j The specific expression of J groups of one-dimensional dilated convolutions (1≤j≤J) is as follows:
[0070]
[0071]
[0072] Where: Represents input features The hidden state of the mth variable at time step t, i is the number of layers, m is the variable, and t is the time step. is the time series feature, Represented by a size of 1×k j The hidden state of the temporal features extracted by the dilated convolution of the mth variable of the i-th layer at time step t, q i is the convolution expansion rate of the i-th layer, Indicates that the size is 1×k j The dilated convolution kernel, Represents input features The mth variable in tq i The hidden state of τ time step, τ represents any time step in the whole process, k j represents the hidden state of the dilated convolution at time step t, k j -1 represents the hidden state of the dilated convolution at t-1.
[0073] S2.2. Because different dimensions can extract different features, the features containing global information are sorted from the two dimensions of variables and time series. (i) Perform global average pooling (GAP) to extract global features and generate weights corresponding to features at different levels Weight With timing characteristics Multiply and sum to get the multi-level time series feature fusion result The specific expression is as follows:
[0074]
[0075] Where: Sel (i) is a feature containing global information, i is the number of layers, is the weight of the feature extracted by the i-th layer of dilated convolution, is the j feature descriptions generated by J sets of parallel convolutional layers, It is the result of multi-level feature fusion.
[0076] S3: Extract key information and global information from the time series data fused in S2 and dynamically adjust the connection weights to optimize the feature representation of the time series data. The specific steps are as follows:
[0077] S3.1. In the channel dimension, use a component consisting of a convolutional layer and a ReLU activation layer. The result of fusing multi-level features Convert to The transformed features Perform maximum pooling and average pooling operations to generate feature representations of the maximum pooling layer And the feature representation of the average pooling layer Two sets of features; these two sets of features are represented as compressed single channels, and the temporal attention weights T are generated through the Sigmoid activation function (i) , the specific expression is as follows:
[0078]
[0079] Where: It is the result of fusing multi-level features. After the convolution component The transformed features, It is a component composed of convolutional layer and ReLU activation layer. It is the result of multi-level feature fusion. is the feature representation of the maximum pooling of the temporal attention part, is the feature representation of the average pooling of the temporal attention part, T (i) is the temporal attention weight, Indicates the features after pooling and concatenation in the temporal attention part Perform convolution operation, Represents the features after temporal attention is partially pooled and concatenated.
[0080] S3.2. Transformed features Apply global maximum pooling and global average pooling layers to generate feature representations of the global maximum pooling layer respectively and the feature representation of the global average pooling layer Extract the key information of variable features in the channel dimension and capture the global timing information, and use the Sigmoid activation function to generate the channel attention weight C (i) :
[0081]
[0082] Where: It is the result of fusing multi-level features. After the convolution component The transformed features, is the feature representation of the maximum pooling of the channel attention part, is the feature representation of the average pooling of the channel attention part, C (i) is the channel attention weight, Componet (i) (·) is a convolution group consisting of a convolution layer and a ReLU activation layer in the channel dimension. Represents the features after the channel attention parts are pooled and concatenated.
[0083] S3.3. The temporal attention weight T (i) and channel attention weight C (i) The transformed features Multiply and combine to get the output
[0084]
[0085] Where: represents the output, and ⊙ is the process of multiplication and combination.
[0086] S4. Information is transferred through node embedding of the dynamic graph structure and node feature updating of the dynamic graph convolution layer to further learn important patterns and changes in the data. The specific steps are as follows:
[0087] S4.1. In the initial state, the embedding matrix is represented as the initial feature of the node. By calculating the similarity score matrix S between the nodes, the score matrix S is normalized using the softmax function to obtain the adjacency matrix A between the nodes and the original adjacency matrix A in each dynamic graph convolution layer. (0) , the specific expression is as follows:
[0088] N 1 =tanh(α 1 E 1 )
[0089] N 2 =tanh(α 2 E 2 )
[0090]
[0091] A=Soft max(S)c
[0092]
[0093] Where: N 1 is the feature representation of the first matrix randomly initialized for embedding, N 2 is the feature representation of the randomly initialized embedding second matrix, α 1is the network parameter of the fully connected layer when randomly initializing the embedding of the first matrix, α 2 is the network parameter of the fully connected layer when randomly initializing the embedding second matrix, E 1 is the first matrix of randomly initialized embedding, E 2 is the second matrix of randomly initialized embedding, S is the similarity score matrix between nodes, The feature representation of the transposed matrix of the randomly initialized embedding second matrix, is the feature representation of the transposed matrix of the randomly initialized embedding first matrix, A is the adjacency matrix between nodes, c is the embedding dimension of the graph nodes, and A (0) is the original adjacency matrix in each dynamic graph convolution layer, β 1 is the first model parameter to be learned, β 2 is the second model parameter that needs to be learned.
[0094] S4.2. At time step t, the adjacency matrix A between the first node u and the second node v (0) The update method of the elements in is The specific expression is as follows:
[0095]
[0096] Where: is the adjacency matrix A between the first node u and the second node v at time step t (0) The updating method of the elements in , f(·) is an updating function used to calculate the new adjacency matrix elements according to the historical adjacency matrix and node feature information; Represents the adjacency matrix A between the first node u and the second node v at time step t-1 (0) How to update the elements in and Represent the feature vectors of the first node u and the second node v at time step t respectively.
[0097] Since when the variable node or edge changes, the adjacency matrix A (0) It needs to be adjusted accordingly to reflect the latest connection status; therefore, at each time step t, a time-dependent convolution kernel K is defined t , according to the update method of the dynamic adjacency matrix, calculate the dynamically updated adjacency matrix A at time step t (t) , and perform convolution operations to dynamically update the adjacency matrix A at time step t (t) The characteristic tensor Z t , the specific expression is as follows:
[0098]
[0099] Where: Z t Indicates the dynamic update of the adjacency matrix A at time step t (t) The feature tensor of is a diagonal matrix, is the adjacency matrix with self-loops added at time step t, X t is the feature vector at time step t, K t is a time-dependent convolution kernel.
[0100] S4.3. The feature tensor Z of all time steps t Concatenate on the channel dimension to get a new tensor Z, pool it, and reduce the dimension to a two-dimensional matrix Get the output feature matrix of each dynamic graph convolution layer Information transmission and feature fusion are performed layer by layer to obtain the final result of dynamic relationship learning. In particular, when the number of dynamic graph convolutional layers is 1, The result of learning the feature representation and dynamic relationship of each part The feature F is obtained by compression and skip connection d , through the output module, the feature F d Mapping to the final prediction result The specific expression is as follows:
[0101]
[0102] Where: represents the output feature matrix of each dynamic graph convolution layer, G represents the number of dynamic graph convolution layers, V is the weight matrix, is the feature tensor Z of all time steps t The two-dimensional matrix obtained by splicing on the channel dimension, e is the bias vector, is the result of dynamic relationship learning. Indicates the pooled and concatenated features Perform convolution operation, represents the pooled and concatenated features, is the predicted value when the future time step is h, h is the predicted future time step, W out-1 is the weight of the first convolution kernel of the output module, W out-2 is the weight of the second convolution kernel of the output module, F d are the features after compression and skip connection.
[0103] S5. Evaluate the effectiveness of the model by calculating the square root error RRSE and the empirical correlation coefficient CORR. The specific steps are as follows:
[0104] S5.1. The accuracy of the prediction model is measured by calculating the RRSE, which is a measure of the degree of deviation between the predicted value and the actual value:
[0105]
[0106] Where: X t represents the actual value at the historical time step t, It indicates the predicted value when the future time step is h, mean(X t ) represents the mean of the actual values;
[0107] S5.2. The accuracy of the prediction model is measured by calculating the metric CORR that quantifies the correlation between the rolling prediction step sequence and the true value:
[0108]
[0109] Where: Represents the mean of the predicted values. The value range of CORR is (-1,1).
[0110] A multi-dimensional time series prediction system based on learning dynamic dependency relationships between variables, mainly including: a time series feature fusion module, a dynamic relationship learning module, and an attention mechanism module; the time series feature fusion module fusions the input multi-level time series feature results Perform feature extraction and fusion; the dynamic relationship learning module combines the hierarchical temporal features after feature extraction and fusion into a fusion result. Generate a dedicated dynamic adjacency matrix to learn and model the dynamic dependency relationship between variables; the attention mechanism module is used to analyze the node connections in the dynamic adjacency matrix generated by the dynamic relationship learning unit and its evolution over time, strengthen more important connections between variables through attention scores, focus on key dependencies, and effectively control the flow and integration of information.
[0111] The time series feature fusion module is used to extract and fuse the features of the input multi-dimensional time series data; the original data set is divided into a training sample set, a verification sample set and a test sample set in a ratio of 6:2:2, and the time series feature fusion module is used to extract and fuse the features of the input multi-dimensional time series data. t Processing, extracting useful information, generating four parallel and different levels of time series features, adaptively adjusting and selecting the weights of multi-level time series features Get the multi-level time series feature fusion result Effectively capture key patterns and structures in time series data.
[0112] The dynamic relationship learning module is used to generate a dedicated dynamic adjacency matrix from the extracted time series features, learn and model the dynamic dependency relationship between the variables; by establishing a dynamic relationship learning unit, a dynamic graph structure, calculating the similarity score matrix between nodes, introducing a dynamic graph convolution layer to automatically adjust the weights in the adjacency matrix and capture the complex dynamic relationship between variables in multidimensional time series data, further learning important patterns and changes in the data.
[0113] The attention mechanism module is used to analyze the node connections in the dynamic adjacency matrix generated by the dynamic relationship learning unit and its evolution over time, strengthen the more important connections between variables through attention scores, focus on key dependencies, and effectively control the flow and integration of information; by establishing an attention mechanism module, the importance of the connection between each node at each time step is evaluated from the two dimensions of time and channel, and the connection weights are dynamically adjusted to optimize the feature representation of time series data.
[0114] The effect of the present invention can be further illustrated by the following experiments, which are specifically as follows:
[0115] The present invention uses Python 3.8 software and PyTorch 15.1 deep learning framework for simulation. Four public real data sets are used in the experiment. Table 1 gives the relevant statistical information of the data sets.
[0116] Table 1 Dataset related statistics
[0117]
[0118] First, the algorithm of the present invention is used on four data sets. In order to prove the effectiveness of the algorithm, considering the popularity and novelty of the algorithm, four comparative methods, VAR-MLP, LST-Attn, TPA-LSTM and MTGNN, are selected for comparison. In this experiment, all data sets are divided into training set (60%), validation set (20%) and test set (20%) in a ratio of 6:2:2. The input sequence length is 168, the output sequence length is 1, the selected prediction ranges are 3, 6, 12 and 24, and the learning rate is 1×10 -4 The Adam (Adaptive Moment Estimation) optimizer is used to optimize the network parameters.
[0119] Secondly, all experimental data sets are trained for 100 iterations, and the square root error and empirical correlation coefficient are calculated according to the formula in step 5. The results are shown in Table 2. The best result is shown in bold, and the second best result is underlined.
[0120] Table 2 Prediction results of different models
[0121]
[0122] As can be seen from Table 2, the prediction effect of the present invention is better than that of the current popular methods. Among them, from the perspective of CORR index, the present invention is significantly higher than other methods in all data sets; compared with other methods, the present invention has lower RRSE indicators. This shows that the present invention can better extract the changing trend of the variables themselves and the dynamic dependency relationship between variables, so the present method is more effective and more robust than other methods.
[0123] Figure 2 and Figure 3 The experimental results of the multidimensional time series prediction method based on learning dynamic dependencies between variables described in the simulation content of the present invention on two representative data sets: Solar-Energy and Electricity. The Solar-Energy data set records the solar power generation records of 137 photovoltaic power stations in Alabama, USA every 10 minutes in 2006; the Electricity data set records the electricity consumption (kWh) of 321 users in Portugal every 1 / 4 hour from 2012 to 2014. For the convenience of processing, the present invention converts the consumption data every 15 minutes into the consumption data per hour.
[0124] In the figure, DRM represents the method of the present invention, and w / o-DRM is a prediction method without a dynamic relationship learning unit. The prediction ranges are selected as 3, 6, and 12, and 50 cycles are run on each validation set. The required running time and the average value of CORR are recorded every 10 cycles.
[0125] like Figure 2 As shown in the figure, in the Solar-Energy dataset, since the dataset does not contain night readings and the power generation of the power station is mainly related to the weather conditions of the day, the dynamic dependence between the variables is not obvious. In this case, the prediction effect with DRM is not significantly different from that without DRM. Figure 3 As shown in the figure, in the Electricity dataset, the prediction method with DRM performs more significantly under different prediction ranges. As the prediction range increases, the average CORR value of the model at each time span is significantly improved. In addition, the training time of the model is reduced in both datasets, indicating that the invention has better performance and efficiency when dealing with a wider prediction range.
[0126] The present invention realizes the effectiveness of using dynamic relationships between variables for prediction in multidimensional time series. In order to extract and utilize as much hidden information as possible from the data, multi-level feature extraction and adaptive weight adjustment are used to capture key patterns and structures; dynamic graph convolution layers are introduced to capture the complex dynamic relationships between variables, and a gating mechanism is used to selectively combine historical dependency graphs based on the degree of relevance between nodes and the current time step. Figure 2 and Figure 3 The numerical results show the effectiveness of the present invention in incorporating captured dynamic dependencies between variables into the prediction.
[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.
Claims
1. A multidimensional time series prediction method based on learning dynamic dependencies between variables, It is characterized in that The following steps are involved: S1. Divide the original data into a training set, a validation sample set, and a test sample set, wherein the original data includes: the power generation of the solar power station in the Solar-Energy data set or the electricity consumption of the user in the Electricity data set; S2, extract and fuse features from the time series data of the training sample set in S1; The specific steps of S2 are as follows: S2.
1. Use one-dimensional dilated convolution to transform the training sample set into input features From the input features Extract information from the , and generate parallel and different levels of time series features. The specific expression is as follows: Where: Represents input features The hidden state of the mth variable at time step t, i is the number of layers, m is the variable, and t is the time step. is the time series feature, Represented by a size of 1×k j The hidden state of the temporal features extracted by the dilated convolution of the mth variable of the i-th layer at time step t, q i is the convolution expansion rate of the i-th layer, Indicates that the size is 1×k j The dilated convolution kernel, Represents input features The mth variable in tq i The hidden state of τ time step, τ represents any time step in the whole process, k j represents the hidden state of the dilated convolution at time step t, k j -1 indicates the hidden state of the dilated convolution at t-1; S2.
2. Sel the features containing global information from the two dimensions of variables and time series (i) Perform global average pooling to extract global features and generate weights corresponding to features at different levels Weight With timing characteristics Multiply and sum to get the multi-level time series feature fusion result The specific expression is as follows: Where: Sel (i) is a feature containing global information, i is the number of layers, is the weight of the feature extracted by the i-th layer of dilated convolution, is the j feature descriptions generated by J sets of parallel convolutional layers, It is the result of multi-level feature fusion; S3, extract key information and global information from the time series data fused in S2 and dynamically adjust the connection weights to optimize the feature representation of the time series data; The specific steps of S3 are as follows: S3.
1. Using components consisting of convolutional layers and ReLU activation layers The result of fusing multi-level features Convert to The transformed features Perform maximum pooling and average pooling operations to generate feature representations of the maximum pooling layer And the feature representation of the average pooling layer Two sets of features; these two sets of features are represented as compressed single channels, and the temporal attention weights T are generated through the Sigmoid activation function (i) , the specific expression is as follows: Where: It is the result of fusing multi-level features. After the convolution component The transformed features, It is a component composed of convolutional layer and ReLU activation layer. It is the result of multi-level feature fusion. is the feature representation of the maximum pooling of the temporal attention part, is the feature representation of the average pooling of the temporal attention part, T (i) is the temporal attention weight, Indicates the features after pooling and concatenation in the temporal attention part Perform convolution operation, Represents the features after temporal attention is partially pooled and concatenated; S3.
2. Transformed features Apply global maximum pooling and global average pooling layers to generate feature representations of the global maximum pooling layer respectively and the feature representation of the global average pooling layer Extract the key information of variable features in the channel dimension and capture the global timing information, and use the Sigmoid activation function to generate the channel attention weight C (i) : Where: It is the result of fusing multi-level features. After the convolution component The transformed features, is the feature representation of the maximum pooling of the channel attention part, is the feature representation of the average pooling of the channel attention part, C (i) is the channel attention weight, Component (i) (·) is a convolution group consisting of a convolution layer and a ReLU activation layer in the channel dimension. Represents the features after the channel attention parts are pooled and concatenated; S3.
3. The temporal attention weight T (i) and channel attention weight C (i) The transformed features Multiply and combine to get the output Where: represents the output, ⊙ is the process of multiplication and combination; S4, through the node embedding of the dynamic graph structure and the node feature update of the dynamic graph convolution layer, information is transferred to further learn the important patterns and changes in the data; The specific steps of S4 are as follows: S4.
1. In the initial state, the embedding matrix is represented as the initial feature of the node. By calculating the similarity score matrix S between the nodes, the score matrix S is normalized using the softmax function to obtain the adjacency matrix A between the nodes and the original adjacency matrix A in each dynamic graph convolution layer. (0) , the specific expression is as follows: N 1 =tanh(α 1 E 1 ) N 2 =tanh(α 2 E 2 ) A=Softmax(S)c Where: N 1 is the feature representation of the first matrix randomly initialized for embedding, N 2 is the feature representation of the randomly initialized embedding second matrix, α 1 is the network parameter of the fully connected layer when randomly initializing the embedding first matrix, α 2 is the network parameter of the fully connected layer when randomly initializing the embedding second matrix, E 1 is the first matrix of randomly initialized embedding, E 2 is the second matrix of randomly initialized embedding, S is the similarity score matrix between nodes, The feature representation of the transposed matrix of the randomly initialized embedding second matrix, is the feature representation of the transposed matrix of the randomly initialized embedding first matrix, A is the adjacency matrix between nodes, c is the embedding dimension of the graph node, and A (0) is the original adjacency matrix in each dynamic graph convolution layer, β 1 is the first model parameter to be learned, β 2 is the second model parameter that needs to be learned; S4.
2. At time step t, the adjacency matrix A between the first node u and the second node v (0) The update method of the elements in is The specific expression is as follows: Where: is the adjacency matrix A between the first node u and the second node v at time step t (0) The updating method of the elements in , f(·) is an updating function used to calculate the new adjacency matrix elements according to the historical adjacency matrix and node feature information; Represents the adjacency matrix A between the first node u and the second node v at time step t-1 (0) How to update the elements in and Represent the feature vectors of the first node u and the second node v at time step t respectively; Since when the variable node or edge changes, the adjacency matrix A (0) It needs to be adjusted accordingly to reflect the latest connection status; therefore, at each time step t, a time-dependent convolution kernel K is defined t , according to the update method of the dynamic adjacency matrix, calculate the dynamically updated adjacency matrix A at time step t (t) , and perform convolution operations to dynamically update the adjacency matrix A at time step t (t) The characteristic tensor Z t , the specific expression is as follows: Where: Z t Indicates the dynamic update of the adjacency matrix A at time step t (t) The feature tensor of is a diagonal matrix, is the adjacency matrix with self-loops added at time step t, X t is the feature vector at time step t, K t is a time-dependent convolution kernel; S4.
3. The feature tensor Z of all time steps t Concatenate on the channel dimension to get a new tensor Z, pool it, and reduce the dimension to a two-dimensional matrix Get the output feature matrix of each dynamic graph convolution layer Information transmission and feature fusion are performed layer by layer to obtain the final result of dynamic relationship learning. In particular, when the number of dynamic graph convolutional layers is 1, The result of learning the feature representation and dynamic relationship of each part The feature F is obtained by compression and skip connection d , through the output module, the feature F d Mapping to the final prediction result The specific expression is as follows: Where: represents the output feature matrix of each dynamic graph convolution layer, G represents the number of dynamic graph convolution layers, V is the weight matrix, is the feature tensor Z of all time steps t The two-dimensional matrix obtained by splicing on the channel dimension, e is the bias vector, is the result of dynamic relationship learning. Indicates the features after pooling and splicing Perform convolution operation, represents the pooled and concatenated features, is the predicted value when the future time step is h, h is the predicted future time step, W out-1 is the weight of the first convolution kernel of the output module, W out-2 is the weight of the second convolution kernel of the output module, F d It is the feature after compression and skip connection; S5. Evaluate the model validity by calculating the square root error RRSE and the empirical correlation coefficient CORR.
2. A multidimensional time series prediction method based on learning dynamic dependencies between variables according to claim 1, Features: The original data set of S1 is divided into a training sample set, a test sample set, and a validation sample set in a ratio of 6:2:
2.
3. According to claim 1, a multidimensional time series prediction method based on learning dynamic dependencies between variables, Features: The specific steps of S5 are as follows: S5.
1. The accuracy of the prediction model is measured by calculating the RRSE, which is a measure of the degree of deviation between the predicted value and the actual value: Where: X t represents the actual value at the historical time step t, It indicates the predicted value when the future time step is h, mean(X t ) represents the mean of the actual values; S5.
2. The accuracy of the prediction model is measured by calculating the metric CORR that quantifies the correlation between the rolling prediction step sequence and the true value: Where: Represents the mean of the predicted values. The value range of CORR is (-1,1).
4. A multidimensional time series prediction system based on learning dynamic dependencies between variables, Features: The prediction system is used to execute the steps in the method according to any one of claims 1 to 3 above.
