Graph aggregation model-based multivariate time sequence prediction method, system and equipment, and medium

Through graph aggregation model and multi-head self-attention mechanism, the problems of time-sequence variable correlation expression and dynamic influence of multi-level graph structure in long-term multivariate time series prediction are solved, and higher prediction accuracy and deep modeling capabilities are achieved.

CN119025855AInactive Publication Date: 2024-11-26XIDIAN UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411086120.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-08
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing long-term multivariate time series prediction methods have insufficient in expressing the explicit correlation between time series sequence variables and the impact of dynamic multilevel graph structure on output, resulting in insufficient prediction accuracy and deep modeling capabilities.

Method used

Using a graph aggregation model-based method, the depth information between sequence variables is extracted through graph convolution, and the multi-head self-attention mechanism is used to dynamically adjust and aggregate the multi-level graph structure to build an end-to-end deep learning training network to update Graph adjacency matrix realizes correlation and deep dependency modeling of multi-level graph structures.

Benefits of technology

It effectively expresses the long-term dependence between time-series variables, improves prediction accuracy and deep modeling capabilities, solves the problems of excessive smoothing and local receptive field limitation, and improves the performance of multivariate timing prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025855A_ABST
    Figure CN119025855A_ABST
Patent Text Reader

Abstract

The invention discloses a multivariate time sequence prediction method, system and device based on a graph aggregation model and a medium, and the method comprises the steps: inputting a set time sequence data sample into a coding layer module, and capturing a time sequence data sample containing time dimension global information; carrying out preprocessing, and adjusting the information into sequence variable information; constructing a graph adjacency matrix reflecting a sequence variable relationship; taking the sequence variable information as an initial input feature matrix of a graph aggregation model GAM, performing graph convolution updating iteration on each feature representation in the initial input feature matrix by adopting a graph adjacency matrix, and obtaining a multi-level graph structure as depth information of correlation between variables; utilizing a multi-head self-attention layer in a graph aggregation model GAM to carry out dynamic weight distribution and aggregation on the prediction influence of each level of graph structure on the output sequence, and obtaining accurate prediction on the output sequence; the system, the equipment and the medium are used for implementing the method. The method has the advantages that the model prediction accuracy is enhanced, and the deep modeling capability of the long-term dependency relationship is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer data quantization, and specifically relates to a multivariate time series prediction method, system, device and medium based on a graph aggregation model. Background Art

[0002] In the rapidly developing IoT ecosystem, multivariate time series data has become ubiquitous, highlighting the fundamental role of time series prediction in numerous applications. The key challenge of long-term MTS prediction requires models that can capture both intra-sequence and inter-sequence dependencies. Recent advances in deep learning, especially the introduction of the Transformer mechanism and graph neural networks, have enabled significant progress in this area.

[0003] However, the accuracy of long-term MTS predictions still faces the following major challenges: 1. It is difficult to express explicit correlations between time series variables; 2. In current prediction methods based on graph convolutional networks, the multi-level graph structure used to represent the dependencies between these sequence variables has a dynamic impact on the output, that is, the effect of the multi-level graph convolution structure on the output varies between levels.

[0004] In order to solve these problems, the initial models of long-term MTS prediction methods focused on using the attention mechanism to obtain information from the time dimension and the variable dimension respectively. However, due to the information redundancy and long tail phenomenon of the attention mechanism, the model is not aware of the correlation between the sequences, which may lead to incorrect data prediction. Recently, researchers have proposed using graph convolutional networks to achieve long-term MTS prediction. Compared with traditional convolutional neural network methods, graph convolutional networks model the long-term dependencies between sequences as a graph structure, and the edges represent the relationship between nodes. The network can capture the spatial relationship between nodes and learn effective feature representations. The main advantage of using graph convolutional networks is that they can capture the dependencies between nodes. The change trend of each sequence is affected by the changes of the remaining sequences. The graph convolutional network can explicitly model these dependencies and model them into a graph structure, allowing the model to treat the relationship between each sequence as a connected system. Although graph convolutional networks have achieved good results in long-term MTS prediction, and the research hotspots of long-term MTS prediction are all based on the expansion of graph convolutional networks, there are still many shortcomings. First, for problem 1, graph convolutional networks that only use single-layer graph convolution information will lead to over-smoothing, and the learned adjacency matrix cannot effectively express the dependencies between nodes. Secondly, for problem 2, the graph structures at all levels without graph aggregation will be limited by the local receptive field of its iterator, and the impact of the multi-level graph convolution structure on the output results cannot be dynamically adjusted. Therefore, there is still a need to find efficient and simple methods to solve these problems in order to further improve the performance and accuracy of long-term MTS prediction.

[0005] The patent application document with publication number CN115630721A discloses a "trajectory prediction method and device based on potential graph structure generation", which collects the position and speed of the target to be predicted at any time; the position and speed at any time are input into a pre-trained graph convolutional neural network, but because the prediction method of this model has defects in target selection and graph structure construction, it cannot construct serial correlation well.

[0006] The patent application document with publication number CN115936108A discloses a "graph neural network compression method for multivariate time series prediction based on knowledge distillation", which includes processing the time convolution layer of the multivariate time series prediction model based on the graph neural network, and converting the expanded dilated convolution into a fixed dilated convolution. However, since the fixed dilated convolution does not aggregate the deep relationship, it cannot reflect the depth of the prediction network and the deep modeling ability, and the model is mainly biased towards the edge computing field. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a multivariate time series prediction method, system, device and medium based on a graph aggregation model, which utilizes graph convolution to extract deep information of the correlation between sequence variables, can effectively express the dependency relationship of the correlation between variables, and has the effect of enhancing the prediction accuracy of the model; secondly, the multi-head self-attention mechanism is used to dynamically adjust the weights of the multi-level graph structure and aggregate the output, which can effectively realize the association of the multi-level graph structure and improve the deep modeling ability of long-term dependencies.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A multivariate time series prediction method based on a graph aggregation model includes the following steps:

[0010] Step 1: Input the set time series data sample into the encoding layer module. The structure of the encoding layer module is a Transformer encoder structure, which specifically includes a self-attention layer and a forward feedback layer, and is used to capture the time series data sample containing global information of the time dimension;

[0011] Step 2: preprocess the time series data samples captured in step 1, and adjust the dimension of the input format of the time series data samples to variable-dominated sequence variable information;

[0012] Step 3, constructing an end-to-end deep learning training network, separately setting a set of learnable parameters in the end-to-end deep learning training network, and updating the learnable parameters in the reverse update process of the end-to-end deep learning training network, wherein the learnable parameters include a set of tensors, the length of which is the number of sequence variables, wherein the sequence variables are nodes in the graph, and based on the learnable parameters, constructing a graph adjacency matrix reflecting the relationship between the sequence variables;

[0013] Step 4: Use the sequence variable information preprocessed in step 2 as the initial input feature matrix of the graph aggregation model GAM, and use the graph adjacency matrix constructed in step 3 to perform multi-level graph convolution operations on each feature representation in the initial input feature matrix to obtain a multi-level graph structure as the deep information of the correlation between variables; use the multi-head self-attention layer in the graph aggregation model GAM to dynamically assign and aggregate the predictive effects of each level of graph structure on the output sequence to obtain an accurate prediction of the output sequence.

[0014] The specific process of step 3 is as follows:

[0015] Create an adjacency matrix based on the time series data sample information in step 1, and convert the edge and node information into an initial random matrix, where the row and column numbers of the matrix correspond to the ID numbers of each variable, and the element values ​​of the matrix indicate whether there is a connection between the corresponding variables. According to the connection of the edges, the adjacency matrix is ​​converted into a symmetric matrix to reflect the bidirectional connection between the variables; the overall equation of step 3 is as follows:

[0016] X1=GELU(MLP1(X0)) (4)

[0017] X2=GELU(MLP2(X0)) (5)

[0018] TPD(X1,X2)=X1X2 T -X2X1 T (6)

[0019] A=GELU(TPD(X1,X2)) (7)

[0020] Where X0 is a learnable parameter with random initial value, which participates in reverse update following network training. MLP1 and MLP2 represent fully connected layer networks with different initial parameters. Their main function is to expand feature representation. X1 and X2 are used to describe different unidirectional dependencies between nodes. Function TPD(X1,X2)=X1X2 T -X2X1 T It represents the transposed product difference between matrices, which is used to construct an adjacency matrix representing symmetry, and finally outputs the result as the graph adjacency matrix A through the GELU activation function.

[0021] The specific process of step 4 is as follows:

[0022] In step 4, the graph aggregation model GAM consists of a three-layer network architecture, including an input layer, a graph convolution layer, and an output layer; the input layer includes a fully connected layer and a GELU activation function; the graph convolution layer includes multi-level graph convolution and a GELU activation function; the output layer includes a dimensional connection layer and a multi-head self-attention layer;

[0023] The graph aggregation model GAM implements the update iteration of the graph convolutional neural network GCN through the graph adjacency matrix constructed in step 3, and updates each feature representation in the graph. The specific implementation steps of step 4 are:

[0024] Step 4.1, input layer: In the input layer, multiply the sequence variable information obtained in step 2 by the learnable weight matrix in the fully connected layer to expand the feature representation dimension, and then use the GELU activation function to obtain its nonlinear features to obtain the output E of this layer;

[0025] Step 4.2, graph convolution layer: First, the graph adjacency matrix A obtained in step 3 is exponentially multiplied by N, that is, A N , and then perform matrix multiplication operation (⊙) with the output E of this layer obtained in step 4.1 to implement multi-level graph convolution operation, that is, A N ⊙E,N=1, 2, 3, ..., where N represents the number of graph convolution levels; secondly, the obtained multi-level graph structure results are processed by the standard processing method and GELU activation function to obtain the result F N The specific operation of the standard processing is: normalize the output of each feature channel; list the associated multi-level graph structure, that is, the matrix list List(F):

[0026] F N =Normal(GELU(A N ⊙E)) (8)

[0027] List(F)=List{F1,F2,F3,…} (9)

[0028] In the formula, Normal means to perform normal processing operations, and F1, F2, F3, ... represent the association of multi-level graph structures;

[0029] Step 4.3, output layer: The matrix list List(F) obtained in step 4.2 is passed through the dimension connection layer to obtain a high-dimensional matrix The dimension connection layer is used to concatenate all elements of the matrix list List(F), and assign weights through multiple self-attention to finally obtain the predicted output Output; this process can be expressed as:

[0030]

[0031] In the formula, Concat represents the dimension connection operation.

[0032] The present invention also provides a multivariate time series prediction system based on a graph aggregation model, comprising:

[0033] A time series data sample acquisition module is used to input the set time series data samples into the encoding layer module. The structure of the encoding layer module is a Transformer encoder structure, and the specific structure includes a self-attention layer and a forward feedback layer, which is used to capture time series data samples containing global information of the time dimension;

[0034] The information preprocessing module of the sequence variable is used to preprocess the captured time series data samples and adjust the dimension of the input format of the time series data samples to the sequence variable information dominated by the variables;

[0035] A graph adjacency matrix construction module is used to construct an end-to-end deep learning training network, set a set of learnable parameters separately in the end-to-end deep learning training network, and update the learnable parameters in the reverse update process of the end-to-end deep learning training network, wherein the learnable parameters include a set of tensors, the length of which is the number of sequence variables, wherein the sequence variables are nodes in the graph, and a graph adjacency matrix reflecting the relationship between the sequence variables is constructed based on the learnable parameters;

[0036] The multivariate time series prediction module is used to use the preprocessed sequence variable information as the initial input feature matrix of the graph aggregation model GAM, and use the constructed graph adjacency matrix to perform multi-level graph convolution operations on each feature representation in the initial input feature matrix to obtain the multi-level graph structure as the deep information of the correlation between variables; using the multi-head self-attention layer in the graph aggregation model GAM, the prediction impact of each level of graph structure on the output sequence is dynamically weighted and aggregated to obtain an accurate prediction of the output sequence.

[0037] The present invention also provides a multivariate time series prediction device based on a graph aggregation model, comprising:

[0038] Memory: a computer program storing the above-mentioned multivariate time series prediction method based on graph aggregation model, which is a computer-readable device;

[0039] Processor: used to implement the multivariate time series prediction method based on the graph aggregation model when executing the computer program.

[0040] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the multivariate time series prediction method based on the graph aggregation model.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The present invention combines the basic principle of multi-level graph convolution and the characteristics of end-to-end learning and updating of adjacency matrix through Markov chain multi-step prediction of medium- and long-term dependency effects. Through this combination, it can effectively solve the problem of expressing explicit correlation between sequence variables in time series long-term prediction.

[0043] (2) The present invention obtains a multi-level graph structure and associates and aggregates the multi-level graph structure in combination with a multi-head self-attention mechanism, thereby effectively solving the problem of the dynamic changes in the impact of the multi-level graph structure on the output during the graph aggregation process, and improving the deep modeling capability of the long-term dependency relationship of variables in the prediction.

[0044] In summary, the present invention uses graph convolution to extract the deep information of the correlation between sequence variables, which can effectively express long-term dependencies and enhance the accuracy of model prediction; the multi-head self-attention mechanism is used to dynamically adjust the weights of multi-level graph convolution structures, which can effectively realize the association of multi-level graph structures and improve the deep modeling ability of long-term dependencies. Different from previous methods, the present invention is a novel multivariate time series prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a multivariate time series long-term prediction method based on a graph aggregation model GAM in a specific implementation of the present invention.

[0046] Figure 2 It is a multi-level graph convolution operation in the graph aggregation model GAM of the present invention.

[0047] Figure 3 It is a graph aggregation module GAM constructed by the present invention, which includes an input layer, a graph convolution layer and an output layer. DETAILED DESCRIPTION

[0048] The specific use method of the present invention is described below in conjunction with the accompanying drawings so that users can better understand the present invention. It should be noted that in the following description, the detailed description of the existing method will be downplayed. The present invention mainly focuses on multivariate time series long-term prediction.

[0049] The present invention explicitly emphasizes the importance of dependencies between sequence variables. As a Transformer model enhanced by the graph aggregation model GAM, the present invention uses graph structures to skillfully identify and model complex relationships between series variables, and can dynamically adjust the impact of multi-level graph structures on output. Figure 1-Figure 3 Based on this method, a prediction network model is proposed to be built for long-term MTS prediction.

[0050] See also Figure 1, a multivariate time series prediction method based on a graph aggregation model, comprising the following steps:

[0051] Step 1: Input the set time series data sample into the coding layer module. The structure of the coding layer module is specifically set to the Transformer encoder structure in the current traditional time series prediction, that is, this module belongs to the existing method. The module operation specifically includes:

[0052] Step 1.1, use the self-attention layer to aggregate the global information of the time dimension to capture the global information of the time dimension in the data sample;

[0053] In step 1.2, the global information is further captured through the forward feedback layer network.

[0054] Step 2: preprocess the time series data samples captured in step 1, and adjust the dimension of the input format of the time series data samples to variable-dominated sequence variable information;

[0055] The specific operation is: extract all variables in the time series data sample, uniformly adjust the input format of all variables, and convert them into information based on the sequence variable dimension, and use this information as the input of the graph aggregation model GAM.

[0056] Step 3, construct an end-to-end deep learning training network, set a set of learnable parameters separately in the end-to-end deep learning training network, and update the set of learnable parameters in the reverse update process of the end-to-end deep learning training network, the learnable parameters include a set of tensors, the length of which is the number of sequence variables, wherein the sequence variables are nodes in the graph, and a graph adjacency matrix reflecting the relationship between the sequence variables is constructed based on the learnable parameters;

[0057] Furthermore, the specific process of step 3 is as follows:

[0058] In step 3, the graph adjacency matrix is ​​constructed using learnable parameters based on the time series data sample information in step 1. The edge and node information is converted into an initial random matrix, where the row and column numbers of the matrix correspond to the ID numbers of each variable, and the element values ​​of the matrix indicate whether there is a connection between the corresponding variables. According to the connection of the edges, the adjacency matrix is ​​converted into a symmetric matrix to reflect the bidirectional connection between the variables. Step 3 is described by the following equation:

[0059] X1=GELU(MLP1(X0)) (4)

[0060] X2=GELU(MLP2(X0)) (5)

[0061] TPD(X1,X2)=X1X2 T -X2X1 T(6)

[0062] A=GELU(TPD(X1,X2)) (7)

[0063] Where X0 is a set of learnable parameters with random initial values, which participate in reverse update following network training. MLP1 and MLP2 represent fully connected layer networks with different initial parameters. Their main function is to expand feature representation. X1 and X2 are used to describe different types of unidirectional dependencies between nodes. Function TPD(X1,X2)=X1X2 T -X2X1 T Represents the transposed product difference between matrices. This function is used to construct a graph adjacency matrix representing symmetry, and finally outputs the result as the graph adjacency matrix A through the GELU activation function.

[0064] See also Figure 2 , step 4, use the sequence variable information preprocessed in step 2 as the initial input feature matrix of the graph aggregation model GAM, and use the graph adjacency matrix constructed in step 3 to perform multi-level graph convolution operations on each feature representation in the initial input feature matrix to obtain a multi-level graph structure as the depth information of the correlation between variables; use the multi-head self-attention layer in the graph aggregation model GAM to dynamically assign and aggregate the predicted influence of each level of graph structure on the output sequence to obtain an accurate prediction of the output sequence.

[0065] See also Figure 3 ,Further, the specific process of step 4 is as follows:

[0066] In step 4, the graph aggregation model GAM consists of a three-layer network architecture, including an input layer, a graph convolution layer, and an output layer; the input layer includes a fully connected layer and a GELU activation function layer; the graph convolution layer includes a multi-level graph convolution operation and a GELU activation function layer; the output layer includes a dimension connection layer, a multi-head self-attention layer, a fully connected layer, and a GELU activation function layer;

[0067] In the present invention, the graph aggregation model GAM realizes multi-level deep graph convolution connection based on the Markov chain multi-step prediction of medium- and long-term dependency effects, thereby improving the prediction model's deep modeling ability for long-term dependencies; the role of the standardization process is to prevent excessive smoothing in aggregation through normalization operations. The graph adjacency matrix constructed in step 3 in the graph aggregation model GAM realizes the update iteration of the graph convolutional neural network GCN, and updates each feature representation in the graph. The specific implementation steps of step 4 are:

[0068] Step 4.1, input layer: In the input layer, multiply the sequence variable information obtained in step 2 with the learnable weight matrix in the fully connected layer to expand the feature representation dimension, and then use the GELU activation function to obtain its nonlinear features. Get the output E of this layer;

[0069] Step 4.2, graph convolution layer: First, the graph adjacency matrix A obtained in step 3 is exponentially multiplied by N, that is, A N , and then perform matrix multiplication operation (⊙) with the output E of this layer obtained in step 4.1 to implement multi-level graph convolution operation, that is, A N ⊙E,N=1,2,3,…, where N represents the number of graph convolution levels. The purpose of this process is to obtain a multi-level graph structure. Secondly, the obtained multi-level graph structure result is processed by standardization and GELU activation function to obtain the result F N The specific operation of the standard processing is: reduce the internal covariate shift by normalizing the output of each feature channel. Finally, the associated multi-level graph structure is listed, that is, the matrix list List(F):

[0070] F N =Normal(GELU(A N ⊙E)) (8)

[0071] List(F)=List{F1,F2,F3,…} (9)

[0072] In the formula, Normal means to perform normal processing operations, and F1, F2, F3, ... represent the association of multi-level graph structures;

[0073] Step 4.3, output layer: The matrix list List(F) obtained in step 4.2 is passed through the dimension connection layer to obtain a high-dimensional matrix The dimension connection layer is mainly used to concatenate all elements of the matrix list List(F) and assign weights through multiple self-attention. Multiple self-attention is mainly used to adjust the input weights that affect the output results, and finally obtain the predicted output Output. The process can be expressed as:

[0074]

[0075] In the formula, Concat represents the dimension connection operation.

[0076] The above steps use graph convolution to update the representation of each feature in the graph. In the graph aggregation model GAM, multiple cycles of iterative updates are implemented, and the optimal value is continuously sought through deep learning training.

[0077] After the experiment, after the validation set of the data set was used for validation (Validate, Vali), the data set was tested, and the mean absolute error (MAE) and mean square error (MSE) between the test value and the true value were calculated. As shown in Table 1, each row of data is the comparison result of different methods. The input length in the validation experiment parameters is set to 96, the output prediction length is 336, and the evaluation indicators of the average prediction result, the mean absolute error MAE and the mean square error MSE, are taken. The smaller these values, the better the effect. After comparison with the previous method, the effectiveness and accuracy of the present invention are confirmed.

[0078] In the experimental results comparison in the table below, each row of data is the mean absolute error (MAE) and mean square error (MSE) of different methods under prediction. The smaller the result values ​​of the two indicators, the better the effect.

[0079] Data in the table Underline Indicates suboptimal results, and bold indicates optimal results.

[0080]

[0081]

[0082] Experimental Results

[0083] Specifically, compared with the current multivariate time series long-term prediction methods, such as Sageformer, Crossformer, MTGCN, PatchTST, Transformer, Dlinear, Informer, etc., the present invention can achieve better prediction effect and accuracy. In terms of mean absolute error MAE and mean square error MSE, the prediction results have smaller errors with the true values.

[0084] The present invention proposes a multivariate time series prediction method, system, device and medium based on a graph aggregation model. As a model of the present invention, the graph aggregation model GAM, firstly, combines the learnable characteristics of the adjacency matrix in the multivariate variable relationship and the basic principle of updating and iterating the multi-level graph structure, and uses a multi-level graph convolutional neural network to extract the deep information of the correlation, thereby solving the problem that the correlation between multivariate sequence variables is difficult to express. Secondly, the model of the present invention introduces a multi-head attention mechanism to adjust the output weight distribution of each level of the graph structure obtained by the multi-level graph convolution in the graph aggregation process, which helps the model to dynamically adjust and aggregate the relationship between the graph structures at all levels, so that the model can show the understanding and dynamic processing capabilities of the dependency between sequence data variables. After experimental verification, compared with most multivariate time series long-term prediction methods, this method can achieve better results, and the estimation results have a smaller error with the true value.

[0085] The present invention also provides a multivariate time series prediction system based on a graph aggregation model, comprising:

[0086] A time series data sample acquisition module is used to input the set time series data samples into the encoding layer module in step 1. The structure of the encoding layer module is a Transformer encoder structure, and the specific structure includes a self-attention layer and a forward feedback layer, which is used to capture time series data samples containing global information of the time dimension;

[0087] The information preprocessing module of the sequence variable is used to implement the preprocessing of the captured time series data samples in step 2, and adjust the dimension of the input format of the time series data samples to the sequence variable information dominated by the variables;

[0088] A graph adjacency matrix construction module is used to implement the construction of an end-to-end deep learning training network in step 3, to set a set of learnable parameters separately in the end-to-end deep learning training network, and to update the learnable parameters in the reverse update process of the end-to-end deep learning training network, wherein the learnable parameters include a set of tensors, the length of which is the number of sequence variables, wherein the sequence variables are nodes in the graph, and based on the learnable parameters, to construct a graph adjacency matrix reflecting the relationship between the sequence variables;

[0089] The multivariate time series prediction module is used to implement the sequence variable information preprocessed in step 2 as the initial input feature matrix of the graph aggregation model GAM in step 4, and use the graph adjacency matrix constructed in step 3 to perform multi-level graph convolution operations on each feature representation in the initial input feature matrix to obtain a multi-level graph structure as the deep information of the correlation between variables; using the multi-head self-attention layer in the graph aggregation model GAM, the prediction impact of each level of graph structure on the output sequence is dynamically weighted and aggregated to obtain an accurate prediction of the output sequence.

[0090] The present invention also provides a multivariate time series prediction device based on a graph aggregation model, comprising:

[0091] Memory: a computer program storing the above-mentioned multivariate time series prediction method based on graph aggregation model, which is a computer-readable device;

[0092] Processor: used to implement the multivariate time series prediction method based on the graph aggregation model when executing the computer program.

[0093] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the multivariate time series prediction method based on the graph aggregation model.

[0094] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A multivariate time series prediction method based on a graph aggregation model, characterized in that: The following steps are involved: Step 1: Input the set time series data sample into the encoding layer module. The structure of the encoding layer module is a Transformer encoder structure, which specifically includes a self-attention layer and a forward feedback layer, and is used to capture the time series data sample containing global information of the time dimension; Step 2: preprocess the time series data samples captured in step 1, and adjust the dimension of the input format of the time series data samples to variable-dominated sequence variable information; Step 3, constructing an end-to-end deep learning training network, separately setting a set of learnable parameters in the end-to-end deep learning training network, and updating the learnable parameters in the reverse update process of the end-to-end deep learning training network, wherein the learnable parameters include a set of tensors, the length of which is the number of sequence variables, wherein the sequence variables are nodes in the graph, and based on the learnable parameters, constructing a graph adjacency matrix reflecting the relationship between the sequence variables; Step 4: Use the sequence variable information preprocessed in step 2 as the initial input feature matrix of the graph aggregation model GAM, and use the graph adjacency matrix constructed in step 3 to perform multi-level graph convolution operations on each feature representation in the initial input feature matrix to obtain a multi-level graph structure as the deep information of the correlation between variables; use the multi-head self-attention layer in the graph aggregation model GAM to dynamically assign and aggregate the predictive effects of each level of graph structure on the output sequence to obtain an accurate prediction of the output sequence.

2. According to claim 1, a multivariate time series prediction method based on a graph aggregation model is characterized in that: The specific process of step 3 is as follows: Create an adjacency matrix based on the time series data sample information in step 1, and convert the edge and node information into an initial random matrix, where the row and column numbers of the matrix correspond to the ID numbers of each variable, and the element values ​​of the matrix indicate whether there is a connection between the corresponding variables. According to the connection of the edges, the adjacency matrix is ​​converted into a symmetric matrix to reflect the bidirectional connection between the variables; the overall equation of step 3 is as follows: X1=GELU(MLP1(X0)) (4) X2=GELU(MLP2(X0)) (5) TPD(X1,X2)=X1X2 T -X2X1 T (6) A=GELU(TPD(X1,X2)) (7) Where X0 is a learnable parameter with random initial value, which participates in reverse update following network training. MLP1 and MLP2 represent fully connected layer networks with different initial parameters. Their main function is to expand feature representation. X1 and X2 are used to describe different unidirectional dependencies between nodes. Function TPD(X1,X2)=X1X2 T -X2X1 T It represents the transposed product difference between matrices, which is used to construct an adjacency matrix representing symmetry, and finally outputs the result as the graph adjacency matrix A through the GELU activation function.

3. A multivariate time series prediction method based on a graph aggregation model according to claim 1 or 2, characterized in that: The specific process of step 4 is as follows: In step 4, the graph aggregation model GAM consists of a three-layer network architecture, including an input layer, a graph convolution layer, and an output layer; the input layer includes a fully connected layer and a GELU activation function; the graph convolution layer includes a multi-level graph convolution operation and a GELU activation function; the output layer includes a dimensional connection layer and a multi-head self-attention layer; The graph aggregation model GAM implements the update iteration of the graph convolutional neural network GCN through the graph adjacency matrix constructed in step 3, and updates each feature representation in the graph. The specific implementation steps of step 4 are: Step 4.1, input layer: In the input layer, multiply the sequence variable information obtained in step 2 with the learnable weight matrix in the fully connected layer to expand the feature representation dimension, and then use the GELU activation function to obtain its nonlinear features. Get the output E of this layer; Step 4.2, graph convolution layer: First, the graph adjacency matrix A obtained in step 3 is exponentially multiplied by N, that is, A N , and then perform matrix multiplication operation (⊙) with the output E of this layer obtained in step 4.1 to implement multi-level graph convolution operation, that is, A N ⊙E,N=1, 2, 3, ..., where N represents the number of graph convolution levels; secondly, the obtained multi-level graph structure results are processed by the standard processing method and GELU activation function to obtain the result F N The specific operation of the standard processing is: normalize the output of each feature channel; list the associated multi-level graph structure, that is, the matrix list List(F): F N =Normal(GELU(A N ⊙E)) (8) List(F)=List{F1,F2,F3,…} (9) In the formula, Normal means to perform normal processing operations, and F1, F2, F3, ... represent the association of multi-level graph structures; Step 4.3, output layer: The matrix list List(F) obtained in step 4.2 is passed through the dimension connection layer to obtain a high-dimensional matrix The dimension connection layer is used to concatenate all elements of the matrix list List(F) and assign weights through multiple self-attention to finally obtain the predicted output Output; this process can be expressed as: In the formula, Concat represents the dimension connection operation.

4. A multivariate time series prediction system based on a graph aggregation model based on the method according to any one of claims 1 to 3, characterized in that: include: A time series data sample acquisition module is used to input the set time series data samples into the encoding layer module. The structure of the encoding layer module is a Transformer encoder structure, and the specific structure includes a self-attention layer and a forward feedback layer, which is used to capture time series data samples containing global information of the time dimension; The information preprocessing module of the sequence variable is used to preprocess the captured time series data samples and adjust the dimension of the input format of the time series data samples to the sequence variable information dominated by the variables; A graph adjacency matrix construction module is used to construct an end-to-end deep learning training network, set a set of learnable parameters separately in the end-to-end deep learning training network, and update the learnable parameters in the reverse update process of the end-to-end deep learning training network, wherein the learnable parameters include a set of tensors, the length of which is the number of sequence variables, wherein the sequence variables are nodes in the graph, and a graph adjacency matrix reflecting the relationship between the sequence variables is constructed based on the learnable parameters; The multivariate time series prediction module is used to use the preprocessed sequence variable information as the initial input feature matrix of the graph aggregation model GAM, and use the constructed graph adjacency matrix to perform multi-level graph convolution operations on each feature representation in the initial input feature matrix to obtain the multi-level graph structure as the deep information of the correlation between variables; using the multi-head self-attention layer in the graph aggregation model GAM, the prediction impact of each level of graph structure on the output sequence is dynamically weighted and aggregated to obtain an accurate prediction of the output sequence.

5. A multivariate time series prediction device based on a graph aggregation model, characterized in that: include: Memory: a computer program storing a multivariate time series prediction method based on a graph aggregation model as described in any one of claims 1 to 3, which is a computer-readable device; Processor: used to implement a multivariate time series prediction method based on a graph aggregation model as described in any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement a multivariate time series prediction method based on a graph aggregation model as described in any one of claims 1 to 3.

Citation Information

Cited By

  • Project quality prediction method based on process variable

    CN119272015A

  • Time sequence prediction method and system based on dynamic hypergraph and multi-scale coding

    CN120278037A