Time sequence prediction system based on causal relationship

Abstract

The invention belongs to the field of data mining in deep learning, and particularly relates to a time series prediction system based on a causal relationship. The time series prediction method comprises a causal inference and enhancement algorithm suitable for time series data; a causal-frequency domain attention module based on a graph attention mechanism and Fourier transform; the invention discloses a decoder based on a multi-layer perceptron. According to the method, the modeling capability of the existing multivariable time sequence prediction method on the dependency relationship between the sequence variables and the feature extraction capability on the long-time sequence data can be improved, and the error of the prediction result is effectively reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of time series analysis, and in particular relates to a time series prediction system. Background Art

[0002] Time series prediction technology has been widely used in financial investment, traffic monitoring, weather forecasting, power supply and other fields, and its results have important reference value for various decisions in these fields. As a popular research direction in the fields of data mining, big data analysis, artificial intelligence, etc., researchers have conducted extensive research on this issue and proposed a large number of theoretical models and prediction methods. Among them, time series prediction technology based on neural networks is one of the hot research topics in this field. However, existing neural network models still have many problems in this task. For example, most neural network models usually focus on how to model the temporal dependencies in time series, while ignoring the dependencies between various variables in the time series. In addition, most existing methods focus on mining the temporal features of time series to model time series, but often ignore the frequency domain features of time series. Therefore, it is urgent to design a deep learning time series prediction method that integrates causal inference and frequency domain analysis to achieve accurate prediction of time series data. Summary of the invention

[0003] The purpose of the present invention is to provide a time series prediction system based on causality, so as to effectively improve the modeling ability between time series variables and the frequency domain feature extraction ability, so as to improve the accuracy of time series prediction in real application scenarios.

[0004] The causal relationship-based time series prediction system provided by the present invention uses a variety of new technical means, including: dynamically mining the causal relationship between time series variables to provide a basis for modeling the relationship between time series variables; constructing a causal-frequency domain attention mechanism based on a neural network; a decoder module based on a multi-layer perceptron; specifically including:

[0005] (1) A dynamic causal relationship inference module for time series, which is used to infer the dynamic causal graph of each variable from the sampled data. It includes a greedy equivalent search algorithm module based on cross-validation log-likelihood score (Huang B, Zhang K, Lin Y, et al. Generalized score functions for causal discovery [C] / / Proceedings of the 24th ACM SIGKDD international conference on knowledgediscovery & data mining. 2018: 1551-1560.), a dynamic causal inference algorithm module based on neural network, and an adaptive causal graph fusion network module;

[0006] (2) Causal-frequency domain attention mechanism module; the specific package extracts the causal-frequency domain features of the time series historical data and encodes the features into a set of feature vectors;

[0007] (3) A decoder module based on a multi-layer perceptron is used to decode the causal-frequency domain features output by the causal-frequency domain attention mechanism to obtain the final prediction result.

[0008] Further:

[0009] (1) In the dynamic causal relationship inference module:

[0010] (1.1) The greedy equivalent search algorithm module based on cross-validation log-likelihood score is as follows:

[0011] For a directed acyclic graph The set of all its equivalent graphs is called Call a set A directed acyclic graph The set of one-edge equivalent graphs of is if and only if for any directed acyclic graph There is a graph Make Is Add an edge to get. Similarly, call a set A directed acyclic graph The set of first-order edge-reduced equivalent graphs of is if and only if for any directed acyclic graph There is a graph Make Is Obtained by deleting an edge. Based on the above definition, the greedy equivalent search algorithm can be summarized as:

[0012] The input of the algorithm is all samples of the time series At initialization, the algorithm constructs a directed acyclic graph with an empty edge set based on the variables in the time series. Afterwards, in the forward search phase, the algorithm will repeatedly Replace with The set of one-edge-added equivalent graphs The algorithm then searches for the graph with the highest score in the search until there is no graph with a higher score than the current graph. Replace with The set of equivalent graphs with reduced edges The graph with the highest median likelihood score is selected until there is no graph with a higher score than the current graph. This is the final causal diagram.

[0013] The cross-validation log-likelihood score is calculated as follows:

[0014] Suppose there is a time series consisting of n variables And the directed acyclic graph composed of these variables The sequence is sampled m times from time 1 to time m to obtain m samples, and the set of these samples is recorded as where x i =[x i (1),x i (2),…,x i (m)] T is the set of samples of the i-th variable. The kernel matrix of the sample set xi is denoted as K(x i ),Right now:

[0015]

[0016] Among them, κ(·,·) is the kernel function. The kernel function used in this algorithm is the Gaussian kernel function. For two vectors or scalars y and y′, it is defined as:

[0017]

[0018] make:

[0019]

[0020] Where I is the n×n identity matrix, 1 is an n-dimensional all-1 vector, then K(x i ) corresponds to the centralization matrix for:

[0021]

[0022] When doing cross validation, you need to Divide into a training set and a test set, a total of Q times. The number of samples in the training set is m tr , the number of samples in the test set is m te , and m tr +m te =m. The training set divided for the qth time is recorded as Test set is denoted as Further, the variable x in the qth division i The training set and test set of its father variable are denoted as and Where the subscript te corresponds to the training set, the subscript tr corresponds to the test set; Based on the above definition, the directed acyclic graph The cross-validated log-likelihood score of can be expressed as:

[0023]

[0024] in, is about a single variable x i and its parent variable, the cross-validated log-likelihood score, which is estimated by changing x i is the target, its parent variable The feature is K-fold cross validation (Stone M. Cross-validatory choice and assessment of statistical predictions [J]. Journal of the royal statistical society: Series B (Methodological), 1974, 36 (2): 111-133.). Its specific definition is:

[0025]

[0026] in, The calculation formula is:

[0027]

[0028] Formulas (5), (6) and (7) are the mathematical representations of the cross-validated log-likelihood scores for time series.

[0029] (1.2) The dynamic causal inference algorithm module based on the neural network, wherein the neural network is a local causal inference network whose input is a time series The sampling at t time points before time T outputs m local causal graphs. Figure 2The structural diagram of the local causal inference network is given. The network as a whole can be divided into three sub-blocks: preprocessing module, temporal feature extraction module and graph structure generation module. Among them:

[0030] The preprocessing module is mainly used to pre-extract features from the input time series data and evenly divide the extracted features into m subsequences in the time dimension for subsequent generation of local causal graphs. The module consists of three layers of one-dimensional atrous convolutional networks activated by the ReLU (·) function.

[0031] For the input sequence Each subsequence in The preprocessing module maps it into a feature vector After obtaining the eigenvectors of all subsequences {γ1,γ2,…,γ n}, the preprocessing module first concatenates these vectors in the feature dimension to obtain a feature matrix Then, the module divides the feature matrix Γ into m sub-matrices in the time dimension as the input of the temporal feature extraction module, namely:

[0032]

[0033] The time series feature extraction module is composed of a layer of long short-term memory network, which is used to extract the time series feature vectors of the time series in different time periods. (i) is the input and outputs a dimension of The time series feature vector Δ (i) The initial state of each gate in the long short-term memory network is 0, and each sub-matrix is ​​Γ (i) After input into the long short-term memory network, its output is the time series feature vector Δ (i) After the LSTM network processes a submatrix, its internal hidden state will be reset to 0 to ensure the independence of each submatrix.

[0034] The graph structure generation module is used to compress the output of the temporal feature extraction module and generate a local causal graph. The module is composed of two parallel multi-layer perceptrons, which are used to generate the probability matrix of the local causal graph. and the mask matrix The final local causal graph is obtained by element-wise multiplication of the probability matrix and the mask matrix, namely:

[0035]

[0036] in, It is the adjacency matrix representation of the i-th local causal graph. Whether the i-th row and j-th column of the adjacency matrix is ​​not zero represents whether there is a directed edge from the i-th node to the j-th node. MLP p (·) is a multilayer perceptron used to generate probability matrices. Its internal neurons are activated by the ReLU(·) function, and the last layer of neurons is activated by the Sigmoid(·) function. MLP m (·) is a multilayer perceptron used to generate the mask matrix. “⊙” is a matrix element multiplication operator, which is used to perform element-by-element multiplication of two matrices of the same dimension.

[0037] (1.3) The adaptive causal graph fusion network, whose input is the global causal graph and m local causal graphs The output is m enhanced causal graphs The network consists of two parallel submodules: probability matrix generation submodule and mask matrix generation submodule. The probability matrix generation submodule consists of four layers of convolutional neural network and one fully connected layer. The input of the first layer of convolutional neural network is the adjacency matrix of the global causal graph. and the adjacency matrix of the i-th local causal graph The concatenated dimensions are Tensor After four convolutional layers, the tensor has a dimension of Tensor The fully connected layer compresses the last dimension of the tensor from 128 to 1 and activates it with the Sigmoid(·) function to obtain the probability matrix. Similar to the probability matrix generation submodule, the mask matrix generation submodule also consists of a four-layer convolutional neural network and a fully connected layer. Its input is the Laplacian matrix of the global causal graph. and the Laplacian matrix of the i-th local causal graph { The concatenated dimensions are Tensor The output dimension is The mask matrix Except that the fully connected layer uses the ReLU(·) function as the activation function, the data flow path of this tensor in the mask matrix generation module is the same as that in the probability matrix generation module, which will not be repeated here. Finally, the Laplacian matrix of the enhanced causal graph is It is obtained by doing Hadamard product of probability matrix and mask matrix, that is:

[0038]

[0039] These m enhanced causal graphs will then be input into the corresponding causal-frequency domain attention mechanism heads for modeling time series.

[0040] The causal-frequency attention mechanism module described in (2) has the following structure: Figure 3 As shown. For a single attention head, its input is a time series Samples within t time points before time T and the lth enhanced causal graph The output is each subsequence x i The set of spectral feature vectors of (T-t+1:T) in, is a subsequence x i The eigenvector corresponding to (T-t+1:T).

[0041] In order to obtain the spectral feature vectors of each subsequence, in the causal-frequency domain attention mechanism, the historical sampling data x of each subsequence is first transformed by real fast Fourier transform. i (T-t+1:T) is transformed from the time domain to the frequency domain f i (T-t+1:T), that is:

[0042]

[0043] in, Stands for Real Fast Fourier Transform algorithm.

[0044] After transforming the sequence from the time domain to the frequency domain, the causal-frequency attention mechanism uses a feature extraction module consisting of three layers of one-dimensional convolution and one fully connected layer. To extract the spectral features

[0045]

[0046] And an attention coefficient To represent the subsequence x j The historical value of x i (T-t+1:T) for the subsequence x i The future value of x i (T-t+1:T) is important. In calculating the attention coefficient Previously, the causal-frequency attention mechanism firstly extracts spectral features through two modules. and x i (T-t+1:T) and x j (T-t+1:T) is mapped into a pair of eigenvectors and

[0047]

[0048] After obtaining the query-key vector pair, the causal-frequency domain attention mechanism calculates the attention coefficient by the following formula

[0049]

[0050] Remember C (l)′ for:

[0051]

[0052] and C (l) for:

[0053]

[0054] but:

[0055]

[0056] Among them, l is the number of the attention head, is the Laplace matrix of the enhanced causal graph corresponding to the lth attention head. And the frequency domain eigenvector After that, the causal-frequency attention mechanism calculates the subsequence x by weighted summation. i The spectral feature vector of Right now:

[0057]

[0058] For the multi-head attention mechanism, the causal-frequency attention mechanism will focus each attention head on the sequence x i The outputs of (T-t+1:T) are concatenated along the feature dimension to obtain the final feature H i :

[0059]

[0060] The output of the causal-frequency domain attention mechanism is the set of spectral feature vectors of each subsequence:

[0061]

[0062] (3) The decoder module described above uses the feature vector H generated by the causal-frequency domain attention network i As input, a multi-layer perceptron MLP(·) is used to All feature vectors in are decoded to produce time series The spectrum from T+1 to T+τ is used, and the inverse real Fourier transform is used to obtain the final prediction result, that is:

[0063] f i (T+1:T+τ)=MLP(H i ) (twenty four)

[0064]

[0065] The present invention designs a multi-layer perceptron with two fully connected layers as a decoder, and its structure is as follows: Figure 4 As shown in Figure 2, the first layer of the multi-layer perceptron uses ReLU (·) as the activation function to transform the feature vector The feature dimension is changed from m×d v The compression is 1. The second layer of multi-layer perceptron decodes the compressed feature vector and transforms it from the frequency domain back to the time domain to obtain the final prediction result.

[0066] The present invention has at least the following beneficial effects:

[0067] The present invention provides a time series prediction system based on causality and Fourier transform, which can effectively mine the dynamic causal relationship and frequency domain characteristics between time series variables, and make up for the shortcomings and technical limitations of existing time series prediction methods in inter-sequence causal modeling and spectrum analysis. The present invention first uses a dynamic causal inference module to dynamically model the causal relationship in the time series. Then, the graph attention mechanism is improved, Fourier transform is introduced into it, and a causal-frequency domain attention mechanism model is proposed to effectively mine the spectral characteristics of the time series and realize accurate modeling of time series data. Finally, a decoder based on a multi-layer perceptron is designed to realize accurate prediction of the time series by decoding the output of the causal-frequency domain attention mechanism. The present invention effectively improves the prediction accuracy and the interpretability of the prediction process in the long-term time series prediction scenario, and has a high practical application value. At the same time, the present invention has good portability and versatility, and can quickly migrate from one application scenario to a new application scenario.

[0068] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a framework diagram of the present invention.

[0070] Figure 2 Schematic diagram of the causal inference and enhancement module.

[0071] Figure 3 Schematic diagram of the multi-head causal-frequency domain attention mechanism.

[0072] Figure 4 A schematic diagram of the decoder. DETAILED DESCRIPTION

[0073] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0074] like Figure 1 As shown, an embodiment of the present invention provides a time series prediction method based on causality and Fourier transform, which includes three important components: a causal inference and enhancement algorithm suitable for time series data; a causal-frequency domain attention module based on graph attention mechanism and Fourier transform; a causal-frequency domain feature extraction module; a decoder based on a multi-layer perceptron; and a causal-time series feature decoding module output by the causal-frequency domain attention mechanism to obtain the final prediction result.

[0075] like Figure 2 As shown, the dynamic causal inference module uses time series All sampled values ​​are used as input to mine the causal relationship between variables and generate the corresponding dynamic causal graph This module includes a global static causal inference algorithm and a local dynamic causal inference module. The global static causal inference algorithm is a greedy equivalent search algorithm based on cross-validation log-likelihood scores. The local dynamic causal inference module consists of multiple layers of fully connected layers, convolutional layers, and long short-term memory network layers, which are used to generate local dynamic causal graphs.

[0076] like Figure 3 As shown, the structure of the causal-frequency domain attention mechanism is as follows Figure 3 As shown. For a single attention head, its input is a time series Samples within t time points before time T and the lth enhanced causal graph The output is each subsequence x i The set of spectral feature vectors of (T-t+1:T) in, is a subsequence x i The eigenvector corresponding to (T-t+1:T).

[0077] In order to obtain the spectral feature vector of each subsequence, the causal-frequency domain attention mechanism first transforms the historical sampling data x of each subsequence into i (T-t+1:T) is transformed from the time domain to the frequency domain f i (T-t+1:T), that is:

[0078]

[0079] in, Stands for Real Fast Fourier Transform algorithm.

[0080] After transforming the sequence from the time domain to the frequency domain, the causal-frequency attention mechanism uses a feature extraction module consisting of three layers of one-dimensional convolution and one fully connected layer. To extract the spectral features

[0081]

[0082] And an attention coefficient To represent the subsequence x j The historical value of x i (T-t+1:T) for the subsequence x i The future value of x i (T-t+1:T) is important. In calculating the attention coefficient Previously, the causal-frequency attention mechanism firstly extracts spectral features through two modules. and x i (T-t+1:T) and x j (T-t+1:T) is mapped into a pair of eigenvectors and

[0083]

[0084] After obtaining the query-key vector pair, the causal-frequency domain attention mechanism calculates the attention coefficient by the following formula

[0085]

[0086] Remember C (l)′ for:

[0087]

[0088] And C(l) is:

[0089]

[0090] but:

[0091]

[0092] Among them, l is the number of the attention head, is the Laplace matrix of the enhanced causal graph corresponding to the lth attention head. And the frequency domain eigenvector After that, the causal-frequency attention mechanism calculates the subsequence x by weighted summation. i The spectral feature vector of Right now:

[0093]

[0094] For the multi-head attention mechanism, the causal-frequency attention mechanism will focus each attention head on the sequence x i The outputs of (T-t+1:T) are concatenated along the feature dimension to obtain the final feature H i :

[0095]

[0096] The output of the causal-frequency domain attention mechanism is the set of spectral feature vectors of each subsequence:

[0097]

[0098] like Figure 4 As shown in Figure 1, the decoder Dec(·) is a multi-layer perceptron consisting of two layers of fully connected neural networks activated by the ReLU(·) function. Its input is the element H in the feature vector set i , the output is the subsequence x corresponding to the feature vector i The predicted value in the interval from T+1 to T+τ. The function of the first layer of fully connected neural network is to compress the feature dimension of the feature vector from d v m is compressed to 1. The second layer of the fully connected neural network is used to decode the compressed feature vector to obtain the subsequence x i Prediction value in the interval from T+1 to T+τ

[0099]

[0100] This application also provides a verification test to further prove the technical effect of this application.

[0101] We selected nine time series data sets to verify the present invention, including four transformer temperature (Electric Transformer Temperature, ETT) data sets (ETTh1, ETTm1, ETTh2 and ETTm2), two meteorological data sets (Weather (Germany) and Weather (USA)), two financial data sets (Exchange Rate and Stock), and one infectious disease data set (National Illness). As usual, the data sets used in the experiment are all normalized.

[0102] To verify the superiority of this method, this embodiment is compared with the following existing time series prediction methods: Reformer (from "Kitaev N, Kaiser Levskaya A.Reformer:The efficient transformer[J].arXiv preprint arXiv:2001.04451,2020."), Autoformer (from "Wu H,Xu J,Wang J,et al.Autoformer:Decomposition transformers with auto-correlation for long-term series forecasting[J].Advances in neural information processing systems, 2021,34:22419-22430."), Fedformer (from "Zhou T, Ma Z, Wen Q, et al. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting[C] / / International conference on machine learning.PMLR, 2022:27268-27286."), NHITS (from "Challu C, Olivares KG, Oreshkin BN, et al. Nhits: Neural hierarchical interpolation for time series forecasting[C] / / Proceedings of the AAAI conference on artificial intelligence.2023,37(6):6989-6997.”) and DLinear (excerpted from “Zeng A, Chen M, Zhang L, et al.Are transformers effective for time series forecasting? [C] / / Proceedings of the AAAI conference on artificial intelligence.2023,37(9):11121-11128.”). This embodiment uses mean square error (MSE) and mean absolute error (MAE) as evaluation criteria to measure the performance of each algorithm.

[0103] Table 1 shows the prediction accuracy of each method on nine datasets. The highest accuracy in the table is marked in bold, and the second highest accuracy is underlined. It can be seen from the data in the table that the causal-temporal attention network proposed in this chapter has achieved a much higher prediction accuracy than other benchmark methods as a whole. Specifically, the causal-temporal attention network proposed in this chapter achieved 44 lowest mean square errors and 49 lowest mean absolute errors, as well as six second lowest mean square errors and one second lowest mean absolute error in a total of 50 experiments on nine datasets. The method proposed in this chapter achieved all the lowest mean square errors and lowest mean absolute errors on the ETTh1, ETTh2, ExchangeRate, Stock and National Illness datasets. Comprehensive experimental data show that the time series prediction method based on the causal-temporal attention network proposed in this paper can effectively handle time series prediction tasks, and this method can achieve higher prediction accuracy than its contemporary benchmark methods, which shows that the method proposed in this paper is effective and advanced in time series prediction tasks.

[0104] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

[0105] Table 1: Prediction accuracy of each method on different datasets

[0106]

Claims

1. A time series prediction system based on causality, characterized in that: Specifically include: (1) A dynamic causal relationship inference module applicable to time series, which is used to infer the dynamic causal graph of each variable from the sampled data, including a greedy equivalent search algorithm module based on cross-validation log-likelihood score, a dynamic causal inference algorithm module based on neural network, and an adaptive causal graph fusion network module; (2) A causal-frequency domain attention mechanism module, which is used to extract the causal-frequency domain features of the time series historical data and encode the features into a set of feature vectors; (3) A decoder module based on a multi-layer perceptron is used to decode the causal-frequency domain features output by the causal-frequency domain attention mechanism to obtain the final prediction result.

2. The causal relationship-based time series prediction system according to claim 1, characterized in that: In the dynamic causal relationship inference module: (1) The greedy equivalent search algorithm module based on cross-validation log-likelihood score is as follows: Definition: For a directed acyclic graph The set of all its equivalent graphs is called Call a set Directed Acyclic Graph The set of one-edge equivalent graphs of is if and only if for any directed acyclic graph There is a graph Make Is Add an edge to get; similarly, call a set Directed Acyclic Graph The set of first-order edge-reduced equivalent graphs of is if and only if for any directed acyclic graph There is a graph Make Is Obtained by deleting an edge; Based on the above definition, the greedy equivalent search algorithm can be summarized as: The input of the algorithm is all samples of the time series At initialization, the algorithm constructs a directed acyclic graph with an empty edge set based on the variables in the time series. Afterwards, in the forward search phase, the algorithm will repeatedly Replace with The set of one-edge-added equivalent graphs until there is no graph with a higher score than the current graph; then, in the reverse search phase, the algorithm will repeatedly Replace with The set of equivalent graphs with reduced edges The graph with the highest median likelihood score is selected until there is no graph with a higher score than the current graph. This is the final causal diagram; The cross-validation log-likelihood score is calculated as follows: Suppose there is a time series consisting of n variables And the directed acyclic graph composed of these variables The sequence is sampled m times from time 1 to time m to obtain m samples, and the set of these samples is recorded as where x i =[x i (1),x i (2),…,x i (m)] T is the sampling set of the i-th variable; the sample set x i The kernel matrix is ​​denoted as K(x i ),Right now: where κ(·,·) is the Gaussian kernel function, which is defined as follows for two vectors or scalars y and y′: make: Where I is the n×n identity matrix, 1 is an n-dimensional all-1 vector, then K(x i ) corresponds to the centralization matrix for: When doing cross validation, Divide into a training set and a test set, a total of Q times; the number of samples in the training set is m tr , the number of samples in the test set is m te , and m tr +m te =m; the training set divided for the qth time is recorded as Test set is denoted as Further, the variable x in the qth division i The training set and test set of its father variable are denoted as and So, the directed acyclic graph The cross-validated log-likelihood score of is expressed as: in, is about a single variable x i and its parent variable, the cross-validated log-likelihood score, which is estimated by changing x i is the target, its parent variable The feature is obtained by K-fold cross validation. Its specific definition is: in, The calculation formula is: Formulas (5), (6) and (7) are the mathematical representations of the cross-validation log-likelihood scores for time series; (2) The dynamic causal inference algorithm module based on the neural network, wherein the neural network is a local causal inference network whose input is a time series The sampling at t time points before time T is output as m local causal graphs; the local causal network is divided into three sub-blocks: preprocessing module, temporal feature extraction module and graph structure generation module; among them: The preprocessing module is used to pre-extract features from the input time series data and evenly divide the extracted features into m subsequences in the time dimension for subsequent generation of local causal graphs; the module consists of three layers of one-dimensional atrous convolutional networks activated by ReLU (·) functions; For the input sequence Each subsequence in The preprocessing module maps it into a feature vector After obtaining the eigenvectors of all subsequences {γ1,γ2,…,γ n }, the preprocessing module first concatenates these vectors in the feature dimension to obtain a feature matrix Then, the feature matrix Γ is evenly divided into m sub-matrices in the time dimension as the input of the temporal feature extraction module, that is: The time series feature extraction module consists of a layer of long short-term memory network, which is used to extract the time series in different time periods. Time series feature vector; this module uses each sub-matrix Γ (i) As input, output a dimension of The time series feature vector Δ (i) , where the initial state of each gate in the long short-term memory network is 0, and each sub-matrix Γ (i) Input into the long short-term memory network respectively, and the output is the time series feature vector Δ (i) ; After the LSTM network processes a submatrix, its internal hidden state will be reset to 0 to ensure the independence of each submatrix; The graph structure generation module is used to compress the output of the time series feature extraction module and generate a local causal graph; the module is composed of two parallel multi-layer perceptrons, which are used to generate the probability matrix of the local causal graph respectively. and the mask matrix The final local causal graph is obtained by element-wise multiplication of the probability matrix and the mask matrix, namely: in, is the adjacency matrix representation of the i-th local causal graph. Whether the i-th row and j-th column of the adjacency matrix is ​​not zero represents whether there is a directed edge from the i-th node to the j-th node; MLP p (·) is a multilayer perceptron used to generate probability matrices. Its internal neurons are activated by the ReLU(·) function, and the last layer of neurons is activated by the Sigmoid(·) function; MLP m (·) is a multilayer perceptron used to generate a mask matrix; "⊙" is a matrix element multiplication operator, which is used to perform element-by-element multiplication of two matrices of the same dimension; (3) The adaptive causal graph fusion network, whose input is the global causal graph and m local causal graphs The output is m enhanced causal graphs The network consists of two parallel submodules: a probability matrix generation submodule and a mask matrix generation submodule; the probability matrix generation submodule consists of four layers of convolutional neural networks and one fully connected layer; the input of the first layer of convolutional neural network is the adjacency matrix of the global causal graph. and the adjacency matrix of the i-th local causal graph The concatenated dimensions are Tensor After four convolutional layers, the tensor has a dimension of Tensor The fully connected layer compresses the last dimension of the tensor from 128 to 1 and activates it with the Sigmoid(·) function to obtain the probability matrix Similar to the probability matrix generation submodule, the mask matrix generation submodule also consists of a four-layer convolutional neural network and a fully connected layer. Its input is the Laplacian matrix of the global causal graph. and the Laplacian matrix of the i-th local causal graph The concatenated dimensions are Tensor The output dimension is The mask matrix Except that the fully connected layer uses the ReLU(·) function as the activation function, the data flow path of this tensor in the mask matrix generation module is the same as that in the probability matrix generation module; finally, the Laplacian matrix of the enhanced causal graph is It is obtained by doing Hadamard product of probability matrix and mask matrix, that is: These m enhanced causal graphs are subsequently input into the corresponding causal-frequency domain attention mechanism heads for modeling time series.

3. The causal relationship-based time series prediction system according to claim 2, characterized in that: The causal-frequency attention mechanism module described above has a time series input for a single attention head. Samples within t time points before time T and the lth enhanced causal graph The output is each subsequence x i The set of spectral feature vectors of (T-t+1:T) in, is a subsequence x i The eigenvector corresponding to (T-t+1:T); In order to obtain the spectral feature vectors of each subsequence, the historical sampling data x of each subsequence is first transformed by real fast Fourier transform. i (T-t+1:T) is transformed from the time domain to the frequency domain f i (T-t+1:T), that is: in, stands for Real Fast Fourier Transform Algorithm; After transforming the sequence from the time domain to the frequency domain, a feature extraction module consisting of three layers of one-dimensional convolution and one fully connected layer is used. To extract the spectral features And an attention coefficient To represent the subsequence x j The historical value of x i (T-t+1:T) for the subsequence x i Future Value x i The importance of (T-t+1:T); in calculating the attention coefficient Previously, the causal-frequency attention mechanism firstly extracts spectral features through two modules. and x i (T-t+1:T) and x j (T-t+1:T) is mapped into a pair of eigenvectors and After obtaining the query-key vector pair, the causal-frequency domain attention mechanism calculates the attention coefficient by the following formula Remember C (l)′ for: and C (l) for: but: Among them, l is the number of the attention head, is the Laplace matrix of the enhanced causal graph corresponding to the lth attention head; after calculating the attention coefficient And the frequency domain eigenvector After that, the causal-frequency attention mechanism calculates the subsequence x by weighted summation. i The spectral feature vector of Right now: For the multi-head attention mechanism, each attention head is used for the sequence x i The outputs of (T-t+1:T) are concatenated along the feature dimension to obtain the final feature H i : The output of the causal-frequency attention mechanism is a set of spectral feature vectors of each subsequence:

4. The causal relationship-based time series prediction system according to claim 3, characterized in that: The decoder module uses the feature vector H generated by the causal-frequency domain attention network i As input, a multi-layer perceptron MLP(·) is used to All feature vectors in are decoded to produce time series The spectrum from T+1 to T+τ is used, and the inverse real Fourier transform is used to obtain the final prediction result, that is: f i (T+1:T+τ)=MLP(H i ) (24) Here, a multilayer perceptron with two fully connected layers is used as the decoder. The first layer of the multilayer perceptron uses ReLU (·) as the activation function to transform the feature vector The feature dimension is changed from m×d v Compressed to 1; the second layer of multi-layer perceptron decodes the compressed feature vector and transforms it from the frequency domain back to the time domain to obtain the final prediction result.

Images