A spatial time series prediction method based on cyclic graph operator neural network

Abstract

This invention discloses a spatial-time series prediction method based on a cyclic graph operator neural network. The method comprises the following steps: modeling a traffic network to obtain a graph structure model, abstracting sensors in the traffic network as nodes in the graph structure model; normalizing the data collected by the sensors; constructing multiple graph operator networks as feature capturers of the data, each of which aggregates information from each node in the graph structure model in different ways; constructing GGRU units, respectively embedding the multiple graph operator networks into the GRU units of the neural network; aggregating the feature information captured by the different graph operator networks through an integrator; and constructing an encoder-decoder architecture using multiple integrators to predict spatial multivariate time series. This method has the ability to understand information from multiple perspectives, effectively improving the understanding of spatial multivariate time series data and thereby improving the accuracy of predictions.

Description

Technical Field

[0001] The present invention relates to the technical field of space-time series prediction, and in particular to a space-time series prediction method based on a cyclic graph operator neural network. Background Art

[0002] Spatial time series prediction has a wide range of application needs in modern times, such as: (1) Weather forecasting: Weather forecasting is an important application of spatial time series prediction. By modeling meteorological data, it is possible to predict the changing trends of meteorological variables such as temperature, rainfall, and wind speed in the future, providing decision support for weather forecasting, disaster warning, agricultural production, etc. (2) Energy demand forecasting: It is very important for energy suppliers and consumers to understand the changing trends of future energy demand. By modeling historical energy demand data, future energy demand can be predicted in order to adjust production plans and supply chain management. (3) Traffic forecasting: Traffic forecasting can help urban planners and traffic managers better manage traffic flow and improve traffic efficiency. By modeling historical traffic data, future road congestion, traffic accident rates, etc. can be predicted so that corresponding measures can be taken. (4) Population flow forecasting: Population flow is an important factor in urban planning and social policy making. By modeling historical population flow data, future population flow trends can be predicted so that urban infrastructure and social resource allocation can be rationally planned. (5) Stock price forecasting: Stock price forecasting is an important application in the financial field. By modeling historical stock price data, we can predict future stock price trends and provide decision support for investors.

[0003] Currently, deep learning methods are mainly used for multivariate time series prediction. This neural network-based model can be effectively used for multivariate time series prediction. For example: (1) Recurrent Neural Network (RNN): It is a neural network that can process sequence data. It captures the temporal relationship of the sequence by using the output of the previous moment as the input of the current moment. In multivariate time series prediction, multiple RNNs can be used to process each sequence separately, and then their outputs are connected to obtain the final prediction result. RNN is widely used in multivariate time series prediction. For example, it has achieved good results in traffic flow prediction, stock price prediction and other fields. (2) Long Short-Term Memory Network (LSTM): It is a special RNN that can better handle long sequences and long-term dependencies. In multivariate time series prediction, multiple LSTMs can be used to process each sequence, and then their outputs are connected to obtain the final prediction result. Compared with RNN, LSTM performs better in processing long sequences and long-term dependencies, so it may be more suitable in some applications. (3) Convolutional Neural Network (CNN): It is a neural network mainly used for image processing, but in some cases it can also be used for multivariate time series prediction. In multivariate time series prediction, multiple sequences can be viewed as multiple channels, and then multiple convolution kernels are used to perform convolution operations on each channel. Finally, the outputs of all channels are connected to obtain the final prediction result. CNN is relatively rarely used in multivariate time series prediction, but it may be more suitable in some applications. (4) Attention mechanism: Attention is a mechanism that can give different weights to different parts of the input data and can be used in multivariate time series prediction. In multivariate time series prediction, the attention mechanism is used to automatically learn the importance of different parts of the sequence and apply the attention weights to the prediction of each sequence. The application of the attention mechanism in multivariate time series prediction is relatively new, but it has achieved good results in some applications.

[0004] However, the aforementioned multivariate time series forecasting methods either completely ignore the spatial dependencies between series or consider them only from a single perspective. In reality, the underlying relationships between these series are complex. Therefore, there is an urgent need for new forecasting methods that can fully explore these underlying relationships from multiple dimensions. Just like how humans read text, the metaphorical meaning of words often goes deeper than the surface meaning. Summary of the Invention

[0005] The purpose of the present invention is to provide a spatial time series prediction method based on a cyclic graph operator neural network, which has the ability to understand information from multiple angles, effectively improves the understanding of spatial multivariate time series data, and thus improves the accuracy of prediction.

[0006] To achieve the above objectives, this application proposes a spatial time series prediction method based on a cyclic graph operator neural network, comprising:

[0007] Modeling the traffic network to obtain a graph structure model, and abstracting sensors in the traffic network as nodes in the graph structure model;

[0008] Normalize the data collected by sensors;

[0009] Construct a variety of graph operator networks as feature capturers of data, which use different methods to aggregate the information of each node in the graph structure model;

[0010] Construct GGRU units and embed various graph operator networks into the GRU units of the neural network;

[0011] Aggregate the feature information captured by different graph operator networks through the integrator;

[0012] By using multiple integrators, a sequence-to-sequence encoder-decoder architecture is constructed to achieve the prediction of spatial multivariate time series.

[0013] Further, use Represents the traffic flow data collected by all sensors at time r, where N represents the number of sensors and P represents the number of traffic indicators detected by each sensor; Represents the historical observation value within T' timestamps, using Represents the predicted value of T time stamps in the future; and then determines the function of the neural network learning graph structure model:

[0014]

[0015] in, is the neural network to be fitted.

[0016] Furthermore, the data collected by the sensor is normalized in the following way:

[0017]

[0018]

[0019] Among them, X represents the training set sample, X * represents the normalized data, E(X) represents the mean of the training set samples, and D(X) represents the variance of the training set samples.

[0020] Furthermore, we construct a variety of graph operator networks as feature capturers of the data, specifically:

[0021] Construct the graph diffusion convolution operator DC Operator, which is a graph operator network based on static graphs;

[0022] Construct a graph-gated attention operator GA Operator, which is a graph operator network based on dynamic graphs.

[0023] Furthermore, the specific method of constructing the graph diffusion convolution operator DC Operator is as follows:

[0024] The graph diffusion convolution operator adopts a random walk strategy:

[0025]

[0026]

[0027]

[0028] Among them, X represents the input of the model, H represents the output of the model; its diffusion process uses a limited S-step truncation; D O , D I represents the out-degree matrix and in-degree matrix in the graph, represents the forward state transfer matrix and the reverse state transfer matrix; α, β∈[0,1] represent the restart probability of random walk respectively; represents the model parameters, and Θ O[q，p，s] =α(1-α) s ,Θ I[q，p，k] =β(1-β) s ;

[0029] Use the distance-based Gaussian kernel to get the critical matrix W:

[0030]

[0031] Here dist(v i , v j ) indicates sensor v i and v j The distance between them; σ is the standard deviation of the distance set.

[0032] Furthermore, the specific method of constructing the graph gated attention operator GA Operator is as follows:

[0033] Using the K-head attention mechanism in the graph gated attention operator, there is a K-dimensional gating vector g for each node i i :

[0034] in, represents the set of all neighbor nodes of node i; x i =Xi，： Represents the feature vector of node i; The reference vectors of all neighboring nodes of node i, and Max means taking the maximum value of each element; Indicates that the final result is mapped to K dimensions and scaled to [0, 1];

[0035] Dynamically obtain the attention weight matrix between nodes:

[0036]

[0037]

[0038] in, Denote the parameter θ xa Linear transformation of Denote the parameter θ za The linear transformation of represents a linear layer; θ represents different parameters.

[0039] Get the output vector y of node i i :

[0040]

[0041] Among them, K represents the number of attention heads, and k represents that the kth attention head is currently being executed; represents a linear transformation with parameter θ0; Indicates that the parameter is Linear transformation of .

[0042] Furthermore, the GGRU unit uses Represent different graph operators:

[0043]

[0044]

[0045]

[0046] H (t) =u (t) ⊙H (t-1) +(1-u (t) )⊙C (t)

[0047] Among them, X (t) , H (t) Represents the input and output of the tth timestamp; r (t) ,u (t)Represents the gated state of the reset gate and update gate of the t-th timestamp; Θ r ,Θ u ,Θ C are different filter parameters; Γ g Represents a specified graph network Execution Operator; ⊙ represents the Hadamard product.

[0048] As a further step, the feature information captured by different graph operator networks is aggregated through the integrator, specifically:

[0049]

[0050] in, represents the GRU unit embedded with graph operators, I represents the number of graph operators, It indicates that the output dimension is d0, and τ indicates that the tanh activation function is used in the last layer.

[0051] As a further step, the input of the encoder is the historical observations The output of the decoder is the future prediction value

[0052] The technical solution employed in this invention offers several advantages over existing technologies: It employs a deep learning model, combined with multiple graph operator networks, to model the spatial dependencies between nodes from multiple perspectives. Furthermore, an integrator is used to effectively aggregate the information captured by these multiple graph operators. This enhances the model's ability to understand the underlying dependencies between multivariate time series and improves forecast accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 For the use of parallel single input, single output structure diagram;

[0054] Figure 2 To use parallel multi-input, multi-output structure diagram;

[0055] Figure 3 For using parallel single-input, single-output and residual-like structure diagrams;

[0056] Figure 4 To use parallel multi-input, multi-output and residual-like structure diagrams;

[0057] Figure 5 To use the series single input, single output structure diagram;

[0058] Figure 6 To use the serial multi-input, multi-output structure diagram;

[0059] Figure 7 To use a serial single input, single output and residual-like structure diagram;

[0060] Figure 8 To use serial multi-input, multi-output and residual-like structure graphs;

[0061] Figure 9 This is the GGRU model structure diagram;

[0062] Figure 10 This is the encoder-decoder architecture diagram. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. That is, the embodiments described are only part of the embodiments of this application, not all of them.

[0064] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present application.

[0065] This application provides a spatial time series prediction method based on a cyclic graph operator neural network, specifically including:

[0066] Step 1: Model the traffic network to obtain a graph structure model, and abstract the sensors in the traffic network into nodes in the graph structure model;

[0067] Specifically, use Represents the traffic flow data collected by all sensors at time t, where N represents the number of sensors and P represents the number of traffic indicators detected by each sensor; Represents the historical observation value within T′ timestamps, using Represents the predicted value of T time stamps in the future; and then determines the function of the neural network learning graph structure model:

[0068]

[0069] in, is the neural network to be fitted.

[0070] Step 2: Normalize the data collected by the sensor:

[0071]

[0072]

[0073] Among them, X represents the training set sample, X * represents the normalized data, E(X) represents the mean of the training set samples, and D(X) represents the variance of the training set samples.

[0074] Step 3: Construct multiple graph operator networks as feature capturers of the data, which aggregate the information of each node in the graph structure model in different ways;

[0075] Specifically, the present invention includes two types of graph operators, which respectively aggregate the information of each node in the graph structure model in different ways and embed it into the GRU unit as a replacement for the original linear unit.

[0076] Step 3.1: Construct the graph diffusion convolution operator DC Operator, which is a graph operator network based on a static graph;

[0077] Step 3.1.1: The graph diffusion convolution operator adopts a random walk strategy:

[0078]

[0079]

[0080]

[0081] Here, X represents the input of the model and H represents the output of the model. Its diffusion process is truncated with a limited number of K steps. O , D I represents the out-degree and in-degree matrices in the graph, Denotes the forward state transfer matrix and the reverse state transfer matrix. α, β∈[0,1] denote the restart probability of the random walk respectively. represents the model parameters, and Θ O[q，p，s] =α(1-α) s ,Θ I[q，p，s] =β(1-β) s . It represents the probability value of performing S-step diffusion from the i-th node to its neighboring nodes. The former represents the forward propagation probability, and the latter represents the reverse propagation probability.

[0082] Step 3.1.2: Use the distance-based Gaussian kernel to obtain the critical matrix W:

[0083]

[0084] Here dist(v i , v j ) indicates sensor vi and v j The distance between them; σ is the standard deviation of the distance set.

[0085] Step 3.2: Construct the graph-gated attention operator GA Operator, which is a graph operator network based on the dynamic graph;

[0086] Step 3.2.1: Use the K-head attention mechanism in the graph gated attention operator, then for each node i there is a K-dimensional gating vector g i :

[0087]

[0088] in, represents the set of all neighbor nodes of node i; x i =X i，： Represents the feature vector of node i; The reference vectors of all neighboring nodes of node i, and Max means taking the maximum value of each element; Indicates that the final result is mapped to K dimensions and scaled to [0, 1];

[0089] Step 3.2.2: Dynamically obtain the attention weight matrix between nodes:

[0090]

[0091]

[0092] in, Denote the parameter θ xa Linear transformation of Denote the parameter θ za Linear transformation of

[0093] Step 3.2.3: Get the output vector y of node i i :

[0094]

[0095] Step 4: Construct GGRU units and embed various graph operator networks into the GRU units of the neural network. Figure 9 ,here Represent different graph operators:

[0096]

[0097]

[0098]

[0099] H (t) =u (t) ⊙H (t-1) +(1-u (t) )⊙C (t)

[0100] Among them, X (t) , H (t) Represents the input and output of the tth timestamp; r (t) ,u (t) Represents the gated state of the reset gate and update gate of the t-th timestamp; Θ r ,Θ u ,Θ C are different filter parameters; Represents a specified graph network Execution Operator; ⊙ represents the Hadamard product.

[0101] Step 5: Aggregate the feature information captured by different graph operator networks through the integrator;

[0102]

[0103] in, Represents a GRU unit embedded with graph operators. Figures 1-8 Eight aggregator structures are given.

[0104] Step 6: Use multiple integrators to build a sequence-to-sequence encoder-decoder architecture to predict spatial multivariate time series, see Figure 10 ;

[0105] Specifically, the input of the encoder is the historical observation value The output of the decoder is the future prediction value

[0106] This embodiment uses the Ubuntu system as the development environment, Python as the development language, and Pytorch to build the framework. A spatial time series prediction method based on a cyclic graph operator neural network of the present invention is used to predict traffic flow: a graph structure model representation of the traffic network is obtained, and the public METR-LA dataset is used; the dataset is divided into a training set, a validation set, and a test set, with a ratio of 7:1:2. In order to compare the performance of the algorithm, it is compared with some commonly used multivariate time series prediction methods. These include the statistical learning method ARIMA; the machine learning method LSVR; the deep learning method FC-LSTM; and the graph-based deep learning methods DCRNN, STGCN, GaAN, ASTGCN, and GMAN. At the same time, a variety of evaluation indicators are used to conduct a more comprehensive evaluation of the performance of the model, including MAE, RMSE, MAPE, etc.

[0107]

[0108]

[0109]

[0110] Among them, x i Represents actual data, Represents the data predicted by the model.

[0111] The comparison results are shown in Table 1: the best evaluation indicators among all algorithms are bolded.

[0112] Table 1: Performance comparison of different traffic speed prediction models on the METR-LA dataset

[0113]

[0114]

[0115] Table 1 compares the prediction results of the iGoRNN model with other baseline models. By comparing the scores of each model in long-term and short-term time series prediction, it can be seen that the graph-based neural network model achieves better prediction accuracy. For the smoother sequence PeMS-BAY, some traditional models achieved good results even in short-term predictions (15 minutes), but performed poorly in long-term predictions (60 minutes). For the unstable sequence Metr-LA, traditional models performed poorly in both short-term and long-term predictions. On the other hand, the GaAN model performed well on the Metr-LA dataset. GMAN is more suitable for smoother sequences. All of this illustrates the importance of introducing graph structures in complex time series prediction. In addition, the model designed by the present invention shows optimal or suboptimal performance in long-term and short-term predictions, especially for more complex time series because it introduces the advantages of multiple graph structures at the same time.

[0116] In summary, the spatial time series prediction method based on cyclic graph operator neural network proposed in the present invention has better performance and robustness than other advanced methods, and can adapt to more complex multivariate time series prediction problems.

[0117] The foregoing descriptions of specific exemplary embodiments of the present invention are for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been selected and described for the purpose of explaining the specific principles of the invention and their practical application, thereby enabling those skilled in the art to realize and utilize a variety of exemplary embodiments of the invention and various options and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A spatial time series prediction method based on a cyclic graph operator neural network, characterized in that: include: Modeling the traffic network to obtain a graph structure model, and abstracting sensors in the traffic network as nodes in the graph structure model; Normalize the data collected by sensors; Construct a variety of graph operator networks as feature capturers of data, which use different methods to aggregate the information of each node in the graph structure model; Construct GGRU units and embed various graph operator networks into the GRU units of the neural network; Aggregate the feature information captured by different graph operator networks through the integrator; Using multiple integrators, we build a sequence-to-sequence encoder-decoder architecture to predict spatial multivariate time series. Construct a variety of graph operator networks as feature capturers of the data, specifically: Construct the graph diffusion convolution operator DC Operator, which is a graph operator network based on static graphs; Construct the graph-gated attention operator GA Operator, which is a graph operator network based on dynamic graphs; The specific method of constructing the graph diffusion convolution operator DC Operator is as follows: The graph diffusion convolution operator adopts a random walk strategy: Among them, X represents the input of the model, H represents the output of the model; its diffusion process uses a limited S-step truncation; D O , D I represents the out-degree matrix and in-degree matrix in the graph, represents the forward state transfer matrix and the reverse state transfer matrix; α, β∈[0,1] represent the restart probability of random walk respectively; represents the model parameters, and Θ O[q，p，s] =α(1-α) s ,Θ I[q，p，k] =β(1-β) s ; Use the distance-based Gaussian kernel to get the critical matrix W: Here dist(v i , v j ) indicates sensor v i and v j The distance between them; σ is the standard deviation of the distance set; The specific method of constructing the graph gated attention operator GAOperator is: Using the K-head attention mechanism in the graph gated attention operator, there is a K-dimensional gating vector g for each node i i : in, represents the set of all neighbor nodes of node i; x i =X i，: Represents the feature vector of node i; The reference vectors of all neighboring nodes of node i, and Max means taking the maximum value of each element; Indicates that the final result is mapped to K dimensions and scaled to [0, 1]; Dynamically obtain the attention weight matrix between nodes: in, Denote the parameter θ xa Linear transformation of Denote the parameter θ za Linear transformation of Get the output vector y of node i i : Among them, K represents the number of attention heads, and k represents that the kth attention head is currently being executed; represents a linear transformation with parameter θ0; Indicates that the parameter is Linear transformation of .

2. The spatial time series prediction method based on a cyclic graph operator neural network according to claim 1, characterized in that: use represents the traffic flow data collected by all sensors at time t, where N represents the number of sensors and P represents the traffic flow of each sensor. The number of traffic indicators detected; Represents the historical observation value within T′ timestamps, using Represents the predicted value of T time stamps in the future; and then determines the function of the neural network learning graph structure model: in, is the neural network to be fitted.

3. The spatial time series prediction method based on a cyclic graph operator neural network according to claim 1, characterized in that: The specific method for normalizing the data collected by the sensor is as follows: Among them, X represents the training set sample, X * represents the normalized data, E(X) represents the mean of the training set samples, and D(X) represents the variance of the training set samples.

4. The spatial time series prediction method based on a cyclic graph operator neural network according to claim 1, characterized in that: Used in GGRU units Represent different graph operators: H (t) =u (t) ⊙H (t-1) +(1-u (t) )⊙C (t) Among them, X (t) , H (t) Represents the input and output of the tth timestamp; r (t) ,u (t) Represents the gating state of the reset gate and update gate at the t-th timestamp; Θ r ,Θ u ,Θ C are different filter parameters; Represents a specified graph network Execution Operator; ⊙ represents the Hadamard product.

5. The method for spatial time series prediction based on a cyclic graph operator neural network according to claim 2, characterized in that: The feature information captured by different graph operator networks is aggregated through the integrator, specifically: in, represents the GRU unit embedded with graph operators, I represents the number of graph operators, It indicates that the output dimension is d0, and τ indicates that the tanh activation function is used in the last layer.

6. The method for spatial time series prediction based on a cyclic graph operator neural network according to claim 2, characterized in that: The input of the encoder is the historical observations The output of the decoder is the future prediction value

Images