Space-time sequence prediction method

Abstract

The invention provides a space-time sequence prediction method, which comprises the following steps of: reading space-time sequence data, and converting the space-time sequence data into a processable three-dimensional space-time sequence data matrix comprising time, space and characteristics; enabling the obtained three-dimensional space-time sequence data matrix to pass through a data embedding layer comprising depth separable convolution, point-level convolution, a multi-layer perceptron and residual connection to obtain a first space-time sequence data matrix. And inputting the first space-time sequence data matrix into a space-time sequence prediction model containing L space-time processing modules to obtain time sequence data of future time. Wherein each space-time processing module comprises a Chebyshev space attention module and a positive random feature function time attention module. Compared with a classical algorithm, the space-time sequence prediction method provided by the invention is good in performance.

Description

Technical Field

[0001] The present invention belongs to the field of spatiotemporal sequence prediction of artificial intelligence, and in particular relates to a spatiotemporal sequence prediction method. Background Art

[0002] Traffic forecasting is based on the historical traffic data of the city to predict the traffic status for a long period of time in the future. The traffic status involved is generally manifested in traffic flow, traffic speed and traffic density. Looking at the existing traffic forecasting models at home and abroad, there are a variety of models. From the perspective of the methodology of the traffic forecasting model, it can be divided into four types, namely, traffic forecasting models based on statistics, traffic forecasting models based on traditional machine learning, traffic forecasting models based on deep learning, and traffic forecasting models based on the combination of graph neural networks and attention. The main descriptions are as follows:

[0003] First, traffic forecasting models based on statistics. This type of model is the main traffic forecasting method used in ITS in the early days. It is widely used in short-term traffic forecasting due to its good model analysis ability and comprehensive feasibility. Even now, this type of model is still suitable for some simple traffic forecasting scenarios due to its inherent advantages, such as auto-regressive integrated moving average (ARIMA), seasonal auto-regressive integrated moving average (SARIMA), vector auto-regressive moving average (VARIMA), etc. Although the traffic forecasting model based on statistics plays an intuitive and important role in early traffic forecasting, it is a simple linear problem and has poor processing effect on traffic data with nonlinear and complex characteristics.

[0004] Secondly, traffic prediction models based on traditional machine learning. At the beginning of this century, traffic prediction models based on traditional machine learning were widely used in short-term traffic prediction due to their advantages of high accuracy and strong nonlinear processing capabilities. They have become the mainstream prediction methods supporting ITS, such as K-nearest neighbors (KNN), support vector machine (SVM) and linear regression algorithm (LRA). Compared with traffic prediction models based on statistics, such algorithms have promoted the accuracy of traffic prediction, but such algorithms rarely consider the spatial dependence of traffic networks, and their dependence on distance functions, kernel functions, etc. is relatively large, resulting in unsatisfactory performance of traffic prediction.

[0005] Furthermore, traffic prediction models based on deep learning. In order to effectively capture the global spatial information of traffic data, traffic prediction models represented by graph convolutional neural networks (CNN) and recurrent neural networks (RNN) have emerged. The idea of ​​such algorithms is to regard the complex traffic network as a grid-like data similar to an image, and then learn and mine the spatial dependency information contained in it. However, it is obvious that such algorithms violate the graph structure of the traffic network, and their effects are still unsatisfactory.

[0006] Finally, a traffic prediction model based on a combination of graph neural networks and attention. In order to deeply mine the control dependencies in traffic data, graph neural networks and self-attention networks that process graph structured data are widely used in traffic prediction, such as the graph wavelet model (GraphWaveNet, GW), attention-based spatial-temporal graph convolutional network (ASTGCN), etc. Although such models have greatly improved the accuracy of traffic prediction, how to more accurately mine the spatiotemporal dependencies, dynamics, and heterogeneity in traffic networks is still one of the hot topics of current research. Summary of the invention

[0007] In view of the relevant text submitted in the background technology, the present invention proposes a spatiotemporal sequence prediction method.

[0008] A spatiotemporal sequence prediction method comprises the following steps:

[0009] Step S1: read the spatiotemporal sequence data and convert it into a processable three-dimensional spatiotemporal sequence data matrix including time, space and features;

[0010] Step S2: The obtained three-dimensional spatiotemporal sequence data matrix is ​​subjected to a data embedding layer including a depthwise separable convolution, a point-level convolution, a multi-layer perceptron and a residual connection to obtain a first spatiotemporal sequence data matrix;

[0011] Step S3: Input the first spatiotemporal sequence data matrix into a spatiotemporal sequence prediction model comprising L spatiotemporal processing modules to obtain time series data for future time periods; wherein each spatiotemporal processing module comprises a Chebyshev spatial attention module and a positive random feature function time attention module.

[0012] Experiments show that the proposed spatiotemporal series prediction method has superior performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The flowchart of the spatiotemporal sequence prediction method of the present invention is

[0014] Figure 2 This is a comparison chart of the effects of the prediction method of the present invention and other classical methods. DETAILED DESCRIPTION

[0015] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0016] like Figure 1 As shown, the present invention discloses a spatiotemporal sequence prediction method, comprising the following steps:

[0017] Step S1: Read the spatiotemporal sequence data and convert it into a processable three-dimensional spatiotemporal sequence data matrix including time, space and features.

[0018] Reading spatiotemporal series data Where t is the current time, t-T1+1, t-T1+2,..., t-1 are the historical times of the spatiotemporal series data, represents the spatiotemporal sequence characteristics captured by N spatiotemporal data acquisition devices at time t, Represents the spatiotemporal sequence characteristics captured by the nth spatiotemporal data acquisition device at time t, n∈[1,N].

[0019] Step S2: The obtained three-dimensional spatiotemporal sequence data matrix is ​​subjected to a data embedding layer including a depth-separable convolution, a point-level convolution, a multi-layer perceptron and a residual connection to obtain a first spatiotemporal sequence data matrix.

[0020] The obtained three-dimensional spatiotemporal sequence data matrix is ​​passed through a data embedding layer including depthwise separable convolution, point-level convolution, multi-layer perceptron and residual connection to obtain the first spatiotemporal sequence data matrix. The specific description is as follows:

[0021]

[0022] in, is the first spatiotemporal sequence data matrix, MLP(·) is the multi-layer perceptron function, DWConv(·) is the depthwise separable convolution, and PWConv(·) is the point-level convolution.

[0023] Step S3: Input the first spatiotemporal sequence data matrix into a spatiotemporal sequence prediction model comprising L spatiotemporal processing modules to obtain time series data for future time periods; wherein each spatiotemporal processing module comprises a Chebyshev spatial attention module and a positive random feature function time attention module.

[0024] The following will take the lth spatiotemporal processing module among the L spatiotemporal processing modules as an example to illustrate the processing flow of each spatiotemporal processing module, as shown below:

[0025] (1) The processing process of the Chebyshev spatial attention module in the lth spatiotemporal processing module is as follows:

[0026] First, assume that the data input to the lth spatiotemporal processing module is The corresponding Jacobi polynomial is as follows:

[0027]

[0028] Among them, p0, p1, p k-1 , p k-2 and p k To calculate the 0th, 1st, k-1st, k-2nd, kth Chebyshev polynomials, To combine the corresponding Laplace matrix of the adjacency matrix and the dynamic adjacency matrix, the specific representation is as follows:

[0029]

[0030] Among them, A is the known adjacency matrix representing the traffic network, α∈(0,1) is a learnable parameter, E1, E2 are learnable matrices, and D is The corresponding degree matrix.

[0031] Secondly, the k Chebyshev polynomials in equation (1) are processed by multi-layer perceptron and combined to obtain the corresponding matrix H k , specifically expressed as shown in formula (3):

[0032] H k=(MLP(p0),MLP(p1),…,MLP(p k )) (3)

[0033] Furthermore, the self-attention mechanism is used to represent the matrix H represented by the k Chebyshev polynomials shown in formula (3): k Learn and get the output of the final Chebyshev spatial attention module of the lth layer The formula is shown in formula (4):

[0034]

[0035] Among them, W Q , W K and W V is a learnable matrix, (W K H k ) T W K H k is the transposed matrix of , LN(·) is the layer normalization operation, tan(·) is the tan function, FFN(·) is the forward feedback network, and D is the feature dimension of the spatiotemporal sequence.

[0036] (2) The processing process of the positive random feature function temporal attention module in the lth spatiotemporal processing module is as follows:

[0037] Assume that the positive random feature function temporal attention module data input to the lth spatiotemporal processing module is Then the positive random feature function temporal attention module can be expressed as:

[0038]

[0039] in is the traffic data at time u, is the traffic data at time v, is the traffic data at time ω, T is the time length, W Q ′,W K ′,W V ′ is a learnable vector, is the new traffic data at time u obtained by formula (5), and θ(·) is the positive random feature function. Finally, all the new traffic data obtained at time T are integrated into the output matrix of the final positive random feature function time attention module

[0040] Figure 2This is a comparison chart of the effects of the present invention and other classical methods. In order to verify the superiority of the proposed spatiotemporal series prediction method, the present invention and the classical differential autoregressive moving average model (ARIMA), support vector machine model (SVM), long short-term memory network (LSTM), attention spatiotemporal graph neural network model (ASTGCN), spatiotemporal graph neural network model (STSGCN), and graph wavelet model (GW) are used to test the spatiotemporal data sets PEMS04 and PEMS08, and MAE, MAPE, and RMSE are used to evaluate their effects. In addition, the SACPRF in the figure is the algorithm proposed by the present invention. From Figure 2 It can be seen that the traditional ARIMA model can only capture linear relationships in essence, so this type of model is mainly used in the early short-term traffic feature prediction, and its traffic prediction performance is poor; although SVM is a nonlinear model, its effect is highly dependent on data and kernel functions, so it is also a short-term traffic feature prediction model, and its traffic prediction effect is also poor. The ASTGCN model is affected by the graph structure. Although it predicts traffic characteristics from the attention and graph neural network levels, it is difficult to deeply explore the inherent space-time dependencies, so compared with the model proposed in the present invention, its prediction results are not good. From the experimental results, although the GM and STSGCN models rely on the capture of the dynamic characteristics of the traffic network, they use a more effective graph convolution layer structure or attention mechanism to capture the complex relationships in the time series data, and have stronger feature extraction capabilities, and can better learn useful features from the time series data, which helps to improve the prediction performance of the model during the training process, thus outperforming the ASTGCN model. However, it cannot be ignored that it still has major defects in mining the space-time dependencies in the traffic network. In summary, compared with the above-mentioned classic algorithms, a spatiotemporal series prediction method proposed in the present invention performs better.

Claims

1. A method for predicting spatiotemporal series, characterized in that The steps include: Step S1: read the spatiotemporal sequence data and convert it into a processable three-dimensional spatiotemporal sequence data matrix including time, space and features; Step S2: The obtained three-dimensional spatiotemporal sequence data matrix is ​​subjected to a data embedding layer including a depthwise separable convolution, a point-level convolution, a multi-layer perceptron and a residual connection to obtain a first spatiotemporal sequence data matrix; Step S3: Input the first spatiotemporal sequence data matrix into a spatiotemporal sequence prediction model comprising L spatiotemporal processing modules to obtain time series data for future time periods; wherein each spatiotemporal processing module comprises a Chebyshev spatial attention module and a positive random feature function time attention module.

2. The spatiotemporal series prediction method according to claim 1, characterized in that: The spatiotemporal sequence data read in step S1 above is Where t is the current time, t-T1+1, t-T1+2,..., t-1 are the historical times of the spatiotemporal series data, represents the spatiotemporal sequence characteristics captured by N spatiotemporal data acquisition devices at time t, Represents the spatiotemporal sequence characteristics captured by the nth spatiotemporal data acquisition device at time t, n∈[1,N].

3. The spatiotemporal series prediction method according to claim 2, characterized in that: The specific process of the above step S2 is: in, is the first spatiotemporal sequence data matrix, MLP(·) is the multi-layer perceptron function, DWConv(·) is the depthwise separable convolution, and PWConv(·) is the point-level convolution.

4. The spatiotemporal series prediction method according to claim 3, characterized in that: The specific process of the above step S3 is: The processing flow of each space-time processing module is described by taking the lth space-time processing module among the L space-time processing modules, as shown below: (1) The processing process of the Chebyshev spatial attention module in the lth spatiotemporal processing module is as follows: First, let the data input to the lth spatiotemporal processing module be The corresponding Jacobi polynomial is as follows: Among them, p0, p1, p k-1 , p k-2 and p k To calculate the 0th, 1st, k-1st, k-2nd, kth Chebyshev polynomials, To combine the corresponding Laplace matrix of the adjacency matrix and the dynamic adjacency matrix, the specific representation is as follows: Among them, A is the known adjacency matrix representing the traffic network, α∈(0,1) is a learnable parameter, E1, E2 are learnable matrices, and D is The corresponding degree matrix; Secondly, the k Chebyshev polynomials in equation (1) are processed and combined using a multilayer perceptron to obtain the corresponding matrix H k , specifically expressed as shown in formula (3): H k =(MLP(p0),MLP(p1),…,MLP(p k )) (3) Furthermore, the self-attention mechanism is used to represent the matrix H represented by the k Chebyshev polynomials shown in formula (3): k Learn and get the output of the final Chebyshev spatial attention module of the lth layer The formula is shown in formula (4): Among them, W Q , W K and W V is a learnable matrix, (W K H k ) T W K H k The transposed matrix of , LN(·) is the layer normalization operation, tan(·) is the tan function, FFN(·) is the forward feedback network, and D is the characteristic dimension of the spatiotemporal sequence; (2) The processing process of the positive random feature function temporal attention module in the lth spatiotemporal processing module is as follows: Let the positive random feature function temporal attention module data input to the lth spatiotemporal processing module be Then the positive random feature function time attention module is expressed as: in is the traffic data at time u, is the traffic data at time v, is the traffic data at time ω, T is the time length, W′ Q , W′ K , W′ V is a learnable vector, is the new traffic data at time u obtained by formula (5), and θ(·) is the positive random feature function. Finally, all the new traffic data obtained at time T are integrated into the output matrix of the final positive random feature function time attention module

Images