Traffic flow prediction method based on residual decomposition recurrent neural network

Abstract

The present invention discloses a traffic flow prediction method based on a residual decomposition recurrent neural network, comprising: obtaining a road occupancy dataset at consecutive moments, wherein the road occupancy dataset at consecutive moments includes a training set and a test set; constructing a residual decomposition recurrent neural network model, inputting the training set into the residual decomposition recurrent neural network model for training to obtain a trained residual decomposition recurrent neural network model; and inputting the test set into the trained residual decomposition recurrent neural network model to obtain a traffic flow time series prediction trend. The present invention models the input road occupancy time series from both the autoregressive and sliding average perspectives, simultaneously considering the impact of direct observations and residuals from past time points on the future direction of the sequence, and can also simultaneously consider the potential impact of higher-order residuals by stacking the network depth.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic flow prediction, and in particular to a traffic flow prediction method based on a residual decomposition recurrent neural network. Background Art

[0002] With recent economic growth worldwide, road construction has increased annually. Traffic control scenarios during highway construction and maintenance operations are becoming increasingly common, making Intelligent Transportation Systems (ITS) increasingly important. A comprehensive ITS, guided by intelligent algorithms, can select appropriate time periods and locations for traffic control, which is crucial for eliminating safety hazards in these scenarios. Traffic flow data is the foundation of various scheduling algorithms, and various data mining algorithms can be used to identify key information. Traffic flow is strongly coupled with time, with significant differences in traffic flow across different time periods. Therefore, traffic flow data is a natural time series. Furthermore, intersections are key nodes for traffic flow analysis, and intersections are strongly spatially coupled, so traffic flow data also exhibits spatial correlation. In summary, traffic flow data is a type of time series data with spatial correlation. Predicting traffic flow time series can provide guiding features for downstream processing algorithms.

[0003] In recent years, traffic flow prediction methods have primarily been based on deep learning approaches, focusing on temporal and spatial dimensions. These include feedforward neural networks, recurrent neural networks (RNNs), and convolutional neural networks (CNNs). These basic networks have undergone extensive research and adaptation, achieving promising results in traffic flow prediction. For example, models based on RNN structures like LSTM and GRU are widely used in traffic flow prediction. Recent research has largely focused on addressing the gradient issues and slow training speeds of LSTM and GRU models, such as the highly effective attention mechanism. The attention mechanism was developed to improve the accuracy of sequence-to-sequence (Seq2Seq) models. It assigns weights to features at different time steps by calculating similarity with a given metric (key), thereby enhancing the model's feature extraction capabilities. In addition to these methods, graph neural networks are also a common foundational approach for traffic flow modeling. Graph neural networks are a deep learning method for processing graph data structures. Compared to convolutional networks, which can only process spatial relationships in Euclidean space represented by two-dimensional matrices or raster images and ignore the importance of non-Euclidean pairwise relationships that are more suitable for describing road networks, graph neural networks are more suitable for describing road spatial relationships. Through methods such as prior knowledge and autonomous learning, a graph-structured adjacency matrix between traffic intersections can be obtained to describe the degree of correlation between different intersections, thereby explicitly modeling the spatial relationships of traffic flow data. Seo et al. proposed a graph convolutional recurrent network method that combines graph convolution with recurrent neural networks. Yu et al. proposed a graph convolutional neural network with a gating mechanism that retains the model's ability to capture long-term temporal correlations. To address the difficulty of constructing an adjacency matrix for traffic flow data graphs, StemGNN encodes traffic flow data using GRU units and then uses an attention mechanism to generate an adjacency matrix to construct a relationship graph between intersections for graph neural network modeling and extract features in the frequency domain.

[0004] But the above-mentioned methods all use the direct observation value of sequence as modeling object, ignoring the influence of sequence residual on the future trend of sequence, and the classical model for residual modeling is sliding average model (MA), and ARIMA model in combination with MA.But ARIMA model can only model univariate stationary sequence, and cannot be directly used for complex non-stationary, multivariate data sets.Therefore, in order to solve the above problems, the present invention proposes a kind of residual decomposition cyclic neural network (Residual Decomposition Recurrent Neural Network, RD-RNN) combining the autoregressive and sliding average of deep neural network as an alternative module of standard RNN. Summary of the Invention

[0005] For highway traffic control, in order to ensure that potential accidents can be eliminated in a timely manner, scientifically and effectively planning traffic control time and ensuring safe passage on highways have become urgent needs of highway operating companies. To achieve this goal, the present invention provides a traffic flow prediction method based on residual decomposition recurrent neural network.

[0006] In order to achieve the above technical objectives, the present invention provides the following technical solutions: a traffic flow prediction method based on residual decomposition recurrent neural network, comprising:

[0007] Acquire a road occupancy rate dataset at consecutive moments, wherein the road occupancy rate dataset at consecutive moments includes a training set and a test set;

[0008] Constructing a residual decomposition recurrent neural network model, inputting the training set into the residual decomposition recurrent neural network model for training, and obtaining a trained residual decomposition recurrent neural network model;

[0009] The test set is input into the trained residual decomposition recurrent neural network model to obtain the traffic flow time series prediction trend.

[0010] Optionally, the residual decomposition recurrent neural network model includes several layers of residual decomposition recurrent neural networks; wherein the residual decomposition recurrent neural network includes an encoding layer, an autoencoding layer and a prediction layer.

[0011] Optionally, the process of constructing the residual decomposition recurrent neural network model includes:

[0012] Constructing an autoregressive model, wherein the autoregressive model includes an autoregressive term and an error term;

[0013] Based on the autoregressive term, an autoregressive module is constructed using a Seq2Seq architecture;

[0014] Based on the error term, construct a residual decomposition module;

[0015] Constructing a residual decomposition recurrent neural network based on the autoregressive module and the residual decomposition module;

[0016] A plurality of the residual decomposition recurrent neural networks are stacked to obtain the residual decomposition recurrent neural network model.

[0017] Optionally, the autoregressive module includes an encoding layer and a decoding layer;

[0018] The formula for obtaining the autoregressive output result based on the autoregressive module is as follows:

[0019]

[0020] Where, is the input sequence, h t is the encoded high-dimensional hidden state, is the hidden state generated at every moment during the decoding process, is the output sequence, f enc is the encoder RNN nonlinear mapping function, f dec is the decoder RNN nonlinear mapping function, x out is the output sequence.

[0021] Optionally, the residual decomposition module includes a coding layer and an auto-coding layer;

[0022] The formula for obtaining the potential fitting residual in the hidden state based on the residual decomposition module is as follows:

[0023]

[0024] Where, is the encoded hidden state of the autoencoder at the first moment, is the encoded hidden state of the autoencoder at the q+1th moment, f aenc is the self-encoder mapping function, u is the residual sequence, is the restored sequence after reversal.

[0025] Optionally, the training process of the residual decomposition neural network model includes:

[0026] Step 1: The encoding layer of the first residual decomposition neural network receives the input sequence, and the encoding layer encodes the input sequence into a high-order hidden state matrix;

[0027] Step 2: The high-order hidden state matrix is ​​input into the autoencoder layer to obtain an inverted restoration sequence, and a first residual sequence is obtained based on the input sequence and the inverted restoration sequence;

[0028] At the same time, the high-order hidden state matrix is ​​input into the prediction layer to obtain a first output sequence;

[0029] Step 3: Input the first residual sequence into the encoding layer of the next residual decomposition neural network to obtain a second residual sequence and a second output sequence. Repeat steps 1 and 2 until the model performance is optimal and output the final prediction result.

[0030] The final prediction result is the sum of the output sequences of the prediction layers of several layers of residual decomposition neural networks.

[0031] Optionally, the mean square error method and the mean absolute error method are used to detect the performance of the model.

[0032] The present invention has the following technical effects:

[0033] Based on the basic ideas of long short-term memory neural network (LSTM) and autoregressive moving average (ARIMA), this paper proposes a residual decomposition recurrent neural network (RD-RNN) for road occupancy time series prediction. This method models the input road occupancy time series from the perspectives of autoregressive method and sliding average method, while considering the impact of direct observations of past time points and their own residuals on the future trend of the series. It can also consider the potential impact of high-order residuals by stacking the network depth. Thanks to the nonlinear expression ability of LSTM itself, compared with the ARIMA model, the model of this invention can handle non-stationary time series and multivariate time series. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 It is an autoregressive module based on Seq2Seq in an embodiment of the present invention;

[0036] Figure 2 : is a structural diagram of a residual decomposition module in an embodiment of the present invention;

[0037] Figure 3 It is a multi-layer stacked RD-RNN network structure in an embodiment of the present invention;

[0038] Figure 4 is the output result of each residual layer of the RD-RNN in the embodiment of the present invention;

[0039] Figure 5 Partial 1-step prediction results of intersection 85 in the embodiment of the present invention;

[0040] Figure 6 These are some 1-step prediction results for the intersection 177 in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0042] Example 1

[0043] 1. Constructing a residual decomposition recurrent neural network

[0044] 1.1 Autoregressive term

[0045] Autoregression is a common method for describing sequence data. It finds the correlation between the time points of the sequence itself in chronological order. The standard autoregression form is as follows:

[0046] x t =a1x t-1 +a2x t-2 +…+a p x t-p +u t

[0047] where χ t represents the value of the time series at time t, p is the autoregressive order, which represents how many historical lookback values ​​the model needs to use, and a p is the autoregressive coefficient, u t is the error term. That is, u t When the noise is white, the model is a pure p-order autoregressive model. Obviously, the standard autoregressive model is a linear model. The RNN model also retains autoregressive capabilities when used to process sequence data. The RNN can naturally model the order of the sequence and has strong nonlinear modeling capabilities. Therefore, this embodiment uses the RNN as the basic implementation of the autoregressive term.

[0048] This embodiment adopts the commonly used encoder-decoder (Seq2Seq) architecture for implementation. The module structure is as follows: Figure 1 shown.

[0049] Among them, the white part is the encoder module, and the black part is the decoder module. is the input sequence, The basic idea of ​​the module is that the decoder uses the hidden state information of the encoder to decode and achieve the purpose of autoregression. The mathematical form of this module can be described as follows:

[0050]

[0051]

[0052]

[0053] Among them, h t is the encoded high-dimensional hidden state, is the hidden state generated at time t during the decoding process, is the decoding result of the decoder at time t, as a result of the autoregressive term, f enc is the encoder RNN nonlinear mapping function, f dec is the decoder RNN nonlinear mapping function, and both are learned by gradient descent during the training process.

[0054] 1.2 Residual term

[0055] The error term in the above standard autoregressive model is usually difficult to meet the requirements of white noise in practical applications, which means that the error term still contains information that can be used to describe the autocorrelation relationship of the sequence. Therefore, this paper proposes a residual decomposition module to obtain the sequence error term and then conduct modeling analysis. The structure of this module is as follows Figure 2 shown.

[0056] The white part is the encoder module, and the black part is the autoencoder module. In order to obtain the residual term of the original sequence, this module reconstructs the original sequence through the RNN autoencoder structure to obtain the potential fitting residual in the hidden state. The mathematical form of this part is as follows:

[0057]

[0058] Among them, h t Also the hidden state of the encoder, is the encoded hidden state of the autoencoder at the first moment, is the encoded hidden state of the autoencoder at the q+1th moment. Unlike the autoregressive layer, the encoding order of the autoencoder is opposite to the decoding order of the autoregressive layer. This is because the encoding process follows the time sequence from moment 1 to moment t, and h t The sequence information contained is also from 1 to t, so in order to restore the input sequence, the autoencoder needs to decode in the reverse order. and the reversed restored sequence Subtracting them, we can obtain the first-order residual sequence for the next module to model the residual sequence, and analyze the high-order residuals of the sequence by stacking multiple layers.

[0059] 1.3 Residual Decomposition Recurrent Neural Network

[0060] Combining the autoregressive term with the residual term yields a complete module. This module can construct a deep network by stacking multiple layers, completing the autoregressive and residual regression tasks for the sequence layer by layer. The stacked network structure is as follows: Figure 3 As shown:

[0061] Figure 3 The figure shows an n+1 layer stacked RD-RNN network, each layer contains three groups of submodules, namely encoding layer, self-encoding layer and prediction layer. The encoding layer of the first layer receives the original sequence input and encodes the sequence into a high-order hidden state matrix h t , h t It is used in both the autoencoding layer and the prediction layer. The autoencoding layer uses h t Reconstruct the original sequence to obtain And obtain the residual term of this layer through the residual connection with the input of this layer, and the prediction layer uses h t Make predictions The output of each layer consists of two items, the first of which is the prediction sequence of the current layer The second item is the residual sequence of the current layer The residual sequence is used as the input of the next layer, and the above process is repeated. Finally, the prediction sequence of each layer is added as the final prediction result.

[0062] The network constructed in this way implements basic autoregressive prediction in the first layer. As the number of layers increases, the network gradually predicts the higher-order residuals of the sequence, allowing the model to model both the sequence itself and the residuals. The next section will verify the proposed method through experiments.

[0063] 2. Datasets and metrics

[0064] This example is experimentally verified based on the public Traffic dataset, which comes from the California Department of Transportation. It contains road occupancy rates at different intersections recorded by 862 sets of sensors from July 1, 2016, to July 2, 2018, recorded every hour, for a total of 17,544 data points. During the experiment, we used the first 70% of the data, a total of 12,184 data points, as the training set, 70%-80% of the data as the validation set, and the last 20% of the data as the test set to verify the model's ability to predict future road conditions. The experimental results use mean squared error (MSE) and mean absolute error (MAE) to measure the performance of the model:

[0065]

[0066]

[0067] 3. Verification of Method Validity

[0068] To demonstrate the advantages of the present invention's method over RNN architectures, this example was compared with two commonly used RNN-based models for traffic flow prediction: a Seq2Seq-based LSTM network and a Conv-LSTM network that considers spatiotemporal correlations. Furthermore, benefiting from the flexibility of the RD-RNN architecture, the present invention replaced the LSTM in the Conv-LSTM network with the RD-RNN of the present invention, thereby introducing a Conv-RD-RNN model for comparison. The model parameter configurations used during the experiment are shown in Table 1. LR represents the initial learning rate. During training, the learning rate is halved after every four iterations to smooth the training process. ITL represents the input time step length, indicating the maximum length of the autoregressive term that can be used in the model. Dim represents the number of LSTM hidden units in each layer and also represents the encoding matrix dimension. Under this configuration, all models were trained and evaluated on the validation set. Each model was trained at different prediction lengths. The validation results are shown in Table 2.

[0069] Table 1

[0070] Model BatchSize Epoch EarlyStop LR ITL Layers Dim RD-RNN 32 40 3 0.001 30 4 512 Conv-RD-RNN 32 40 3 0.001 30 5 512 LSTM 32 40 3 0.001 30 4 512 Conv-LSTM 32 40 3 0.001 30 5 512

[0071] Table 2

[0072]

[0073] This example performs one-step prediction (1 hour), two-step prediction (2 hours), three-step prediction (3 hours), and four-step prediction (4 hours). The results clearly show that RD-RNN significantly improves performance over LSTM at different prediction lengths. The MAE and MSE indicators decrease by an average of 4.7% and 6.8% across the four prediction lengths. Furthermore, compared to the Conv-LSTM, which incorporates spatial features, RD-RNN achieves even better performance at some prediction lengths. The Conv-RD-RNN, which incorporates spatial features, achieves the best performance of all models, with MAE and MSE decreasing by an average of 5.3% and 5.8% compared to the Conv-LSTM. This demonstrates the superiority of RD-RNN over LSTM for short-term traffic flow time series prediction.

[0074] 4. Results Visualization and Analysis

[0075] This embodiment deeply analyzes the role played by each layer of the RD-RNN proposed in the present invention in prediction, and shows the output results of the residual modules of each layer in the form of a line graph, as shown in Figure 4 shown. Figure 4The figure shows some experimental results of the validation set of RD-RNN on the Traffic dataset. In the line graph, the dotted line represents the output of the residual module of a certain layer, and the dotted curve represents the input sequence actually received by the layer. From left to right, the figure shows the input and residual module output of layers 1 to 4 respectively. The rightmost side shows the sum of the residual module output of each layer and the original input sequence, which is used to observe the reconstruction capability of the model after summing up each layer.

[0076] Depend on Figure 4 It is clear that the combination of multiple layers of residual modules can accurately fit the original input sequence, ensuring that the hidden features captured by the encoding layer are fully utilized. Note that the reconstruction effect of the residual modules in the first layer is poor, but as the number of layers increases, the model's reconstruction ability continues to improve. Therefore, when modeling only the original sequence, the features that the model can capture are very limited. By introducing residual modules to analyze the high-order residuals of the sequence, the model's efficiency in utilizing the encoding layer features can be greatly improved.

[0077] This is also reflected in the prediction results, such as Figure 5 、 Figure 6 shown. Figure 5 and Figure 6 The following figure shows the 1-step prediction results of four models at different intersections in the Traffic dataset. The dashed lines in the figure represent the model's predictions, the line graph is derived from multiple iterations of the model's predictions, and the dotted and curved lines represent the observed values ​​in the dataset. As shown in the figure, the ellipse marks a mutation point in both figures. Both the original RD-RNN model and the Conv-RD-RNN combined with the RD-RNN accurately predict the mutation point. However, the prediction results of the LSTM and ConvLSTM are too low-frequency to capture this hidden trend, ignoring the information in some encoded features.

[0078] This paper addresses the problem of time series prediction of traffic occupancy rates. Based on a recurrent neural network (RNN), a residual decomposition recurrent neural network (RD-RNN) based on autoregression and residual decomposition is proposed. The model consists of an encoding module, an autoregressive module, and a residual decomposition module. The encoding module encodes the input time series into a high-dimensional latent space, representing it as a hidden state. The autoregressive module then uses this hidden state for prediction, iteratively generating prediction results. The residual decomposition module uses an autoencoder to restore this hidden state to the input sequence as closely as possible, forcing the hidden state generated by the encoding module to contain more usable information. The residual of the sequence is obtained through a residual connection with the input sequence for further analysis. By stacking multiple layers, high-order residual terms of a sequence can be quickly obtained, and the residual is used to further refine the prediction results, improving the performance of the LSTM structure for time series problems. Comparisons with the currently commonly used basic models LSTM and ConvLSTM on the open source Traffic dataset and the improvement of ConvLSTM to Conv-RD-RNN fully demonstrate the effectiveness of the present method in improving the accuracy of traffic flow time series prediction.

[0079] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A traffic flow prediction method based on residual decomposition recurrent neural network, characterized in that: include: Acquire a road occupancy rate dataset at consecutive moments, wherein the road occupancy rate dataset at consecutive moments includes a training set and a test set; Constructing a residual decomposition recurrent neural network model, inputting the training set into the residual decomposition recurrent neural network model for training, and obtaining a trained residual decomposition recurrent neural network model; The residual decomposition recurrent neural network model includes several layers of residual decomposition recurrent neural networks; wherein the residual decomposition recurrent neural network includes an encoding layer, an autoencoding layer and a prediction layer; The construction process of the residual decomposition recurrent neural network model includes: constructing an autoregressive model, wherein the autoregressive model includes an autoregressive term and an error term; based on the autoregressive term, an autoregressive module is constructed using a Seq2Seq architecture; based on the error term, a residual decomposition module is constructed; based on the autoregressive module and the residual decomposition module, a residual decomposition recurrent neural network is constructed; and a plurality of the residual decomposition recurrent neural networks are stacked to obtain the residual decomposition recurrent neural network model.

2. The traffic flow prediction method based on residual decomposition recurrent neural network according to claim 1 is characterized by: The autoregressive module includes an encoding layer and a decoding layer; The formula for obtaining the autoregressive output result based on the autoregressive module is as follows: Where, is the input sequence, t=1,...n, h t is the encoded high-dimensional hidden state, is the hidden state generated at every moment during the decoding process, is the output sequence, f enc is the encoder RNN nonlinear mapping function, f dec is the decoder RNN nonlinear mapping function, is the output sequence.

3. The traffic flow prediction method based on residual decomposition recurrent neural network according to claim 2 is characterized by: The residual decomposition module includes a coding layer and an auto-coding layer; The formula for obtaining the potential fitting residual in the hidden state based on the residual decomposition module is as follows: Where, is the encoded hidden state of the autoencoder at the first moment, is the encoded hidden state of the autoencoder at the q+1th moment, f aenc is the self-encoder mapping function, u is the residual sequence, is the restored sequence after reversal.

4. The traffic flow prediction method based on residual decomposition recurrent neural network according to claim 3 is characterized by: The training process of the residual decomposition recurrent neural network model includes: Step 1: The encoding layer of the first-layer residual decomposition recurrent neural network receives the input sequence, and the encoding layer encodes the input sequence into a high-order hidden state matrix; Step 2: The high-order hidden state matrix is ​​input into the autoencoder layer to obtain an inverted restoration sequence, and a first residual sequence is obtained based on the input sequence and the inverted restoration sequence; At the same time, the high-order hidden state matrix is ​​input into the prediction layer to obtain a first output sequence; Step 3: Input the first residual sequence into the encoding layer of the next residual decomposition recurrent neural network to obtain a second residual sequence and a second output sequence. Repeat steps 1 and 2 until the model performance is optimal and output the final prediction result. The final prediction result is the sum of the output sequences of the prediction layers of several layers of residual decomposition recurrent neural networks.

5. The traffic flow prediction method based on residual decomposition recurrent neural network according to claim 1 is characterized by: The mean square error method and the mean absolute error method are used to test the performance of the residual decomposition recurrent neural network model.

Images