A road traffic flow prediction method based on stacked regression

Abstract

The present invention discloses a road traffic flow prediction method based on stacked regression, comprising obtaining a public data set, performing preprocessing and feature extraction, dividing a feature sequence into a training set and a test set as input for a subsequent model; constructing a Bi‑GRU model and a GBR model, setting model parameters corresponding to the two models, and completing the construction of the two models; using the Bi‑GRU model and the GBR model as basic model units, using the ridge algorithm as a meta-model, using the stacked regression algorithm to build a hybrid model, and then centralizing the model; training the hybrid model and implementing weight division to predict model data; verifying the results of the model prediction and detecting their accuracy. The present invention utilizes the stacked regression algorithm to stack the GBR and Bi‑GRU models with the ridge algorithm as a meta-model, trains and predicts the hybrid model, and can improve the precision and accuracy of traffic flow prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic flow prediction, and in particular relates to a road traffic flow prediction method based on stacking regression. Background Art

[0002] Traffic flow forecasting is one of the most important issues in the transportation sector, crucial for real-time traffic flow adjustments, traffic planning, road design, and traffic management and control. Over the past few decades, traffic flow forecasting has evolved from traditional time series methods (such as ARIMA) to machine learning approaches. In recent years, with the rise of deep learning technology, traffic flow forecasting based on neural networks has also garnered widespread attention. With the continuous advancement of forecasting methods, modern traffic flow forecasting has diversified and integrated, evolving from periodic forecasting to trend forecasting and finally to traffic flow forecasting based on spatiotemporal big data.

[0003] Deep learning (DL) and graph neural networks (GNNs) have achieved tremendous success in many diverse fields thanks to their remarkable capabilities in feature engineering and function fitting. Several researchers have investigated the application of various DL algorithms and GNNs to improve traffic flow prediction performance, which has become a dominant paradigm in this field. Inspired by these research findings, researchers worldwide have constructed traffic flow prediction models based on deep learning algorithms such as LSTM, GRU, CNN, and GBR. However, the complexity of traffic flow data leads to high dimensionality and uncertainty. This data typically contains extensive time series information, spatial correlation, and the influence of various external factors, such as weather and road construction. Therefore, using only a single algorithm for prediction can lead to improper data processing, failure to extract key feature sequences, and inadequate or redundant feature representation. Therefore, for traffic flow data, multiple models can be used to capture characteristics such as time series, spatial correlation, and external factors. By combining the outputs of these models, a richer feature representation can be obtained, thereby improving prediction performance.

[0004] Purpose of the invention: In response to the problems pointed out in the background technology, the present invention provides a road traffic flow prediction method based on stacking regression, aiming to solve the current technical problems of low accuracy and stability of traffic flow prediction.

[0005] Technical solution: The present invention proposes a road traffic flow prediction method based on stacked regression, comprising the following steps:

[0006] S1. Obtain historical traffic flow statistics as a public dataset, perform preprocessing and feature extraction, and divide the feature sequence into training and test sets as input for subsequent models;

[0007] S2. Build a bidirectional gated recurrent unit (Bi-GRU) model and a gradient boosting regression (GBR) model. First, set the corresponding model parameters of the two models, and then complete the construction of the two models based on the corresponding parameters.

[0008] S3. Using the Bi-GRU model and the GBR model as basic model units, the ridge algorithm is used as a meta-model, and the stacked regression algorithm is used to build a hybrid model, and the model is centralized;

[0009] S4. Training the hybrid model generated by the stacking algorithm and performing weight division to achieve prediction of the model data;

[0010] S5. Verify the model prediction results and test their accuracy.

[0011] Furthermore, the specific steps of S1 are as follows:

[0012] S1.1. Obtain historical traffic flow statistics and store the dataset in traffic.csv;

[0013] S1.2. Read data from the file 'traffic.csv' and parse the data using '\t' as the delimiter. Set the random seed to ensure the reproducibility of the experiment.

[0014] S1.3. Preprocess the data by filling missing values ​​with the median, standardize the features to have a mean of 0 and a variance of 1, then standardize the target values ​​in the same way as the features, and finally add quadratic features to fit a curvilinear relationship in the model.

[0015] S1.4. Convert the features in the dataset into a feature matrix: X = [X1, X2, X3, ... X t ], where X represents the features extracted from the dataset, X t Represents the feature sequence of the input at time t;

[0016] S1.5. Divide the feature sequence into a training set and a test set.

[0017] Furthermore, the specific steps of S2 are as follows:

[0018] S2.1, Bi-GRU neural network structure of bidirectional gated recurrent unit, set its number of neurons and activation function, and then create a Bi-GRU model. The Bi-GRU model includes input layer, bidirectional GRU layer, merging layer, and output layer. The bidirectional GRU layer includes front and back GRU layers. The parameters of the front and back GRU layers are the same. Its operation formula is as follows:

[0019] Z t =σ(W z x t +U z h t-1 +b z )

[0020] r t =σ(W r x t +U r h t-1 +b r )

[0021]

[0022]

[0023] Among them, x t is the tth time step of the input feature sequence, h t-1 is the hidden state of the previous time step, r t 、Z t are update gate and reset gate respectively, h t ' is the candidate hidden state, h t is the current hidden state, W z 、U z 、b z 、W r 、U r 、b r 、W h 、U h 、b h are the weights and biases of the model;

[0024] S2.2, Gradient Boosting Regression (GBR) iterates the decision tree structure, sets the number of decision trees, decision tree depth, learning rate, and loss function, and then creates a GBR model. The GBR model uses the gradient boosting regression algorithm:

[0025] F t (x,h t-1 )=F t-1 (x,h t-1 )+λ t *g t (x,h t-1 )

[0026] Among them, F t-1 (x,h t-1 ) is the prediction result of the previous time step, λ t is the step size (learning rate), g t (x,h t-1) is the regression prediction result of the GBR model at time step t.

[0027] Furthermore, the specific steps of S3 are as follows:

[0028] S3.1. Create Bi-GRU model and GBR model to facilitate the creation of subsequent hybrid models;

[0029] S3.2. The stacked regression model consists of a Bi-GRU model and a GBR model, and uses the ridge regression model as a meta-model to splice the two models into a hybrid model:

[0030] F(x,h0)=ridge[F0(x)+λ1*g1(x,h0)+λ2*g2(x,h1)+......+λ t *g t (x,h t-1 )

[0031] Among them, F0(x) is the initial predicted value, h0 is the initial hidden state, λ1,λ2......λ t is the step size of each time step.

[0032] Furthermore, the specific steps of S4 are as follows:

[0033] S4.1. First, cross-validation is performed to test the performance of the GBR model, Bi-GRU model, and hybrid model using the cross-validation algorithm.

[0034] S4.2. Train the hybrid model, then use the trained model to predict the data, and then use the GBR model and Bi-GRU model to predict the data, perform weight processing on the predicted results, and finally obtain the final prediction results.

[0035] Furthermore, the weight processing in S4.2 is specifically as follows:

[0036] Assume there are n base models, the prediction results are y_pred_1,y_pred_2,…,y_pred_n, w_1,w_2,…,w_n are the weights of each base model, satisfying w_1+w_2+…+w_n=1;

[0037] The mathematical formula for using weights to adjust prediction results is:

[0038] y_pred_weighted=w_1*y_pred_1+w_2*y_pred_2+…+w_n*y_pred_n

[0039] Among them, y_pred_weighted is the final prediction result of the stacking model.

[0040] Beneficial effects:

[0041] The present invention is based on an existing public data set for traffic flow, and uses a bidirectional gated recurrent unit (Bi-GRU) model and a gradient boosting regression (GBR) model as basic model units. The ridge regression algorithm is used as a meta-model, and the stacked regression algorithm is used to fuse the two basic model units into a hybrid model. The hybrid model is then used to predict traffic data. After standard preprocessing of the data, the present invention combines the GBR model with the Bi-GRU model using the ridge algorithm using the stacked regression algorithm. The GBR model can be used to process nonlinear data in the public data set, while the Bi-GRU model processes sequence data. By combining the two models, the robustness and generalizability of the hybrid model are improved, thereby improving the accuracy of data prediction, while reducing the computational pressure of the model and improving the operational efficiency of the model.

[0042] The GBR model of the present invention can more effectively process nonlinear data sets by creating a decision tree model. The Bi-GRU model is a variant of the LSTM model. The two models are similar in effect, but the Bi-GRU model has a lighter structure than the LSTM model, so it has certain advantages in terms of operation. Because the Bi-GRU model does not need to maintain the complex mechanisms such as input gates, forget gates, and output gates in the LSTM, it can reduce the computational complexity of the model, thereby increasing the running speed. This is of great significance for tasks that require efficient processing of large amounts of data, such as traffic flow prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The overall flow chart of traffic flow prediction based on GBR and Bi-GRU;

[0044] Figure 2 Flowchart of the preprocessing method for public datasets;

[0045] Figure 3 This is the single-layer GRU structure diagram of the Bi-GRU model;

[0046] Figure 4 Build a flow chart for the GBR model;

[0047] Figure 5 This is the structure diagram of the stacked regression model;

[0048] Figure 6 This is the K-fold cross validation flowchart;

[0049] Figure 7 is the bipartite graph of the errors of each model.

[0050] Specific implementation steps

[0051] The present invention will be further illustrated below with reference to specific embodiments based on public transportation data in Baoshan District, Shenzhen. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims appended to this application.

[0052] The present invention discloses a road traffic flow prediction method based on stacking regression, and a specific example is as follows:

[0053] S1: The present invention obtains the public data set of "Bao'an District Traffic Flow Statistics" from the Shenzhen Traffic Safety Official Website, performs preprocessing and feature extraction, and divides the feature sequence into a training set and a test set as the input of the subsequent model, as shown in the following example. Figure 2 As shown:

[0054] S1.1: Obtain the public dataset of "Bao'an District Traffic Flow Statistics" from the Shenzhen Traffic Safety Official Website and save the dataset to traffic.csv.

[0055] S1.2: Read data from the file 'traffic.csv' and parse the data using '\t' as the delimiter. Also, set the random seed to ensure the reproducibility of the experiment.

[0056] S1.3: Preprocess the data by filling missing values ​​(if any) with the median, standardize the features so that their mean is 0 and variance is 1, and then standardize the target value to make it the same as the standardization of the features. Finally, add the quadratic feature to fit the curvilinear relationship in the model.

[0057] S1.4: Convert the features in the dataset into a feature matrix: X = [X1, X2, X3, ... X t ]

[0058] Among them, X represents the features extracted from the dataset, X t Represents the feature sequence of the input at time t.

[0059] S1.5: Divide the feature sequence into training sets and test sets to facilitate subsequent model practice and testing.

[0060] S2: Build the Bi-GRU (bidirectional gated recurrent unit) model and the GBR (gradient boosting regression model) model. First, set the model parameters corresponding to the two models, and then complete the construction of the two models based on the corresponding parameters.

[0061] S2.1. Based on the Bi-GRU (bidirectional gated recurrent unit) neural network structure, the number of neurons and activation function are set to create a Bi-GRU regression model. The Bi-GRU model consists of an input layer, a bidirectional GRU layer, a merging layer, and an output layer. The bidirectional GRU layer includes the front and back GRU layers. The parameters set for the front and back GRU layers are the same, including the reset gate, update gate, hidden state and other parameters. Figure 3 As shown:

[0062] Assume that the current time step is t and the input of the Bi-GRU model is x t , the hidden state is h t .

[0063] Reset gate calculation formula: r t =σ(W r *x t +U r *h t-1 +b r )

[0064] Among them, W r and U r The input x is t and hidden state h t-1 and the weight matrix of the reset gate, b r is the bias vector and σ is the sigmoid function.

[0065] The reset gate determines whether the past hidden state affects the calculation of the current time step. It controls the degree of mixing of past information in the current time step, thereby affecting the model's memory and forgetting ability.

[0066] Update gate calculation formula: Z t =σ(W z *x t +U z *h t-1 +b z )

[0067] Among them, W z and U z The input x is t and hidden state h t-1 and the weight matrix of the update gate, b z is the bias vector and σ is the sigmoid function.

[0068] By setting the update gate, we can control the degree of mixing between the input of the current time step and the hidden state of the previous time step, and decide which information needs to be updated and retained.

[0069] Candidate hidden state calculation formula:

[0070] Among them, W h is the input x t With the candidate hidden state h t 'weight matrix, r t is the output of the reset gate, It represents element-wise multiplication, and tanh is the hyperbolic tangent function.

[0071] The hidden state calculation formula for the current time step is:

[0072] Among them, Z t is the output of the update gate, This formula can be seen as the hidden state h of the previous moment. t-1 Perform a decay, leaving behind a "memory" related to the current input, and combine this "memory" with the current input to get a new hidden state h t .

[0073] By setting the reset gate and update gate of the Bi-GRU (bidirectional gated recurrent unit) model, through the setting of the two gates, by inputting the data set and calling the formula, the information status of the current data set and the information status of the previous data can be obtained, which facilitates the use of subsequent models and the calling of GBR (gradient boosting regression model).

[0074] S2.2, based on the GBR (gradient boosting regression) iterative decision tree structure, set the number of decision trees, decision tree depth, learning rate and loss function, and then create a GBR regression model. The model uses the gradient boosting regression algorithm, as follows Figure 4 As shown:

[0075] Assume that y i is the true label of the i-th sample, f m (x i ) is the predicted value of the mth decision tree, then the output f(x i ) can be expressed as:

[0076]

[0077] Where M is the number of trees, γ m is the weight of the mth tree.

[0078] The main goal of GBR training is to minimize the loss (loss function). Commonly used loss functions include mean square error (MSE) and mean absolute error (MAE). The mathematical formula of the loss function is as follows:

[0079]

[0080] Among them, L(y i ,f(x i )) is the loss function, and n is the number of samples.

[0081] At the same time, the model uses the gradient boosting regression algorithm:

[0082] F t (x,h t-1 )=F t-1 (x,h t-1 )+λ t *g t (x,h t-1 )

[0083] Among them, F t-1 (x,h t-1 ) is the prediction result of the previous time step, λ t is the step size (learning rate), g t (x,h t-1 ) is the regression prediction result of the GBR model at time step t.

[0084] The gradient boosting algorithm can obtain the prediction results of the current data set through the GBR (gradient boosting regression) calculation formula and the current state information of the data obtained by the Bi-GRU model.

[0085] S3, taking the Bi-GRU model and GBR model as the basic model units, using the ridge algorithm as the meta-model, using the stacked regression algorithm to build a hybrid model, and then centralizing the model, as follows Figure 5 As shown:

[0086] S3.1. Create Bi-GRU and GBR pipeline sub-models to facilitate the creation of subsequent hybrid models;

[0087] S3.2. The stacked regression model consists of a Bi-GRU model and a GBR model, and uses the ridge regression model as a meta-model to splice the two models into a hybrid model:

[0088] F(x,h0)=ridge[F0(x)+λ1*g1(x,h0)+λ2*g2(x,h1)+......+λ t *g t (x,h t-1 )

[0089] Among them, F0(x) is the initial predicted value, h0 is the initial hidden state. t is the step size of each time step.

[0090] S4. Train the hybrid model generated by the stacking algorithm and perform weight division to achieve prediction of model data:

[0091] S4.1. First, perform K-fold cross validation and use the cross validation algorithm to test the performance of GBR, Bi-GRU and hybrid models. The specific process is as follows: Figure 6 As shown:

[0092] Assume there are n observations and divide them into K groups. The size of each group is n / K.

[0093] 1. Define K-Fold cross validation:

[0094] Divide the dataset into K groups, each of size n / K. For k = 1, 2, ..., K, loop through the following steps:

[0095] (1) Use the kth group of data as the test set and the remaining K-1 groups of data as the training set.

[0096] (2) Train the model on the training set.

[0097] (3) Make predictions on the test set and calculate the prediction error.

[0098] 2. Calculate the cross validation error:

[0099] For each k=1,2,…,K, record the k-th prediction error, average the K prediction errors, and get the cross-validation error.

[0100] The cross validation error can be expressed mathematically as follows:

[0101] Cross validation error = (1 / K)*∑(E k )

[0102] Among them, E k represents the k-th prediction error, and ∑ is the summation symbol indicating the summation of all k = 1, 2, 3..., K.

[0103] By adopting the K-fold cross-validation algorithm, the prediction errors of the Bi-GRU model, GBR model and hybrid model are obtained to further judge the quality of the model.

[0104] S4.2. Then train the hybrid model, and then use the trained model to predict the data. Then use the GBR and Bi-GRU models to predict the data, and perform weight processing on the predicted results to obtain the final prediction results. The specific process is as follows:

[0105] Assume there are n base models, the prediction results are y_pred_1,y_pred_2,…,y_pred_n, w_1,w_2,…,w_n are the weights of each base model, satisfying w_1+w_2+…+w_n=1.

[0106] The mathematical formula for using weights to adjust prediction results is:

[0107] y_pred_weighted=w_1*y_pred_1+w_2*y_pred_2+…+w_n*y_pred_n.

[0108] Among them, y_pred_weighted is the final prediction result of the stacking model.

[0109] Through the weight adjustment algorithm, the final prediction result of the hybrid model can be obtained.

[0110] S5. Verify the model prediction results and test their accuracy to determine whether the model can improve the accuracy of traffic flow prediction:

[0111] After obtaining the prediction results through the model, the library functions are used to judge the accuracy of the prediction results, including performance indicators such as RMSE, MSE, MAE, R square value, accuracy, etc., and the real value and the predicted value are displayed in an image using the dichotomy method. The specific process is as follows:

[0112] 1. Accuracy: Accuracy = (TP + TN) / (TP + TN + FP + FN)

[0113] 2. Recall: Recall = TP / (TP + FN)

[0114] 3.F1 value (F1-Score): F1=2*(Precision*Recall) / (Precision+Recall)

[0115] 4. Mean Square Error (MSE): MSE = (1 / n)*Σ(y_true-y_pred)^2

[0116] 5. Mean absolute error (MAE): MAE = (1 / n)*Σ|y_true-y_pred|

[0117] 6.R-squared: R^2=1-(SSR / SST)

[0118] 7. Root Mean Square Error (RMSE): RMSE = sqrt((1 / n)*Σ(y_true - y_pred)^2)

[0119] Where TP represents the number of true positive samples, TN represents the number of true negative samples, FP represents the number of false positive samples, and FN represents the number of false negative samples. y_true represents the true value, y_pred represents the predicted value of the model, and n represents the number of samples.

[0120] These performance indicators can be used to determine the quality of the model and the accuracy of the model's prediction results. The quality of this model can be determined by comparing it with several other models commonly used for traffic flow prediction.

[0121] From the comparison results in Table 1, it can be seen that this model is superior to other models in terms of these 8 performance indicators. Therefore, the accuracy of traffic flow prediction can be improved by using this model. Compared with other models, the accuracy of this model (a hybrid model of GBR and Bi-GRU with ridge algorithm as the meta-model) is 1% higher. At the same time, the RMSE, MSE, MAE and R-squared values ​​of this model are all optimal. Therefore, this model can improve the accuracy of the prediction results, as shown in Table 1. At the same time, the error between the predicted value and the true value is displayed by using the dichotomy method, as shown in Figure 7 shown.

[0122] Table 1 Comparison results of model performance indicators

[0123]

[0124]

[0125] The present invention creatively proposes a road traffic flow prediction method based on stacking regression, which uses a stacking algorithm with a ridge model as a meta-model to form a hybrid model, and realizes the prediction of traffic flow data through the hybrid model.

[0126] The road traffic flow prediction method based on stacked regression proposed in the present invention can be used not only in the field of traffic flow prediction, but also in other data prediction fields.

[0127] The above description is merely an example of the present invention and does not limit the present invention. Any equivalent substitutions made within the principles of the present invention should be included in the scope of protection of the present invention. Any content not elaborated in detail in the present invention belongs to the prior art known to those skilled in the art.

Claims

1. A road traffic flow prediction method based on stacked regression, characterized in that: The steps include: S1. Obtain historical traffic flow statistics as a public dataset, perform preprocessing and feature extraction, and divide the feature sequence into training and test sets as input for subsequent models; S2. Build a bidirectional gated recurrent unit (Bi-GRU) model and a gradient boosting regression (GBR) model. First, set the corresponding model parameters of the two models, and then complete the construction of the two models based on the corresponding parameters. S2.1, Bi-GRU neural network structure of bidirectional gated recurrent unit, set its number of neurons and activation function, and then create a Bi-GRU model. The Bi-GRU model includes input layer, bidirectional GRU layer, merging layer, and output layer. The bidirectional GRU layer includes front and back GRU layers. The parameters of the front and back GRU layers are the same. Its operation formula is as follows: Z t =σ(W z x t +U z h t-1 +b z ) r t =σ(W r x t +U r h t-1 +b r ) Where σ is the sigmoid function, x t is the tth time step of the input feature sequence, h t-1 is the hidden state of the previous time step, r t 、Z t are reset gate and update gate respectively, h t ' is the candidate hidden state, h t is the current hidden state, W z and U z The input x is t and hidden state h t-1 and the weight matrix of the update gate, b z is the bias vector, W h is the input x t With the candidate hidden state h t 'The weight matrix, W r and U r The input x is t and hidden state h t-1 and the weight matrix of the reset gate, b r is the bias vector, U h 、b h are the weights and biases of the model, It represents element-wise multiplication, and tanh is the hyperbolic tangent function; S2.2, Gradient Boosting Regression (GBR) iterates the decision tree structure, sets the number of decision trees, decision tree depth, learning rate, and loss function, and then creates a GBR model. The GBR model uses the gradient boosting regression algorithm: F t (x,h t-1 )=F t-1 (x,h t-1 )+λ t *g t (x,h t-1 ) Among them, F t-1 (x,h t-1 ) is the prediction result of the previous time step, λ t is the step size (learning rate), g t (x,h t-1 ) is the regression prediction result of the GBR model at time step t; S3. Using the Bi-GRU model and the GBR model as basic model units, the ridge algorithm is used as a meta-model, and the stacked regression algorithm is used to build a hybrid model, and the model is centralized; S3.

1. Create Bi-GRU model and GBR model to facilitate the creation of subsequent hybrid models; S3.

2. The stacked regression model consists of a Bi-GRU model and a GBR model, and uses the ridge regression model as a meta-model to splice the two models into a hybrid model: F(x,h0)=ridge[F0(x)+λ1*g1(x,h0)+λ2*g2(x,h1)+......+λ t *g t (x,h t-1 ) Among them, F0(x) is the initial predicted value, h0 is the initial hidden state, λ1,λ2......λ t is the step size of each time step; S4. Training the hybrid model generated by the stacking algorithm and performing weight division to achieve prediction of the model data; S5. Verify the model prediction results and test their accuracy.

2. The method for predicting road traffic flow based on stacked regression according to claim 1, characterized in that: The specific steps of S1 are as follows: S1.

1. Obtain historical traffic flow statistics and store the dataset in traffic.csv; S1.

2. Read data from the file 'traffic.csv' and parse the data using '\t' as the delimiter. Set the random seed to ensure the reproducibility of the experiment. S1.

3. Preprocess the data by filling missing values ​​with the median, standardize the features to have a mean of 0 and a variance of 1, then standardize the target values ​​in the same way as the features, and finally add quadratic features to fit a curvilinear relationship in the model. S1.

4. Convert the features in the dataset into a feature matrix: X=[X1,X2,X3,......X t ], where X represents the features extracted from the dataset, X t Represents the feature sequence of the input at time t; S1.

5. Divide the feature sequence into a training set and a test set.

3. The method for predicting road traffic flow based on stacked regression according to claim 1, characterized in that: The specific steps of S4 are as follows: S4.

1. First, cross-validation is performed to test the performance of the GBR model, Bi-GRU model, and hybrid model using the cross-validation algorithm. S4.

2. Train the hybrid model, then use the trained model to predict the data, and then use the GBR model and Bi-GRU model to predict the data, perform weight processing on the predicted results, and finally obtain the final prediction results.

4. The method for predicting road traffic flow based on stacked regression according to claim 3, characterized in that: The weight processing in S4.2 is as follows: Assume there are n base models, the prediction results are y_pred_1,y_pred_2,…,y_pred_n, w_1,w_2,…,w_n are the weights of each base model, satisfying w_1+w_2+…+w_n=1; The mathematical formula for using weights to adjust prediction results is: y_pred_weighted=w_1*y_pred_1+w_2*y_pred_2+…+w_n*y_pred_n Among them, y_pred_weighted is the final prediction result of the stacking model.

Images