Traffic flow prediction method of block mixed structure

Abstract

The invention discloses a traffic flow prediction method of a block mixed structure, and the method sequentially comprises the following steps: firstly, constructing a traffic characteristic sequence data input, and dividing an original sequence into overlapped time blocks; then, a deep convolution module is utilized to extract local features in each block, and a global dependency relationship between time blocks is modeled through a Transform encoder; then, cross-block feature fusion is carried out on the features through a point-by-point convolution module, and linear trend and nonlinear dynamic prediction results are generated through double prediction heads; experimental results show that the method is excellent in performance in traffic flow prediction tasks, and has the advantages of efficient processing of long-time sequences, low prediction error, high calculation efficiency and strong generalization ability.

Description

Technical Field

[0001] The present invention is a block-mixed structured traffic flow prediction method, which involves the intersection of artificial intelligence and traffic science and is suitable for high-precision prediction of traffic flow. The method models traffic data and, through adaptive feature adjustment and multi-level feature extraction, can play an important role in traffic scenarios that require accurate predictions, such as urban planning, traffic management, and congestion warning, and can effectively improve the timeliness and accuracy of traffic predictions. Background Art

[0002] Traditional traffic congestion prediction methods often rely on simple linear models or basic time series analysis methods (such as ARIMA), which show great limitations when processing complex traffic data. Especially when processing multi-dimensional and highly dynamic traffic data, it is impossible to effectively capture long-term dependencies, which limits the accuracy and real-time performance of predictions. In recent years, the rise of deep learning technology has made traffic prediction possible, but existing models still have problems such as insufficient feature extraction and incomplete capture of long-term dependencies. In order to solve these shortcomings, this paper proposes a traffic flow prediction method with a block hybrid structure, which combines block and structure hybrid to improve the accuracy and stability of traffic data prediction. The block hybrid structure divides the feature matrix into multiple local feature blocks and then processes them by fusing depthwise separable convolution with Transformer encoder. Among them, the depthwise separable convolution consists of depthwise convolution and pointwise convolution. The depthwise convolution is used to extract spatial features, and the pointwise convolution is used to extract channel features. The depthwise separable convolution can not only realize multi-scale feature extraction, but also retain key feature information using residual connections, thereby enhancing the integrity of feature transfer. The Transformer module efficiently captures long-distance dependencies in time series through the self-attention mechanism, further improving the computational efficiency and prediction accuracy of the model. This method provides a new solution for efficient and accurate traffic flow prediction. Summary of the invention

[0003] This paper proposes a traffic flow prediction method with a block hybrid structure, which aims to process complex time series data and effectively extract local and global features through a block hybrid structure to achieve high-precision traffic flow prediction. The technical solution of this method is as follows:

[0004] Step 1: Input traffic data sequence represented as a matrix Where L is the number of time steps, and M is the number of variables in each time step, including traffic flow, speed, etc.;

[0005] Step 2: Perform instance normalization on the input data to obtain the normalized feature matrix x norm , the specific formula is as follows:

[0006]

[0007] Where μ is the mean of the input data and σ is the standard deviation of the input data;

[0008] Step 3: Convert the feature matrix x norm It is divided into multiple local feature blocks, and these blocks are fused through deep separable convolution and Transformer encoder to build a block hybrid structure. The specific processing steps are as follows:

[0009] Step 3.1: For the feature matrix x norm Expand and divide into N feature matrices x with stride S and block length P through the padding layer patch ;

[0010] Step 3.2: After the block feature matrix x patch Perform linear mapping and map it to the specified high-dimensional space. The linear mapping formula is as follows:

[0011] x proj =W P ·x patch

[0012] Where W P is the mapping matrix used to increase the feature dimension, x proj is the high-dimensional feature matrix after mapping;

[0013] Step 3.3: The Mixingformer structure effectively extracts and fuses feature information by combining Depthwise convolution, Transformer encoder and Pointwise convolution. The specific steps are as follows:

[0014] Step 3.3.1: First, the high-dimensional feature matrix x is transformed through multiple Depthwise convolutional layers. proj Processing to get x depthwise , each channel performs convolution operation independently to extract the local time dependency within each patch. For each input channel c, the formula of Depthwise convolution is as follows:

[0015]

[0016] Among them, x h,w,c is the input feature matrix x proj The value of position (h,w,c), k c,i,j is the weight of the (i,j)th position in the convolution kernel, y h,w,c is the output feature matrix x depthwise The value of the position (h,w,c) in the middle;

[0017] Step 3.3.2: Depthwise convolution feature x depthwise Input to the Transformer encoder to capture the global dependencies between blocks, use the multi-head attention mechanism to encode the input features, and calculate the multi-head attention layer output matrix x encoder , the specific formula is as follows:

[0018]

[0019] Among them, Q, K, and V are query, key, and value matrices respectively, and d k represents the dimension of the attention head;

[0020] Step 3.3.3: Transformer encoder output feature matrix x encoder Fusion is performed through Pointwise convolution. Pointwise convolution uses a 1×1 convolution kernel to perform a convolution operation on the input matrix. The formula for Pointwise convolution is as follows:

[0021]

[0022] Among them, y h,w,c′ is the output of Pointwise convolution, x h,w,c represents the eigenvalue of the cth channel at the position (h,w), W c,c′ is the value of the (c,c′)th position in the weight matrix;

[0023] Step 3.3.4: Fuse the output of Transformer with the output of Pointwise convolution to get the output matrix x pointwise ;

[0024] Step 4: Use the Mixingformer structure to transform the high-dimensional feature matrix x proj After adaptive adjustment, the GELU activation function and batch normalization layer are used to transform the feature x pointwise Perform nonlinear transformation and normalization;

[0025] Step 5: Convert the output matrix x of step 3.2 to proj Add to the output matrix of step 4 to get the final output feature matrix x final , for x final Flatten and map to the final prediction space to generate traffic flow prediction results for the next T time steps:

[0026]

[0027] Step 6: Calculate the loss using the mean squared error (MSE) loss function using the following formula:

[0028]

[0029] in, represents the predicted value of the i-th time series, represents the true value of the i-th time series, M represents the number of features of the meteorological data series, It means to average the loss of each feature channel, summarize the losses of M channels, average the summarized losses over the entire time series, get the overall target loss, and evaluate the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A flow chart of the traffic flow prediction method with a block-mixed structure provided in an embodiment of the present patent.

[0032] Figure 2 A schematic diagram of a model for a traffic flow prediction method with a block-mixed structure provided in an embodiment of this patent.

[0033] Figure 3 This is the combined structure diagram of the Mixingformer of this patent.

[0034] Figure 4 These are the experimental comparison results between this patent model and the current mainstream architecture for traffic congestion prediction. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings;

[0036] like Figure 2 As shown, an embodiment of the present invention provides a traffic flow prediction method with a block hybrid structure, comprising:

[0037] Step 1: Select the public dataset Traffic, which contains hourly traffic data from July 2016 to July 2018. Each set of traffic data contains 860 features, including traffic volume, speed, etc. Set the lookback window length of the time series to 336, that is, each input data contains the data of the past 336 time steps, and each input sequence can be represented as a 336×860 matrix. Set the output step length to 96, that is, each time the model predicts the data of the next 96 time steps. By inputting 336 time steps of data, the model predicts the traffic data of the next 96 time steps.

[0038] Step 2: Normalize the input data to obtain the normalized feature matrix x norm , to avoid some features being too large or too small to affect model training.

[0039] Step 3: Convert the feature matrix x norm It is divided into multiple local feature blocks, and these blocks are fused through deep separable convolution and Transformer encoder to build a block hybrid structure. The specific processing steps are as follows:

[0040] Step 3.1: Normalize the feature matrix x norm After expansion through the padding layer, the feature matrix is ​​split with a stride of 8 to obtain multiple blocks x of size 16×860 patch (i.e. each block contains 16 time steps of data).

[0041] Step 3.2: After the block feature matrix x patch Perform linear mapping and map it to the specified high-dimensional space so that subsequent modules can extract its features.

[0042] Step 3.3: Use Mixingformer to perform multi-level processing on the feature matrix. The architecture consists of three parts, such as Figure 3 As shown:

[0043] Step 3.3.1: Input feature matrix It is sent to the deep convolution layer, and the convolution uses a depthwise convolution kernel of size 8, where each channel is convolved separately, and the output feature matrix This allows capturing local temporal dependencies;

[0044] Step 3.3.2: After Depthwise convolution, the feature matrix x is obtained depthwise The encoder calculates the similarity between each patch and other patches through the multi-head self-attention mechanism, fuses local information with global information, and generates a feature matrix containing global information.

[0045] Step 3.3.3: Output feature matrix x after encoder encoder Use a 1×1 convolution kernel to convolve each channel to get the output matrix Without changing the length and width dimensions of the time series, the feature fusion between channels is enhanced.

[0046] Step 4: Use the Mixingformer structure to transform the high-dimensional feature matrix x proj After adaptive adjustment, the GELU activation function and batch normalization layer are used to transform the feature x pointwise Nonlinear transformation and normalization are performed to enhance the nonlinear modeling capability of the model and further stabilize the training process.

[0047] Step 5: Convert the output matrix x of step 3.2 to proj Add to the output matrix of step 4 to get the final output feature matrix x final , for x final Flatten and map to the final prediction space to generate traffic flow prediction results for the next 96 time steps:

[0048]

[0049] Step 6: Use the mean square error (MSE) loss function to calculate the error between the predicted result and the actual value. The formula is as follows:

[0050]

[0051] in, represents the predicted value of the i-th time series, represents the true value of the i-th time series, 860 means that the traffic data contains 860 features, It means to average the losses in each feature channel, summarize the losses of each channel, average the summarized losses over the entire time series, get the overall target loss, and evaluate the prediction results.

[0052] Figure 4The experimental comparison results of this patent model and the current mainstream architecture for traffic congestion prediction are shown, where Data_Name represents the data set, Input_Length represents the input length, Output_Length represents the output length, Model_name represents the model name, MSE represents the mean square error, and MAE represents the mean absolute error. The results of MSE and MAE show that this patent model has improved the average performance of the traffic feature prediction results in the Traffic data set, and the model has a good prediction effect in terms of traffic volume, speed, etc. The model not only improves the prediction accuracy, but also maintains good stability and response speed in the environment of real-time traffic data updates, and adapts to rapidly changing traffic conditions, which is particularly important for short-term traffic prediction in practical applications.

Claims

1. A traffic flow prediction method with a block-mixed structure, characterized in that The following steps are involved: Step 1: Input traffic data sequence represented as a matrix Where L is the number of time steps, and M is the number of variables in each time step, including traffic flow, speed, etc.; Step 2: Perform instance normalization on the input data to obtain the normalized feature matrix x norm , the specific formula is as follows: Where μ is the mean of the input data and σ is the standard deviation of the input data; Step 3: Convert the feature matrix x norm It is divided into multiple local feature blocks, and these blocks are fused through deep separable convolution and Transformer encoder to build a block hybrid structure. The specific processing steps are as follows: Step 3.1: For the feature matrix x norm Expand and divide into N feature matrices x with stride S and block length P through the padding layer patch ; Step 3.2: After the block feature matrix x patch Perform linear mapping and map it to the specified high-dimensional space. The linear mapping formula is as follows: x proj =W P ·x patch Where W P is the mapping matrix used to increase the feature dimension, x proj is the high-dimensional feature matrix after mapping; Step 3.3: The Mixingformer structure effectively extracts and fuses feature information by combining Depthwise convolution, Transformer encoder and Pointwise convolution. The specific steps are as follows: Step 3.3.1: First, the high-dimensional feature matrix x is transformed through multiple Depthwise convolutional layers. proj Processing to get x depthwise , each channel performs convolution operation independently to extract the local time dependency within each patch. For each input channel c, the formula of Depthwise convolution is as follows: Among them, x h,w,c is the input feature matrix x proj The value of position (h,w,c), k c,i,j is the weight of the (i,j)th position in the convolution kernel, y h,w,c is the output feature matrix x depthwise The value of the position (h,w,c) in the middle; Step 3.3.2: Depthwise convolution feature x depthwise Input to the Transformer encoder to capture the global dependencies between blocks, use the multi-head attention mechanism to encode the input features, and calculate the multi-head attention layer output matrix x encoder , the specific formula is as follows: Among them, Q, K, and V are query, key, and value matrices respectively, and d k represents the dimension of the attention head; Step 3.3.3: Transformer encoder output feature matrix x encoder Fusion is performed through Pointwise convolution. Pointwise convolution uses a 1×1 convolution kernel to perform a convolution operation on the input matrix. The formula for Pointwise convolution is as follows: Among them, y h,w,c′ is the output of Pointwise convolution, x h,w,c represents the eigenvalue of the cth channel at the position (h,w), W c,c′ is the value of the (c,c′)th position in the weight matrix; Step 3.3.4: Fuse the output of Transformer with the output of Pointwise convolution to get the output matrix x pointwise ; Step 4: Use the Mixingformer structure to transform the high-dimensional feature matrix x proj After adaptive adjustment, the GELU activation function and batch normalization layer are used to transform the feature x pointwise Perform nonlinear transformation and normalization; Step 5: Convert the output matrix x of step 3.2 to proj Add to the output matrix of step 4 to get the final output feature matrix x final , for x final Flatten and map to the final prediction space to generate traffic flow prediction results for the next T time steps: Step 6: Calculate the loss using the mean squared error (MSE) loss function using the following formula: in, represents the predicted value of the i-th time series, represents the true value of the i-th time series, M represents the number of features of the meteorological data series, It means to average the loss of each feature channel, summarize the losses of M channels, average the summarized losses over the entire time series, get the overall target loss, and evaluate the prediction results.

Images