Highway traffic state estimation method based on Transformer

Through the highway traffic state estimation method based on Transformer, the multi-head self-attention mechanism is used to analyze the association between microvehicle motion parameters and traffic flow state, which solves the problem of traditional models ignoring the micro level, and achieves a more accurate and stable traffic state estimation. Improved the decision-making support capabilities of traffic management.

CN117576897BActive Publication Date: 2024-12-03SOUTHEAST UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202311448915.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-02
Publication Date
2024-12-03
Estimated Expiration
2043-11-02

AI Technical Summary

Technical Problem

The traditional traffic state analysis model mainly focuses on the macro characteristics of traffic flow, ignores the micro-level vehicle movement behavior, and is difficult to capture the global changes in traffic flow and analyze the dynamic changes after traffic accidents. It is impossible to effectively understand the micro-vehicle movement parameters and traffic flow state. Intrinsic connection.

Method used

Using the highway traffic state estimation method based on Transformer, a traffic state estimation model with a multi-headed self-attention mechanism is constructed by collecting and preprocessing microvehicle motion parameters, and the model weight is optimized using the mean square error loss function to perform real-time estimation of traffic state.

Benefits of technology

It improves the prediction accuracy and stability of traffic data analysis, provides a more comprehensive methodology to understand traffic flow state, provides decision-making support for traffic management departments, and improves the targeted control and evaluation capabilities of highway traffic operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117576897B_ABST
    Figure CN117576897B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for estimating the traffic state of a highway based on a Transformer model, comprising: collecting and preprocessing the traffic data of the highway, converting the historical traffic data into a traffic state feature vector; constructing a traffic state estimation model based on a Transformer, using a multi-head self-attention mechanism to capture the spatiotemporal characteristics of the traffic data, constructing a deep learning model to learn the hidden patterns in the traffic data, and training a traffic state estimator using the traffic state feature vector; when performing state estimation, the method inputs the real-time traffic data into the Transformer model, obtains its feature vector, and obtains the current traffic state parameters of the highway through the aforementioned trained traffic state estimator. The method of the present invention not only improves the accuracy of traffic state estimation, but also provides an efficient and reliable tool for highway managers to help them better manage and dispatch traffic, thereby optimizing the traffic fluidity and overall performance of the road network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent transportation system and traffic state estimation, and in particular relates to a highway traffic state estimation method based on Transformer. Background Art

[0002] In the management of modern transportation systems, the identification and analysis of traffic flow states always occupy an important position. Traffic states provide us with insights into the entire traffic environment, laying the foundation for optimizing traffic management, improving road use efficiency and alleviating traffic congestion. However, road traffic states do not exist in isolation, but are composed of the microscopic behaviors of a large number of vehicles. Driven by emerging technologies such as vehicle networking and autonomous driving, the impact of vehicle motion behavior on traffic flow states has become increasingly significant. Understanding the relationship between microscopic vehicle motion behavior and traffic flow characteristics has become the key to modern transportation system management. Traditional traffic state analysis models mainly focus on the macroscopic characteristics of traffic flow, but these models often ignore the impact of vehicle motion behavior at the micro level. However, in recent years, driven by multidisciplinary cross-disciplinary research and emerging detection technologies, the study of vehicle motion parameters at the micro level has gradually become the focus of the transportation field.

[0003] Traffic flow is regarded as a compressible continuous medium composed of many traffic individuals. When the number of traffic individuals contained in the traffic system is reduced to a certain level, the averageness of the macroscopic system will disappear. Microscopic vehicle motion parameters mainly focus on the behavioral characteristics of a single vehicle. This analysis method often ignores the macroscopic characteristics of traffic flow as a whole. Such limitations greatly reduce the scope of application and explanatory power of microscopic traffic theory when dealing with complex traffic phenomena. It is difficult to capture the global changes in traffic flow, such as the formation and diffusion of traffic congestion, and it is also impossible to effectively analyze the dynamic changes of traffic flow after traffic accidents. The intrinsic correlation between microscopic vehicle motion parameters and traffic flow status and the supervenience between the two can extract the deep pattern of traffic status, which helps to understand traffic phenomena more comprehensively and accurately. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a highway traffic state estimation method based on Transformer, to perform correlation analysis and traffic state estimation between microscopic vehicle motion parameters and traffic flow states, thereby facilitating highway managers to conduct targeted control and evaluation of highway traffic operations and improve the overall performance of road network traffic.

[0005] The present invention adopts the following technical solution: a method for estimating highway traffic status based on Transformer, comprising the following steps:

[0006] S1, collect and pre-process the microscopic vehicle motion parameters of the highway, convert the microscopic vehicle motion parameters into traffic feature vectors, and obtain a standardized time series data set;

[0007] S2. Build a Transformer-based traffic state estimation model and use the multi-head self-attention mechanism to learn hidden patterns in traffic data;

[0008] S3, using the traffic feature vector to train the traffic state estimation model, using the mean square error loss function to optimize the weight of the traffic state estimation model, and verifying and adjusting the traffic state estimation model;

[0009] S4. Input the traffic data in real time into the traffic state estimation model trained in step S3, obtain the real-time traffic state parameters of the expressway, and perform highway traffic state estimation.

[0010] Furthermore, in step S1, the process of converting the traffic feature vector includes the following steps:

[0011] S11, collecting original data of microscopic vehicle motion parameters on highways;

[0012] S12, cleaning the collected raw data of microscopic vehicle motion parameters, identifying abnormal values ​​and outliers using the quartile method, and eliminating outliers;

[0013] S13, use the mean method to fill missing values;

[0014] S14, normalizing the traffic data so that different feature dimensions have the same value range;

[0015] S15, microscopic vehicle motion parameter conversion, converting the specific motion parameters of the vehicle into a traffic feature vector, wherein the motion parameters include vehicle speed and driving trajectory.

[0016] Furthermore, in step S2, the process of constructing the Transformer-based traffic state estimation model includes the following steps:

[0017] S21. Construct the input layer, including input embedding and position encoding. The microscopic vehicle motion parameters are converted into fixed-length vectors through the embedding matrix. Position encoding is introduced to represent the individual vehicle sequence information and learn the vehicle sequence relationship.

[0018] S22, construct a feature encoder to encode the vehicle motion parameters after position encoding and pass them to the feature decoder. The encoder is composed of multiple identical encoder layers, each layer contains a multi-head self-attention module and a feedforward neural network module;

[0019] The feedforward network is a two-layer fully connected network, in which the middle hidden layer is processed with the Relu activation function. The calculation process is:

[0020] FFN(x)=max(0,xB1+b1)B2+b2

[0021] Among them, x is the normalized output matrix of the attention layer, B1 and B2 are weight vectors, b1 and b2 are bias terms, and max(0, ) is the ReLU activation function.

[0022] The multi-head self-attention module is based on the self-attention mechanism and represents the correlation between features by calculating the weight coefficient. The Scaled Dot-Product Attention calculation method is used to calculate the approximate correlation between the query matrix Q and the key matrix K by scaling the dot product, normalizing it with the SoftMax function, and multiplying it with the value matrix V. The calculation process is:

[0023]

[0024] Where I is the feature matrix, W q ,W k ,W v represents the weight matrix of Q, K, V, d represents the dimension of matrix K, and T is the transpose of matrix K.

[0025] S23. Construct a feature decoder, receive the output of the feature encoder and decode it. The feature decoder is constructed by multiple decoder layers. Each layer contains a multi-head self-attention module, a multi-head encoding-decoding attention module and a feedforward neural network module. Residual connection and layer normalization are implemented between modules, and the result is transmitted to the output layer.

[0026] S24, the output layer receives the output of the feature decoder, and processes it through the fully connected layer, the Softmax layer and the regularization layer to expand the microscopic vehicle motion parameters into a one-dimensional vector to obtain the estimation result of the traffic flow state parameters.

[0027] Furthermore, in step S3, training the traffic state estimator includes the following steps:

[0028] S31, dividing the microscopic vehicle motion parameters into a training set, a validation set and a test set;

[0029] S32, using the mean square error loss function to evaluate the accuracy of the model;

[0030] S33, select Adam algorithm to optimize model weights;

[0031] S34, training the model according to the training set data and the set hyperparameters;

[0032] S35. Use the validation set to evaluate the performance of the model and adjust the model as needed;

[0033] S36. Use the randomized search method to perform hyperparameter tuning, randomly select parameter values ​​from the predefined parameter distribution, evaluate a given number of parameter combinations, and obtain the optimal traffic state estimation model.

[0034] Furthermore, in step S4, the input traffic data is fed into a trained traffic state estimation model to obtain real-time highway traffic state parameters and perform highway traffic state estimation.

[0035] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0036] First, the Transformer-based highway traffic state estimation method of the present invention performs traffic data analysis based on a multi-head attention mechanism and adapts to the characteristics of traffic data, thereby improving the prediction accuracy and stability of the model; it demonstrates stronger performance and better generalization ability in the task of processing the relationship between microscopic vehicle motion parameters and traffic flow states.

[0037] Second, the Transformer-based highway traffic state estimation method of the present invention provides a new research perspective on the relationship between traffic flow state and microscopic vehicle motion parameters; this method provides a new methodological basis for future traffic management strategies. A comprehensive understanding helps to better understand and predict traffic flow states, thereby providing powerful decision-making support for traffic management departments. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a method for estimating highway traffic status based on Transformer according to the present invention;

[0039] Figure 2 A schematic diagram of highway traffic feature vector conversion according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of a traffic state estimator according to an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of a traffic state estimator training according to an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of iterative solution of estimator weight parameters according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the application is further elaborated in detail below in conjunction with the accompanying drawings. The following embodiments can enable the professional and technical personnel to understand the present invention more comprehensively, but do not limit the present invention in any way. All non-innovative embodiments of other researchers in this field on this embodiment belong to the protection scope of the present invention. At the same time, for the step numbering in the embodiment of the present invention, it is only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0044] The present invention discloses a highway traffic state estimation method based on Transformer, see Figure 1 , the highway traffic state estimation method comprises the following steps:

[0045] S1. Collect and preprocess the traffic data of highways. The traffic feature vector conversion includes removing outliers, filling missing values, and data standardization. The microscopic vehicle motion parameters are converted into traffic feature vectors, and a standardized time series data set is obtained based on the sliding window method.

[0046] S2. Build a Transformer-based traffic state estimation model, using a multi-head self-attention mechanism to learn hidden patterns in traffic data. Its basic structure consists of four parts: input layer, feature encoder, feature decoder, and output layer;

[0047] S3, using traffic feature vectors to train the traffic state estimator, using the mean square error loss function to optimize the model weights, and perform model verification and adjustment;

[0048] S4. Input the traffic data in real time into the traffic state estimation model trained in step S3 to obtain the real-time traffic state parameters of the expressway.

[0049] In an optional embodiment of the present invention, Figure 2 As shown, step S1 includes the following sub-steps S11 to S15:

[0050] Step S11, selecting original microscopic vehicle motion parameters.

[0051] Specifically, this embodiment selects a basic section of a highway (about 7 kilometers long), and deploys multiple drone devices in the section to provide the required traffic parameters, including vehicle speed and driving trajectory. This highway traffic state estimation uses 7 days (24 hours a day) of detection data as the training set and 3 days of detection data as the test set.

[0052] It should be noted that traffic data includes a lot of content, including macro traffic data and micro vehicle motion data. In this embodiment, the traffic parameters are mainly the micro vehicle motion data in the traffic data.

[0053] Step S12: clean the collected data, use the quartile method to identify abnormal values ​​and outliers, and remove outliers.

[0054] Step S13: Use the mean method to fill in missing values.

[0055] Step S14: normalize the traffic data so that different feature dimensions have the same value range. Given a feature, specifically, the normalization process can be expressed as:

[0056]

[0057] Among them, μx i and σx i They are the features x i The mean and standard deviation of .

[0058] Step S15: complete the microscopic vehicle motion parameter conversion, and convert the specific motion parameters of the vehicle (speed, driving trajectory) into a traffic feature vector.

[0059] In an optional embodiment of the present invention, Figure 3 As shown, step S2 relies on the self-attention mechanism for encoding and decoding. The basic structure of the constructed traffic state model consists of four parts: input layer, feature encoder, feature decoder and output layer, which specifically includes the following sub-steps S21 to S24:

[0060] Step S21: construct the input layer, including input embedding and position encoding. The microscopic vehicle motion parameters are converted into fixed-length vectors through the embedding matrix. Position encoding is introduced to represent the individual vehicle sequence information and learn the vehicle sequence relationship. The formula is as follows:

[0061]

[0062] In the formula, p represents the position of the input data, i represents the corresponding feature dimension, dmodel represents the dimension of the input and output data, and each input data is calculated using sin and cos functions with different periods, and the wavelength gradually increases from 2π to 10000*2π.

[0063] S22. Construct a feature encoder to perform feature encoding on the microscopic vehicle motion parameters after position encoding and pass them to a feature decoder. The encoder is composed of multiple identical encoder layers, each of which contains a multi-head self-attention module and a feedforward neural network module.

[0064] The feedforward network is a two-layer fully connected network, in which the middle hidden layer is processed with the Relu activation function. The specific formula is as follows:

[0065] FFN(x)=max(0,xB1+b1)B2+b2

[0066] Where x is the normalized output matrix of the attention layer, B is the weight vector, b is the bias term, and max(0, ) is the ReLU activation function.

[0067] The multi-head self-attention module, based on the self-attention mechanism, characterizes the correlation between features by calculating the weight coefficient. It uses the Scaled Dot-Product Attention calculation method to calculate the approximate correlation between the query matrix Q and the key matrix K by scaling the dot product, normalizes it with the SoftMax function, and multiplies it with the value matrix V. The specific formula is as follows:

[0068]

[0069] Where I is the feature matrix, W q ,W k ,W v represents the weight matrix of Q, K, V, d represents the dimension of matrix K, and T is the transpose of matrix K.

[0070] It is important to note that the multi-head self-attention mechanism is a parallel connection of multiple self-attention mechanisms, which is used to learn microscopic vehicle motion parameters in different dimensional spaces.

[0071] Step S23, construct a feature decoder, receive the output of the feature encoder and decode it. The feature decoder is constructed by multiple decoder layers, each layer contains a multi-head self-attention module, a multi-head encoding-decoding attention module and a feedforward neural network module, and residual connections and layer normalization are implemented between modules, and the results are transmitted to the output layer.

[0072] Step S24, construct an output layer, receive the output of the feature decoder, and expand the microscopic vehicle motion parameters into a one-dimensional vector through the full connection layer, the Softmax layer and the regularization layer to obtain the estimation result of the traffic flow state parameters.

[0073] In an optional embodiment of the present invention, Figure 4 As shown, step S3 includes the following sub-steps S31 to S36:

[0074] Step S31, divide the data into training set, validation set and test set. The training set is used to debug the estimation model, the validation set is used to check the training effect, and the test set is used to test the actual learning ability of the model. The proportions of the three are 60%, 20% and 20% respectively.

[0075] Step S32: Use the mean square error loss function to evaluate the accuracy of the model. The specific formula is as follows:

[0076]

[0077] In the formula, y i is the true value, is an estimated value, and N is the number of training set data.

[0078] Step S33: Select the Adam algorithm to optimize the model weights.

[0079] Step S34: train the model according to the training data and the set hyperparameters.

[0080] Step S35: Use the validation set to evaluate the performance of the model and adjust the model as needed.

[0081] Step S36: Use a randomized search method to perform hyperparameter tuning, randomly select parameter values ​​from a predefined parameter distribution, evaluate a given number of parameter combinations, and obtain an optimal traffic state estimation model.

[0082] In an optional embodiment of the present invention, Figure 5 As shown, after the estimator weight parameter solution iteration of the embodiment of the present invention is completed based on the Transformer model, step S4 inputs the traffic data into the trained traffic state estimation model to obtain the real-time traffic state parameters of the highway, including flow data, congestion degree parameters, and various aspects of the traffic conditions to understand the traffic state of the highway for refined control of the highway.

[0083] The Transformer-based highway traffic state estimation method of this embodiment uses the Transformer idea to conduct research on highway traffic state estimation, makes full use of Transformer and random search cross-validation hyperparameter optimization, has good performance for highway traffic state estimation, and can improve the accuracy of traffic estimation.

[0084] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A highway traffic state estimation method based on Transformer, characterized in that: The following steps are involved: S1, collect and pre-process the microscopic vehicle motion parameters of the highway, convert the microscopic vehicle motion parameters into traffic feature vectors, and obtain a standardized time series data set; The motion parameters include vehicle speed and driving trajectory; S2. Build a Transformer-based traffic state estimation model and use the multi-head self-attention mechanism to learn hidden patterns in traffic data; The traffic state estimation model construction process includes the following steps: S21, construct the input layer, including input embedding and position encoding, convert the micro-vehicle motion parameters into fixed-length vectors through the embedding matrix, introduce position encoding to represent the individual order information of micro-vehicles, and learn the order relationship of micro-vehicles; S22, constructing a feature encoder, performing feature encoding on the microscopic vehicle motion parameters after position encoding, and passing the feature encoder to a feature decoder, wherein the feature encoder is composed of a plurality of identical encoder layers, each layer containing a multi-head self-attention module and a feedforward neural network module; the multi-head self-attention mechanism is used to learn the microscopic vehicle motion parameters in different dimensional spaces; S23, construct a feature decoder, receive the feature encoder output and decode it, the feature decoder is constructed by multiple decoder layers, each layer contains a multi-head self-attention module, a multi-head encoding-decoding attention module and a feedforward neural network module, and residual connections and layer normalization are implemented between modules, and the results are transmitted to the output layer; S24, the output layer receives the output of the feature decoder, and processes it through the fully connected layer, the Softmax layer and the regularization layer to expand the microscopic vehicle motion parameters into a one-dimensional vector to obtain the estimation result of the traffic flow state parameters; S3, using the traffic feature vector to train the traffic state estimation model, using the mean square error loss function to optimize the weight of the traffic state estimation model, and verifying and adjusting the traffic state estimation model; S4. Input the traffic data in real time into the traffic state estimation model trained in step S3, obtain the real-time traffic state parameters of the expressway, and perform highway traffic state estimation.

2. The method for estimating highway traffic status based on Transformer according to claim 1, characterized in that: In step S1, traffic data of the expressway is collected and preprocessed, and microscopic vehicle motion parameters are converted into traffic feature vectors to obtain a standardized time series data set. The method includes the following steps: S11, collecting original data of microscopic vehicle motion parameters on highways; S12, cleaning the collected raw data of microscopic vehicle motion parameters, identifying abnormal values ​​and outliers using the quartile method, and eliminating outliers; S13, use the mean method to fill missing values; S14, normalizing the microscopic vehicle motion parameters so that the microscopic vehicle motion parameters of different feature dimensions have the same numerical range; S15, microscopic vehicle motion parameter conversion, converting the specific motion parameters of the vehicle into a traffic feature vector, wherein the motion parameters include vehicle speed and driving trajectory.

3. The method for estimating highway traffic status based on Transformer according to claim 2, characterized in that: In step S14, for a given feature x i , the normalization process is expressed as: Among them, μx i and σx i They are the features x i The mean and standard deviation of .

4. The method for estimating highway traffic status based on Transformer according to claim 1, characterized in that: In step S21, the calculation process of the position code PE is: Among them, p represents the position of the input data, i represents the corresponding feature dimension, dmodel represents the dimension of the input and output data, and each input data is calculated using sin and cos functions with different periods, and the wavelength gradually increases from 2π to 10000*2π.

5. The method for estimating highway traffic status based on Transformer according to claim 1, characterized in that: In step S22, the multi-head self-attention module characterizes the correlation between features by calculating the weight coefficient based on the self-attention mechanism, adopts the Scaled Dot-Product Attention calculation method, calculates the approximate correlation between the query matrix Q and the key matrix K by scaling the dot product, normalizes it with the SoftMax function, and multiplies it with the value matrix V. The calculation process is: Where I is the feature matrix, W q ,W k ,W v represents the weight matrix of Q, K, V, d represents the dimension of matrix K, and T is the transpose of matrix K.

6. The method for estimating highway traffic status based on Transformer according to claim 5, characterized in that: In step S22, the feedforward neural network is a two-layer fully connected network, and the middle hidden layer is processed with a Relu activation function. The calculation process is: FFN(x)=max(0,xB1+b1)B2+b2 Among them, x is the normalized output matrix of the attention layer, B1 and B2 are weight vectors, b1 and b2 are bias terms, and max(0, ) is the ReLU activation function.

7. The method for estimating highway traffic status based on Transformer according to claim 1, characterized in that: In step S3, the weight of the model is optimized using the mean square error loss function to verify and adjust the traffic state estimation model, including the following steps: S31, dividing the microscopic vehicle motion parameters into a training set, a validation set and a test set; S32. The mean square error loss function is used to evaluate the accuracy of the traffic state estimation model. The calculation process is: Among them, y i is the true value, is an estimated value, N is the number of training set data; S33, select the Adam algorithm to optimize the weight of the traffic state estimation model; S34, training the traffic state estimation model according to the training set and the set hyperparameters; S35. Evaluate the performance of the traffic state estimation model using the validation set and adjust the model as needed; S36. Use the random search method to perform hyperparameter tuning, randomly select parameter values ​​from the predefined parameter distribution, evaluate a given number of parameter combinations, and obtain the optimal traffic state estimation model.

8. The method for estimating highway traffic status based on Transformer according to claim 7, characterized in that: In step S4, real-time traffic data is input into the trained optimal traffic state estimation model to obtain real-time highway traffic state parameters and perform highway traffic state estimation. The traffic state parameters include: flow data, congestion degree parameters, and traffic conditions.

Citation Information

Patent Citations

  • Expressway congestion prediction method based on space-time multi-head attention mechanism

    CN115050177A

  • Highway traffic flow prediction method based on Transform and graph attention network

    CN116092294A

  • Traffic flow model and data fusion driven traffic state estimation method and device

    CN116153069A