Traffic flow prediction method based on Prompt-Tuning strategy

Abstract

The invention discloses a traffic flow prediction method based on a Prompt-Tuning strategy, and relates to the technical field of traffic flow prediction. According to the traffic flow prediction method based on the Prompt-Tuning strategy, for traffic long-tail data, through scenarized Prompt and a multi-expert model, in combination with long-tail sample oversampling, abnormal scene prediction precision is improved, complex space-time dependence is accurately captured by depending on Transform backbone and space-time attention gating, flow characteristic differences of different road sections and time periods are adapted, Prompt-Tuning efficient fine tuning is adopted, and the traffic flow prediction precision is improved. According to the method, generalization ability and reasoning efficiency can be taken into consideration, the comprehensive prediction precision is improved, meanwhile, prediction confidence and key influence factors are output, accurate decision support is provided for traffic scheduling, and intelligent traffic management efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of traffic flow prediction technology, specifically a traffic flow prediction method based on the Prompt-Tuning strategy. Background Technology

[0002] With the acceleration of urbanization and the surge in motor vehicle ownership, urban traffic congestion has become a global challenge. Accurate traffic flow forecasting is a core prerequisite for intelligent traffic management and control, and is crucial for improving road network efficiency, reducing energy consumption, and lowering exhaust emissions. Traditional traffic flow forecasting methods can be mainly categorized as follows: Statistical models, such as autoregressive integral moving average models and Kalman filters, are based on the assumption of linear stationarity of the data. They struggle to capture the inherent nonlinearity, time-varying nature, and complex spatiotemporal dependencies of traffic flow, resulting in limited prediction accuracy, especially when facing unexpected situations.

[0003] Machine learning models, such as support vector machines and gradient boosting decision trees, can learn non-linear relationships in data. However, when dealing with high-dimensional spatiotemporal data, feature engineering is complex and time-consuming, and the model's generalization ability and ability to capture long-term dependencies still need improvement.

[0004] Deep learning models: In recent years, deep learning models, represented by recurrent neural networks and their variants such as LSTM and GRU, have made significant progress in the field of traffic prediction. They can effectively model the dependencies of time series data. Subsequently, Transformer-based models, with their powerful self-attention mechanism, have shown even greater potential in capturing long-range spatiotemporal dependencies.

[0005] However, traditional traffic flow prediction methods still have the following shortcomings in practical applications: 1. Sensitive to long-tail distribution data: In traffic flow data, regular daily traffic flow is the main component, while abnormal traffic flow caused by extreme weather such as rainstorms, snowstorms, traffic accidents, and large-scale events is a minority, but has a huge impact. Existing models tend to learn the regular patterns that account for a higher proportion, and the prediction accuracy of these long-tail abnormal scenarios is low. Second, the spatiotemporal correlation modeling is insufficient: Traffic flow is affected by complex spatiotemporal factors. Although the existing models can model some spatiotemporal dependencies, they lack a refined adaptation mechanism for the specificities of different road segments and time periods, such as the differences in traffic characteristics between expressways and branch roads, and the differences in peak and off-peak patterns. III. The difficulty of balancing model generalization ability and efficiency: In pursuit of higher accuracy, model structures are becoming increasingly complex and the number of parameters is huge, resulting in high training and inference costs. IV. Limitations of the Prompt-Tuning Strategy: As an efficient model fine-tuning paradigm, Prompt-Tuning has been widely used in the field of natural language processing. However, its application in traffic prediction is mostly limited to simple template construction and fails to fully integrate the spatiotemporal characteristics of traffic flow for in-depth customization, thus limiting its scope of application.

[0006] In view of this, a traffic flow prediction method based on the Prompt-Tuning strategy is proposed. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a traffic flow prediction method based on the Prompt-Tuning strategy. This method solves the problems of traditional traffic flow prediction methods in practical applications, such as sensitivity to long-tailed distributed data, insufficient modeling of spatiotemporal correlations, difficulty in balancing model generalization ability and efficiency, and the limited application scope of the Prompt-Tuning strategy.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a traffic flow prediction method based on a Prompt-Tuning strategy, specifically comprising the following steps: S1. Collect basic traffic data and construct a multi-source spatiotemporal long-tail dataset by combining it with multi-source auxiliary data; the basic traffic data includes standard passenger car equivalent flow rate, spatial average speed, time occupancy rate, number of lanes in the current road segment, speed limit value, and latitude and longitude coordinates; the multi-source auxiliary data includes weather data, spatiotemporal label data, POI distribution data, and public transportation operation data. S2. Construct a spatiotemporally integrated Prompt and complete the construction of the traffic flow Prompt-Tuning strategy after setting scenario-based vectors; the scenario-based vectors include peak congestion vectors, off-peak regular vectors, and special event vectors. S3. Construct a spatiotemporal-aware multi-expert prediction model based on the traffic flow Prompt-Tuning strategy, train and validate the multi-expert prediction model based on multi-source spatiotemporal long-tail datasets, and obtain the final multi-expert prediction model. S4. Collect basic traffic data and multi-source auxiliary data of the road segment to be predicted, generate model input vectors and input them into the final multi-expert prediction model. After outputting standardized traffic flow prediction values, the model is converted into the actual traffic flow value, prediction confidence and key influencing factors of the road segment to be predicted at a future set time through inverse standardization.

[0009] The present invention is further configured such that the method for constructing the multi-source spatiotemporal long-tail dataset includes: S11. After collecting basic traffic flow data at a 5-minute granularity, correlate it with real-time multi-source auxiliary data; S12. Divide the urban road network into grids of 100m×100m size, and map the basic traffic data of the corresponding road segments to the corresponding grids; S13. After outlier filtering using the 3σ principle, missing data is filled in. The equivalent flow rate, average speed, and time occupancy rate of standard passenger cars are standardized using Z-score. The data are then divided into training, validation, and test sets in a 7:1:2 ratio, with long-tail samples in the training set being oversampled in a 1:3 ratio.

[0010] The present invention is further configured such that the missing data imputation method includes: ; In the formula, For the target road section In time slice The filler standard passenger car equivalent flow value, Adjacent road sections The set, Adjacent road sections In time slice Standard passenger car equivalent flow rate value, Adjacent road sections In time slice The weighting coefficient of the standard passenger car equivalent flow rate value For the target road section In the Standard passenger car equivalent flow rate values ​​for adjacent historical time slices All are weighting coefficients, and .

[0011] The present invention is further configured such that the method for constructing the spatiotemporal fusion Prompt includes: Using the T5-small pre-trained Transformer model as the basic architecture, 90% of its underlying parameters are frozen. The road ID embedding vector, time slice encoding vector, and historical standard passenger car equivalent traffic flow trend vector are concatenated with the learnable Prompt. The historical standard passenger car equivalent traffic flow trend vector is the trend feature extracted within a 2-hour sliding window using 1D-CNN.

[0012] The present invention is further configured such that: the peak congestion vector is obtained by weighted summation of typical traffic flow peak features and congestion or peak word vectors; The off-peak routine vector is obtained by weighted summation of typical off-peak traffic flow features and off-peak or routine word vectors; The special event vector is obtained by weighted summation of the typical abnormal features and the word vectors of rainstorms or accidents.

[0013] The present invention is further configured such that the spatiotemporally aware multi-expert prediction model includes a Transformer backbone, three sets of parallel expert modules, three sets of spatiotemporal attention gating, and three sets of cosine classification heads; A set of expert modules and a set of cosine classification heads are connected via a set of spatiotemporal attention gating; The expert module includes a peak expert module, an off-peak expert module, and a special expert module.

[0014] The present invention is further configured such that the formula for calculating the prediction confidence level is: ; In the formula, To predict confidence levels, This represents the predicted probability of the Peak Expert module. This represents the predicted probability of the off-peak expert module. This refers to the predicted probability for a special expert module. All are weighting coefficients.

[0015] The present invention is further configured such that the key influencing factor is the contextualized vector feature with the highest attention weight.

[0016] This invention provides a traffic flow prediction method based on a prompt-turning strategy. It has the following advantages: This invention targets long-tail traffic data, improving the prediction accuracy of abnormal scenarios through scenario-based prompts and multi-expert models, combined with long-tail sample oversampling. Relying on the Transformer backbone and spatiotemporal attention gating, it accurately captures complex spatiotemporal dependencies, adapts to the differences in traffic characteristics of different road segments and time periods, and uses Prompt-Tuning for efficient fine-tuning. It can balance generalization ability and inference efficiency, improving the overall prediction accuracy while outputting prediction confidence and key influencing factors, providing accurate decision support for traffic scheduling and improving the efficiency of intelligent traffic management. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0019] Please see Figure 1 This invention provides a traffic flow prediction method based on a Prompt-Tuning strategy, which predicts traffic flow based on a multi-expert prediction model. The construction of the multi-expert prediction model specifically includes the following steps: S1. Collect basic traffic data, including standard passenger car equivalent traffic volume, spatial average speed, time occupancy rate, number of lanes on the current road segment, speed limit, and latitude and longitude coordinates. Combine this with multi-source auxiliary data to construct a multi-source spatiotemporal long-tail dataset. The multi-source auxiliary data includes weather data, spatiotemporal label data, POI distribution data, and public transportation operation data. The methods for constructing the multi-source spatiotemporal long-tail dataset include: S11. After collecting basic traffic flow data at a 5-minute granularity, correlate it with real-time multi-source auxiliary data; S12. Divide the urban road network into grids of 100m×100m size, and map the basic traffic data of the corresponding road segments to the corresponding grids; S13. Differentiated thresholds are set according to the road grades of expressways, arterial roads, and branch roads. Outlier filtering is performed using the 3σ principle, and missing data is filled in. After Z-score standardization of standard passenger car equivalent flow, spatial average speed, and time occupancy rate, the data is divided into training set, validation set, and test set in a 7:1:2 ratio. The training set is used for multi-expert prediction model parameter learning, the validation set is used for multi-expert prediction model parameter adjustment, and the test set is used for multi-expert prediction model performance verification. Long-tail samples of sudden congestion and rainstorms in the training set are oversampled in a 1:3 ratio to ensure balanced sample distribution.

[0020] As detailed in detail, methods for filling in missing data include: ; In the formula, For the target road section In time slice The filler standard passenger car equivalent flow value, Adjacent road sections The set, Adjacent road sections In time slice Standard passenger car equivalent flow rate value, Adjacent road sections In time slice The weighting coefficient of the standard passenger car equivalent flow rate value For the target road section In the Standard passenger car equivalent flow rate values ​​for adjacent historical time slices All are weighting coefficients, and .

[0021] S2. Constructing a spatiotemporal fusion Prompt: Using the T5-small pre-trained Transformer model as the basic architecture, freeze 90% of its underlying parameters, and concatenate the road ID embedding vector, time slice encoding vector, and historical standard passenger car equivalent traffic flow trend vector with the learnable Prompt; where the historical standard passenger car equivalent traffic flow trend vector is the trend feature extracted within a 2-hour sliding window through 1D-CNN. After setting up scenario-based vectors, the traffic flow prompt-tuning strategy is completed. Scenario-based vectors include peak congestion vectors, off-peak regular vectors, and special event vectors. Specifically: Peak congestion vectors are adapted to scenarios with a sudden increase in traffic during morning and evening rush hours. The initial vectors are generated by fusing typical peak traffic features, such as a peak traffic volume of 1200 pcu / h and a duration of 60 minutes, with congestion / peak word vectors in a 6:4 ratio and then normalized by LayerNorm. The congestion / peak word vectors are pre-trained in a traffic corpus. The off-peak regular vector is adapted to the stable traffic scenario during off-peak hours. The initial vector is generated by fusing off-peak / regular word vectors in a 6:4 ratio with typical off-peak traffic features, such as an average traffic volume of 600 pcu / h and a fluctuation range of <10%. The off-peak / regular word vectors are pre-trained in a traffic corpus. Special event vectors are adapted to abnormal scenarios such as rainstorms and accidents. The initial vectors are generated by fusing typical traffic features of special events, such as a 30% drop in traffic caused by rainstorms or a 50% drop in traffic caused by accidents, with rainstorm / accident word vectors in a 6:4 ratio. The rainstorm / accident word vectors are pre-trained in a traffic corpus.

[0022] S3. Construct a spatiotemporal perception-based multi-expert prediction model based on the traffic flow prompt-tuning strategy. The multi-expert prediction model includes a Transformer backbone, three sets of parallel expert modules, three sets of spatiotemporal attention gating, and three sets of cosine classification heads. Each set of expert modules and each set of cosine classification heads is connected by a set of spatiotemporal attention gating. The expert modules include peak-hour expert modules, off-peak expert modules, and special expert modules, specifically: The Peak Expert module embeds peak congestion vectors to focus on learning the patterns of traffic surges during peak hours. The off-peak expert module embeds off-peak routine vectors to focus on learning the stable traffic patterns during off-peak periods. The special expert module embeds special event vectors to focus on learning the patterns of traffic mutations in abnormal scenarios.

[0023] As a detailed explanation, for setting spatiotemporal attention gating at the output of the three expert modules, the dynamic calculation of the output weights of each expert module is as follows: ; ; ; In the formula, To output weights, Spatial weights, As time weight, Adjacent road sections Traffic capacity, This represents the connectivity of the road network topology. For example, a value of 1.0 is assigned for unrestricted two-way traffic, 0.8 for two-way traffic with no left turns, and 0.6 for one-way traffic. For road section and The straight-line distance This represents the time difference for the same period in history.

[0024] For the cosine classification head, the classification head weights are initialized to road segments. The L2 normalized result of the historical average flow rate is used to map the feature vector output by the Transformer backbone to the flow rate prediction value.

[0025] Furthermore, a multi-expert prediction model is trained and validated based on a multi-source spatiotemporal long-tail dataset to obtain the final multi-expert prediction model, wherein the optimized loss function is: ; In the formula, For time and space loss, This is the loss for long-tailed samples.

[0026] In an exemplary embodiment, the method of applying the above-described multi-expert prediction model to predict AC traffic flow includes: Basic traffic data and multi-source auxiliary data at a 5-minute granularity are collected for the road segment to be predicted. Outlier filtering is performed using the 3σ principle, and missing data is filled in. Standardized data is generated by Z-score standardization of standard passenger car equivalent flow, spatial average speed and time occupancy. The corresponding scenario vector is selected according to the real-time scenario of the road segment to be predicted. The vector is then concatenated with the spatiotemporal prompt and standardized data to generate the model input vector. The model input vector is fed into the final multi-expert prediction model. Spatiotemporal attention gating is used to calculate the output weights of the three expert modules, activating the corresponding expert modules. The expert module with an output weight ≥60% is designated as the dominant expert module. After outputting standardized traffic flow prediction values, these are converted to actual traffic flow values ​​for the road segment to be predicted at a future set time using inverse standardization. The prediction confidence and key influencing factors are then output. The key influencing factor is the contextualized vector feature with the highest attention weight. The prediction confidence is calculated as follows: ; In the formula, To predict confidence levels, This represents the predicted probability of the Peak Expert module. This represents the predicted probability of the off-peak expert module. This refers to the predicted probability for a special expert module. All are weighting coefficients.

[0027] This invention solves the problems of inaccurate prediction of long-tail data and insufficient modeling of spatiotemporal correlation by combining spatiotemporal Prompt with multi-expert prediction models. The prediction accuracy is more than 20% higher than that of traditional methods, with MAE ≤ 16.5 and MAPE ≤ 0.11 during peak periods and special event scenarios.

[0028] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A traffic flow prediction method based on a Prompt-Tuning strategy, characterized in that, Specifically, the following steps are included: S1. Collect basic traffic data and construct a multi-source spatiotemporal long-tail dataset by combining it with multi-source auxiliary data; the basic traffic data includes standard passenger car equivalent flow rate, spatial average speed, time occupancy rate, number of lanes in the current road segment, speed limit value, and latitude and longitude coordinates; the multi-source auxiliary data includes weather data, spatiotemporal label data, POI distribution data, and public transportation operation data. S2. Construct a spatiotemporal fusion Prompt and complete the construction of the traffic flow Prompt-Tuning strategy after setting scenario-based vectors. The scenario-based vectors include peak congestion vectors, off-peak routine vectors, and special event vectors. S3. Construct a spatiotemporal-aware multi-expert prediction model based on the traffic flow Prompt-Tuning strategy, train and validate the multi-expert prediction model based on multi-source spatiotemporal long-tail datasets, and obtain the final multi-expert prediction model. S4. Collect basic traffic data and multi-source auxiliary data of the road segment to be predicted, generate model input vectors and input them into the final multi-expert prediction model. After outputting standardized traffic flow prediction values, the model is converted into the actual traffic flow value, prediction confidence and key influencing factors of the road segment to be predicted at a future set time through inverse standardization.

2. The traffic flow prediction method based on the Prompt-Tuning strategy according to claim 1, characterized in that, The methods for constructing multi-source spatiotemporal long-tail datasets include: S11. After collecting basic traffic flow data at a 5-minute granularity, correlate it with real-time multi-source auxiliary data; S12. Divide the urban road network into grids of 100m×100m size, and map the basic traffic data of the corresponding road segments to the corresponding grids; S13. After outlier filtering using the 3σ principle, missing data is filled in. The equivalent flow rate, average speed, and time occupancy rate of standard passenger cars are standardized using Z-score. The data are then divided into training, validation, and test sets in a 7:1:2 ratio, with long-tail samples in the training set being oversampled in a 1:3 ratio.

3. The traffic flow prediction method based on the Prompt-Tuning strategy according to claim 2, characterized in that, The methods for filling in the missing data include: ； In the formula, For the target road section In time slice The filler standard passenger car equivalent flow value, Adjacent road sections The set, Adjacent road sections In time slice Standard passenger car equivalent flow rate value, Adjacent road sections In time slice The weighting coefficient of the standard passenger car equivalent flow rate value. For the target road section In the Standard passenger car equivalent flow rate values ​​for adjacent historical time slices All are weighting coefficients, and .

4. The traffic flow prediction method based on the Prompt-Tuning strategy according to claim 1, characterized in that, The methods for constructing a spatiotemporally fused Prompt include: Using the T5-small pre-trained Transformer model as the basic architecture, 90% of its underlying parameters are frozen. The road ID embedding vector, time slice encoding vector, and historical standard passenger car equivalent traffic flow trend vector are concatenated with the learnable Prompt. The historical standard passenger car equivalent traffic flow trend vector is the trend feature extracted within a 2-hour sliding window using 1D-CNN.

5. The traffic flow prediction method based on the Prompt-Tuning strategy according to claim 1, characterized in that, The peak congestion vector is obtained by weighted summation of typical traffic flow peak features and congestion or peak word vectors; The off-peak routine vector is obtained by weighted summation of typical off-peak traffic flow features and off-peak or routine word vectors; The special event vector is obtained by weighted summation of the typical abnormal features and the word vectors of rainstorms or accidents.

6. The traffic flow prediction method based on the Prompt-Tuning strategy according to claim 4, characterized in that, The spatiotemporal awareness multi-expert prediction model includes a Transformer backbone, three sets of parallel expert modules, three sets of spatiotemporal attention gating, and three sets of cosine classification heads. A set of expert modules and a set of cosine classification heads are connected via a set of spatiotemporal attention gating; The expert module includes a peak expert module, an off-peak expert module, and a special expert module.

7. A traffic flow prediction method based on a Prompt-Tuning strategy according to claim 6, characterized in that, The formula for calculating the prediction confidence level is: ； In the formula, To predict confidence levels, This represents the predicted probability of the Peak Expert module. This represents the predicted probability from the off-peak expert module. This refers to the predicted probability for a special expert module. All are weighting coefficients.

8. The traffic flow prediction method based on the Prompt-Tuning strategy according to claim 1, characterized in that, The key influencing factor is the contextualized vector feature with the highest attention weight.

Images