A Highway Congestion Prediction Method Based on Attention Mechanism and Online Evolution

Abstract

This invention discloses a highway congestion prediction method based on an attention mechanism and online evolution, belonging to the field of intelligent transportation technology. The method first collects traffic data from all ETC gantries on the highway, and constructs ETC traffic chains by selecting associated gantries based on time accessibility rules, thereby generating end-to-end learning samples and corresponding congestion labels. Then, it builds an end-to-end time-series prediction model embedded with a multi-head attention mechanism, automatically extracting multi-level spatiotemporal correlation features between road segments. Furthermore, it introduces a dual strategy of sliding window and experience replay to achieve online model evolution, preserving historical effective traffic patterns while continuously adapting to the dynamic changes in real-time traffic flow. Finally, it can output real-time warnings of road segment congestion risks for multiple future time periods. This invention effectively improves the accuracy and robustness of long-term congestion prediction, providing highly reliable technical support for proactive highway traffic management and intelligent public travel guidance.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically relating to a highway congestion prediction method based on attention mechanisms and online evolution. Background Technology

[0002] Effective traffic condition prediction for highway sections is crucial for ensuring efficient road network operation and implementing proactive traffic management. Highway congestion prediction technology analyzes ETC gantry traffic flow data to predict potential traffic congestion in future periods, enabling traffic management departments to take control measures in advance. However, due to the highly nonlinear and time-varying nature of traffic flow, and the spatiotemporal discreteness of ETC data, traditional prediction methods often fail to achieve ideal results.

[0003] Despite the existence of various traffic congestion prediction methods, they still have many shortcomings in practical applications. Traditional research methods typically rely on statistical models or shallow machine learning algorithms, such as the Autoregressive Integral Moving Average (ARIMA) model and Support Vector Machine (SVM). These methods often have limitations when processing complex traffic data, failing to capture the nonlinear relationships and spatiotemporal dependencies within the data. Furthermore, traditional methods usually require complex feature engineering and model design, employing a cascaded, block-based architecture of "manual feature extraction—intermediate parameter prediction—congestion state determination." This step-by-step processing approach disrupts the inherent coherence of traffic flow spatiotemporal information, easily leading to multi-stage error accumulation and the loss of high-dimensional dynamic features.

[0004] In recent years, end-to-end learning methods have been gradually introduced into the field of traffic prediction. End-to-end learning can directly extract features from raw data and establish complex mapping relationships between inputs and outputs, avoiding tedious feature engineering. However, existing end-to-end prediction models still have the following limitations in practical applications: First, the model assigns the same weight to each spatiotemporal node of the input data, failing to capture key abnormal vehicle features that induce congestion; second, traffic flow distribution undergoes conceptual drift with factors such as seasons and holidays, and the predictive performance of static offline models degrades significantly over long periods of operation, while fine-tuning using only new data suffers from catastrophic forgetting problems.

[0005] Therefore, there is an urgent need for a highway congestion prediction method that can enhance the perception of key features and has the ability to continuously evolve. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a highway congestion prediction method based on attention mechanism and online evolution, aiming to solve two major defects of existing highway congestion prediction technologies: First, traditional prediction models use equal weight to aggregate traffic flow data, which cannot adaptively identify key samples of congestion causes such as low-speed trucks and broken vehicles, have poor ability to capture fine-grained spatiotemporal features of congestion precursors, and have low accuracy in short-term congestion warnings; Second, offline static models are difficult to adapt to the concept drift of traffic flow caused by seasonal changes, road construction, and holiday travel, and the prediction accuracy continuously decays over long-term online operation. Conventional incremental fine-tuning methods are prone to catastrophic forgetting, and the technical problem of being unable to take into account both the adaptability to new scenarios and the ability to remember historical typical congestion samples is difficult to solve.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A highway congestion prediction method based on attention mechanisms and online evolution includes the following steps:

[0009] S1. Construct an end-to-end learning sample and label system for ETC;

[0010] S1.1 Obtain historical traffic data from highway toll stations and ETC gantries, and clean, deduplicate, and handle outliers from the historical traffic data;

[0011] The historical traffic data includes vehicle identification information, vehicle passage time information, vehicle type, and travel data that can be used to estimate travel time;

[0012] S1.2 The target road segment or adjacent detection point interval is segmented according to the preset time window length to form an input sequence composed of multiple consecutive time windows;

[0013] S1.3 Obtain traffic operation status labels (operation status index TPI) aligned with the time window, and establish the training sample set required for end-to-end supervised learning;

[0014] The traffic operation status label is based on the FCM construction improved by the entropy weight method. It constructs a highway traffic flow operation status evaluation system with flow rate and speed as the core evaluation indicators, and divides the traffic status into six categories: very smooth, smooth, relatively smooth, relatively congested, congested and severely congested.

[0015] S2. Selection of gantry range and construction of ETC chain based on "time accessibility";

[0016] S3. Establish an end-to-end time series prediction model based on attention mechanism;

[0017] The end-to-end time-series prediction model includes a vehicle feature encoding module, an attention aggregation module for characterizing differences in vehicle contributions, a time-series modeling module for characterizing cross-time window evolution patterns, and a classifier. Through the above structure, it automatically learns the multi-level spatiotemporal correlation features of "vehicle-time window-road segment" and outputs the traffic operation status at the target prediction time.

[0018] S4. Online evolution strategy based on sliding window and experience playback;

[0019] We designed an online update mechanism that integrates sliding window and experience playback. While dynamically introducing road segment data, we mixed historical samples to balance the model's plasticity to new traffic environments and the stability of its memory of old knowledge. We also designed a dynamic learning rate adaptive adjustment strategy based on drift perception. This strategy dynamically assesses the degree of data distribution drift based on the model's prediction error of the current new window data and adaptively scales the learning rate to achieve continuous updates of the model in long-term monitoring.

[0020] S5. Online reasoning and congestion risk output;

[0021] During the real-time operation phase, the latest ETC passage data of the target location in the latest time window is continuously acquired. The real-time input sequence is generated according to the same time window division, feature encoding and standardization process as in the training phase, and then input into the end-to-end time series prediction model to output the traffic operation status of the target location under the preset prediction step size.

[0022] Furthermore, in step S1.3, during the construction of traffic operation status tags:

[0023] A silhouette coefficient was introduced to determine the number of clusters, which enhanced the objectivity and scientific rigor of the clustering results.

[0024] The entropy weight method is used to objectively assign weights to the importance of indicators, thereby obtaining the weight coefficients of each indicator, which can reflect the contribution of different indicators to the state information.

[0025] Furthermore, step S2 includes the following sub-steps:

[0026] S2.1 Target object and basic data definition;

[0027] Let the target gantry corresponding to the target road segment be... To retrieve historical ETC gantry vehicle passage records, each record must include at least: vehicle identification number. Inspection point signage Passing time and vehicle model;

[0028] The original records are cleaned, including at least: duplicate record removal, timestamp correction, and anomaly removal;

[0029] S2.2 Centered on the target road segment, it statistically analyzes the typical running time and propagation delay of vehicles in the road network based on historical ETC passage records;

[0030] Typical runtime:

[0031]

[0032]

[0033] in, Indicates upstream gantry To the target gantry Typical operating time of the road network; This is a quantile statistics function used to extract the q-quantiles of a travel time dataset; For all vehicles from to The historical travel time sample set; q is the preset quantile threshold; Vehicle ID passing through the target gantry The moment; For the same vehicle ID passing through the upstream gantry The moment;

[0034] Propagation delay margin term :

[0035]

[0036] in, These are empirical coefficients, and IQR is the interquartile range.

[0037] S2.3 Time Accessibility Criteria and Gantry Range Screening;

[0038] Upstream accessible gantry assembly:

[0039]

[0040]

[0041] in, For the time-reachable upstream effective gantry set; The original set of all upstream ETC gantries for the target road segment within the road network; H represents the model prediction duration; Input the total duration of the historical time series into the model; The number of consecutive time windows contained in the input sequence; A fixed duration for a single basic time window;

[0042] S2.4 Associate the continuous vehicle passage records between candidate detection points according to the actual passage sequence of the vehicles to construct the ETC passage chain between adjacent gantries;

[0043] S2.5 calculates travel time, travel speed, and their statistical characteristics on the ETC chain, forming a spatiotemporal correlation input representation for an end-to-end learning model. .

[0044] Furthermore, in step S3, the calculation process of each module of the end-to-end time series prediction model is as follows:

[0045] I. Vehicle Feature Encoding Module: Using the features calculated in step S2.5 as input, the vehicle embedding is obtained through two fully connected layers;

[0046] First layer fully connected:

[0047] Second layer fully connected:

[0048] In the formula, This represents the original ETC passage feature vector of vehicle v within the w-th time window of batch b. ; This is the bias vector for the first fully connected layer; This is the weight matrix for the first fully connected layer; Indicates batch normalization; It is a linear rectification activation function; The 64-dimensional intermediate hidden features of a single vehicle after passing through the first layer of the network; This is the weight matrix for the second fully connected layer; This is the bias vector for the second fully connected layer; The final output is the vehicle embedding vector;

[0049] II. Attention Aggregation Module: Calculates vehicle-level attention weights for all vehicle embedding vectors within a single time window, and aggregates them to obtain the global traffic flow features for the current time window;

[0050] The traffic flow features output by the continuous multi-time-window attention aggregation module are used as the input to the temporal modeling module;

[0051] III. Temporal Encoding Module: A bidirectional long short-term memory (Bi-LSTM) network is used to extract temporal features from aggregated feature sequences across multiple time windows. The forward and backward latent vectors are then concatenated to obtain global temporal features. ;

[0052] IV. Classifier, i.e., classification network;

[0053] Temporal features are processed through a three-layer fully connected network. Mapped to multi-class probability logits, the maximum value index is used to output the corresponding congestion level label;

[0054] During the model training phase, a weighted cross-entropy loss function is used to optimize the network parameters.

[0055] Furthermore, step S4 includes the following sub-steps:

[0056] S4.1 Sliding window sampling: Set the update step size and observation window, and each update extracts the recent real-time traffic flow data to form a new dataset;

[0057] S4.2 Hierarchical Experience Replay: Build a fixed-capacity buffer, randomly replace low-congestion samples, and forcibly retain high-congestion samples; extract historical samples proportionally and merge them with the new dataset to obtain a mixed training set;

[0058] S4.3 Drift Adaptive Learning Rate: Quantifies data drift using the model's cross-entropy loss on the new dataset, and dynamically calculates the fine-tuning learning rate for this round using the Sigmoid function;

[0059]

[0060]

[0061] in, and These are the upper and lower bounds of the learning rate allowed for online fine-tuning; The drift tolerance threshold that triggers a surge in the learning rate; The sigmoid activation function is used to smoothly map the scaling factors to... interval; To smooth out control parameters; This represents the average cross-entropy loss of the old model on the new traffic flow dataset at the k-th update. This is the latest real-time traffic flow dataset captured by the k-th round of the sliding window; The model is based on the parameters from the previous round. For the sample The congestion prediction output; For the sample The corresponding real-world road congestion TPI label; The cross-entropy loss function;

[0062] S4.4 Incremental Fine-tuning Update: The model is fine-tuned on the mixed training set using the Adam optimizer and adaptive learning rate. The training takes 5-10 rounds. If the F1 score does not improve after 3 consecutive rounds of verification, the process is terminated early. After completion, the model parameters are updated and the new dataset is cleared.

[0063] Beneficial effects:

[0064] 1. Accurate Traffic Condition Characterization and Classification: The FCM clustering algorithm, improved by the entropy weight method, is used to construct an evaluation system for highway traffic flow operation status with flow rate and speed as the core evaluation indicators. The traffic status is divided into six categories, which can more scientifically and accurately characterize and classify the traffic operation status of highways, providing reliable basic labels for subsequent prediction.

[0065] 2. Effective spatiotemporal correlation feature extraction: Based on "time accessibility", upstream gantries are selected to construct ETC chains, and statistical features such as travel time and speed are calculated to form an end-to-end input sequence representation. This can effectively extract spatiotemporal correlation features related to congestion on the target road segment, taking into account the actual traffic conditions of vehicles in the road network and the upstream and downstream influence relationships, making the model input more targeted and effective.

[0066] 3. Powerful model learning capability: The end-to-end time series prediction model based on the attention mechanism can automatically learn the multi-level spatiotemporal correlation features of "vehicle-time window-road segment" through modules such as vehicle feature encoding, attention aggregation, time series modeling and classification, and capture the complex relationship between different vehicles, time windows and road segments, so as to more accurately predict the traffic operation status.

[0067] 4. Excellent dynamic adaptability: The proposed online evolution strategy based on sliding window and experience playback can dynamically introduce the latest road segment data and mix historical typical samples to address the concept drift phenomenon in traffic flow data, balancing the model's plasticity to new traffic environments and the stability of its memory of old knowledge. At the same time, the dynamic learning rate adaptive adjustment strategy based on drift perception can adaptively scale the learning rate according to the degree of data distribution drift, enabling the model to continuously update in long-term monitoring, allowing the model to better adapt to constantly changing traffic conditions and maintain high prediction accuracy.

[0068] 5. Real-time and effective congestion risk warning: During the real-time operation phase, by continuously acquiring the latest ETC traffic data and performing online inference, when the predicted TPI level is ≥4, "congestion risk" is output and an alarm is triggered. This can form a stable short-term traffic state prediction sequence, providing timely and effective congestion risk warnings for highway management departments and travelers. This helps to take countermeasures in advance, alleviate traffic congestion, and improve the operational efficiency and service level of highways.

[0069] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0070] Figure 1 This is a flowchart of a highway congestion prediction method based on attention mechanism and online evolution according to the present invention;

[0071] Figure 2 A diagram showing the structure of an end-to-end traffic congestion prediction model that takes into account attention mechanisms;

[0072] Figure 3 This is a schematic diagram illustrating the online evolution strategy of sliding windows and experience playback. Detailed Implementation

[0073] To make the technical solutions, advantages, and objectives of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the protection scope of this application.

[0074] This embodiment proposes a highway congestion prediction method based on attention mechanisms and online evolution. The method uses traffic data generated by toll stations and ETC gantries as a foundation. Based on "time accessibility," it determines the range of gantries that influence the traffic state of the target location and constructs a continuous ETC chain for vehicles to form a stable spatiotemporal sequence input. On this basis, an end-to-end deep learning model is used to automatically extract and fuse temporal correlation features from multiple time windows and gantries, and a vehicle-level attention mechanism is introduced to enhance the ability to focus on key vehicles. Simultaneously, an online evolution strategy combining sliding windows and experience replay is designed to achieve long-term continuous model updates, avoiding model aging and catastrophic forgetting, and finally outputting the prediction results of the short-term traffic operation state of the target road segment.

[0075] like Figure 1 As shown, this invention provides a highway congestion prediction method based on attention mechanisms and online evolution, comprising the following steps:

[0076] Step 1: Construct an end-to-end learning sample and label system for ETC. Acquire historical traffic data from highway toll stations and ETC gantries. This data includes at least vehicle identification information, passage time information, vehicle type, and travel data that can be used to estimate travel time. Clean, deduplicate, and remove anomalies from the historical data. Divide the target road segment or adjacent detection point intervals into samples according to a preset time window length, forming an input sequence composed of multiple consecutive time windows. Simultaneously, acquire traffic operation status labels (Traffic Operation Status Index TPI) aligned with the time windows, establishing the training sample set required for end-to-end supervised learning. The traffic operation status labels are constructed based on an improved FCM using the entropy weight method, with flow rate and speed as the core evaluation indicators to build a highway traffic flow operation status evaluation system. Traffic states are classified into six categories: very smooth, smooth, relatively smooth, relatively congested, congested, and severely congested.

[0077] The traffic operation state classification results obtained by the FCM improved by the entropy weight method are introduced as the Traffic Performance Index of Link at the highway segment level. The classification criteria serve as labels for the model. The flow-density thresholds for each traffic flow state are determined based on the entropy weight method-improved FCM, and the traffic flow density index for road segments is defined using speed and flow thresholds according to the relationship between the three parameters: flow, density, and speed. .

[0078] Step 1.1: Data Acquisition and State Clustering Modeling. Collect ETC gantry vehicle passage data, clean the data, remove duplicates and handle outliers; calculate macroscopic characteristic indicators such as speed, flow and density of the corresponding road segment with a statistical period of 5 minutes, and use the FCM clustering algorithm with entropy weight method to perform cluster analysis on traffic flow operation status.

[0079] (1) Traffic status indicators

[0080] Traffic flow is the number of vehicles passing through a specific point or cross-section of a road per unit time at a given moment. Based on the definition, it can be mathematically described as follows:

[0081]

[0082] in for arrive Traffic flow within a specific time period; for The total number of vehicles passing through the observation point within the specified time period; This is the statistical time window interval.

[0083] Speed ​​refers to the distance a vehicle travels per unit of time, i.e., the vehicle's rate of movement, reflecting the movement of vehicles on the road and the efficiency of traffic flow. The average speed of a vehicle on a specific road segment can be calculated by the time difference between its passage through upstream and downstream gantries; this speed is called travel speed. Furthermore, by averaging the travel speeds of all vehicles within a set observation time window, the traffic flow speed for that time period can be obtained, reflecting the overall traffic conditions of that road segment. The mathematical definition is as follows:

[0084]

[0085] in, To calculate the average speed of the i-th vehicle within the statistical time window; Let m be the length of the m-th road segment; The time when the i-th vehicle enters the statistical road segment; The time when the i-th vehicle leaves the statistical road segment; for arrive Average vehicle speed over a given time period.

[0086] Traffic density refers to the number of vehicles contained within a unit length of a road segment at a given moment. According to traffic flow theory, traffic density can be calculated based on the relationships between parameters.

[0087]

[0088] in, express Traffic density at any given moment, i.e., per time window interval Calculate the traffic flow density once.

[0089] (2) Fuzzy C-means clustering algorithm

[0090] Fuzzy C-Means (FCM) clustering, as an unsupervised data classification method, can divide a dataset into... The method identifies several fuzzy categories and calculates the cluster center for each category. It iteratively optimizes the cluster centers and membership matrix alternately, causing the objective function to continuously decrease until it converges or meets a preset termination condition. Let the sample set be... ,in Indicates the first For each sample, the objective function of FCM is defined as:

[0091]

[0092] in, This is the membership matrix. Indicates the first The nth sample pair The membership degree of each cluster center, satisfying: . For the sample With cluster center The distance is usually expressed in Euclidean form, i.e.: m is the fuzzy weighting coefficient, used to adjust the degree of fuzziness in clustering, and is generally taken as... .

[0093] During initialization, the number of clusters needs to be predetermined during the FCM solution process. Maximum number of iterations Convergence threshold and the initial membership matrix In the first In the next iteration, the cluster centers are first updated based on the current membership matrix. Cluster center The update formula is:

[0094]

[0095] Subsequently, the membership degree of each sample to each category is updated based on the new cluster centers. The update formula is as follows:

[0096]

[0097]

[0098] Iteration stops when the maximum number of iterations is reached or the termination condition is met early. Common termination conditions can be expressed as:

[0099]

[0100] (3) Determining the number of clusters

[0101] Due to the initial cluster centers and the number of clusters The settings will have a significant impact on FCM results, while empirical selection The results are highly subjective. To enhance the objectivity and scientific rigor of the clustering results, the silhouette coefficient is introduced. The number of clusters is determined. The silhouette coefficient evaluates cluster quality by simultaneously measuring the tightness of samples within clusters and the separation between clusters; its value ranges from [value missing]. The closer the value is to 1, the better the clustering effect. For any sample Its profile coefficient is defined as:

[0102]

[0103] in For the sample The cohesion degree represents the average distance between a sample and other samples within the same cluster; For the sample The separation degree represents the average distance between a sample and its nearest neighbor samples. Let... The cluster to which it belongs is The sample size is ,but:

[0104]

[0105] For any non-cluster ,definition:

[0106]

[0107]

[0108] The average of the silhouette coefficients for all samples is:

[0109]

[0110] Calculate the optimal number of clusters within the candidate cluster number set, and select the cluster number with the largest average silhouette coefficient as the optimal number of clusters. When taking the maximum, the corresponding This represents the optimal number of clusters.

[0111] (4) Determination of indicator weights

[0112] In the classification of traffic flow operation states on highways, indicators such as speed and flow rate have varying abilities to represent traffic states. To reflect the different contributions of different indicators to state information, this paper uses the entropy weight method to objectively assign weights to the importance of indicators, thereby obtaining the weight coefficients of each indicator. Let there be a total of Individual indicators There are samples, and the original data matrix is... ,in For the first The sample at the th The values ​​for each indicator are taken. First, the data is dimensionless, resulting in... Methods such as range standardization were employed, and the first step was calculated. The first indicator The proportion of each sample:

[0113]

[0114] Thus, the first The entropy value of each indicator:

[0115]

[0116] And define the difference coefficient:

[0117]

[0118] The final indicator weights are:

[0119]

[0120] in Indicates the first Information entropy of each indicator Indicates the first The first indicator The proportion of each sample These are the weighting coefficients for the corresponding indicators. Using the entropy weighting results to construct weighted features or weighted distances can more reasonably reflect the differences in the contributions of indicators such as speed and flow to the classification of traffic flow operation states during the clustering process.

[0121] Step 1.2: Consistency Verification of Traffic State Classification and Mechanism. Traffic flow states are classified into six levels based on clustering output. Combining the macroscopic traffic flow basic diagram and three-phase traffic flow theory, the meaning of each category is interpreted accordingly, representing them as: Very Smooth, Smooth, Slightly Smooth, Slightly Congested, Congested, and Severely Congested.

[0122] Step 1.3: Threshold Extraction. Based on the distribution characteristics of each cluster category, extract the corresponding flow-density threshold range; further utilize the inherent constraint relationship among flow, density, and speed to combine the speed threshold and flow threshold to characterize the congestion level and state changes of the road segment.

[0123] Step 2: Selection of Gantry Range and Construction of ETC Chain Based on "Time Accessibility". Centered on the target road segment, based on historical ETC passage records, the typical travel time and propagation delay of vehicles in the road network are statistically analyzed to construct a "time accessibility" criterion reflecting the upstream and downstream influence range. Under the conditions of preset prediction lead and time window length, upstream gantries / toll stations that have a significant impact on the traffic state of the target location within the time accessibility range are selected as candidate detection points. Furthermore, the continuous vehicle passage records between candidate detection points are associated according to the actual vehicle passage order to construct an ETC passage chain (ETC chain) between adjacent gantries. The travel time, travel speed and their statistical characteristics are calculated on the ETC chain to form a spatiotemporal correlation input representation for the end-to-end learning model.

[0124] Step 2.1: Define the target object and basic data. Let the target gantry corresponding to the target road segment be... To retrieve historical ETC gantry vehicle passage records, each record must include at least: vehicle identification number. Inspection point signage Passing time Vehicle model. The original records should be cleaned, including at least: duplicate record removal, timestamp correction, and removal of obvious anomalies.

[0125] Step 2.2: Typical running time and propagation delay statistics. (Based on the target gantry) Centered on the upstream gantry that has a topological connection with it. To conduct statistics.

[0126] For any candidate upstream gantry Identify vehicles from historical data Drive to A valid sample, meaning the same vehicle exists. In terms of time sequence, it satisfies the condition of passing first. After passing Calculate the travel time for each sample:

[0127]

[0128] For each Typical runtime is obtained:

[0129]

[0130] in Desirable To adapt to the propagation and fluctuations of congestion, a propagation time delay margin term can be further introduced to reflect the propagation characteristics of traffic disturbances on the target cross section. To cover arrival time spread caused by queuing waves and vehicle speed fluctuations, it is set as follows:

[0131]

[0132] in The coefficient is an empirical coefficient, and IQR is the interquartile range; it can also be directly replaced by the historical quantile difference.

[0133] Step 2.3: Time Accessibility Criteria and Gantry Range Screening. Let the forecast lead time be... For example, in advance Predict the target state within minutes, with an input time window length of [time window length]. (For example (minutes), input sequence length is (common (One time window). The length of the historical time domain covered by the input is:

[0134]

[0135] Define gantry For the target gantry Under the current forecast settings, the criterion for "time-reachable" is: its typical runtime falls within the observable time domain of the model, i.e.

[0136]

[0137] When it is necessary to further control the number of gantry frames and avoid introducing weakly correlated noise from excessively distant gantry frames, a lower bound can be set simultaneously to exclude points that are "too close, causing information redundancy," thus forming a bilateral constraint:

[0138]

[0139] in, Desirable or .

[0140] Based on this, the set of upstream reachable gantry frames is obtained:

[0141]

[0142] Step 2.4: ETC chain construction based on candidate gantry set. Within the scope, according to vehicle identification The vehicle passage records are correlated and sorted by timestamp to form a vehicle passage sequence:

[0143]

[0144] For two consecutive vehicle passage records, if the following conditions are met:

[0145] ;

[0146] The time sequence is consistent and the time difference is within a reasonable range. This is used to exclude situations such as leaving the highway midway or making abnormal stops;

[0147] Two gantries can be directly adjacent in the road network topology or belong to a predefined set of adjacency relationships. ;

[0148] This is then treated as a valid link segment, forming an ETC link edge:

[0149]

[0150] Calculate the chain segment travel time and chain segment speed for each valid chain segment:

[0151]

[0152] in, This refers to the distance between gantry frames.

[0153] Step 2.5: Time window aggregation and end-to-end input representation generation. For each time window, the length is [length]. and each chain segment Summarize statistical features to form the spatiotemporal tensor of the model input. Then, use the target gantry. The statistical features of the corresponding chain segments of the gantry, along with their time reachability, are stacked in a "gantry dimension - feature dimension - time dimension" manner to obtain the input sequence representation of the end-to-end learning model:

[0154]

[0155] Each of them All are composed of chain segment features spliced ​​together within the selected gantry range.

[0156] Step 3: Establish an end-to-end temporal prediction model based on an attention mechanism. This step constructs an end-to-end learning network to map the input sequence to future congestion states. The model includes a vehicle feature encoding module, an attention aggregation module to characterize the differences in vehicle contributions, a temporal modeling module to characterize the evolutionary patterns across time windows, and a classifier. Through this structure, it automatically learns multi-level spatiotemporal correlation features of "vehicle-time window-road segment," outputting the traffic operation state at the target prediction time, thus predicting the short-term traffic state. In this embodiment, the structure of the end-to-end traffic congestion prediction model considering the attention mechanism can be referenced. Figure 2 .

[0157] Step 3.1: Vehicle Feature Encoding Module. Input Tensor For the feature vector of each vehicle First layer fully connected:

[0158]

[0159] in BN represents normalization.

[0160] Second layer fully connected:

[0161]

[0162] In the formula, This represents the original ETC passage feature vector of vehicle v within the w-th time window of batch b. ; This is the bias vector for the first fully connected layer; This is the weight matrix for the first fully connected layer; Indicates batch normalization; It is a linear rectification activation function; The 64-dimensional intermediate hidden features of a single vehicle after passing through the first layer of the network; This is the weight matrix for the second fully connected layer; This is the bias vector for the second fully connected layer; This is the final output vehicle embedding vector.

[0163] Step 3.2: Attention Aggregation Module. Let the current time window contain N cars, and let the high-dimensional feature vector of each car be denoted as... Attention score is calculated as follows:

[0164]

[0165] in, It is a non-linear activation function; and The weight matrix is ​​a learnable matrix. , This is the bias term. The model optimizes based on the error gradient of downstream congestion classification. and This enables it to accurately capture the characteristics of abnormal vehicles.

[0166] The attention weights are obtained by normalizing the scores using the Softmax function.

[0167]

[0168] The macroscopic eigenvector of this time window is obtained by weighted summation:

[0169]

[0170] Aggregate features from 4 time windows , , , The sequences are arranged in chronological order and used as input for the time series modeling module.

[0171] Step 3.3: Temporal Modeling Module. A 3-layer bidirectional long short-term memory network (Bi-LSTM) is used, with 256 hidden layer dimensions. The forward computation process for each LSTM unit is as follows:

[0172] Forgotten Gate:

[0173] Input Gate:

[0174] Candidate cell status:

[0175] Cell status update:

[0176] Output gate:

[0177] Hidden state:

[0178] in This is the Sigmoid activation function, and * indicates element-wise multiplication.

[0179] Unlike unidirectional LSTM, Bi-LSTM contains two parallel hidden layer computation paths: one processes the input sequence in ascending chronological order from front to back to obtain the forward hidden state. The other approach processes the input sequence in reverse chronological order, from back to front, to obtain the backward hidden state. At each time step The final output of Bi-LSTM is the concatenation of the hidden states in these two directions:

[0180]

[0181] Dropout is introduced between network layers with a rate of 0.3 to improve generalization ability. After processing the entire sequence of length W=4, this model only takes the output of the last time step as a comprehensive representation of the traffic state of the sequence:

[0182]

[0183] And The input is fed into the subsequent classification network to achieve TPI state determination.

[0184] Step 3.4: Classification Network. The classification network maps the 512-dimensional spatiotemporal features output by the prediction network to the category space of the Traffic Performance Index (TPI). This module employs a three-layer feedforward neural network. First, the... Mapped to 256 dimensions, and combined with BatchNorm1d and high-inactivation-rate Dropout, with a rate of 0.4, generalization ability is improved.

[0185]

[0186] The second layer further reduces the dimensionality to 128 dimensions:

[0187]

[0188] The final layer is linearly mapped to the number of categories (6), and outputs the unnormalized logits:

[0189]

[0190] The final layer of this model outputs logits, rather than directly outputting discrete class labels. During the inference phase, the final predicted class can be obtained by taking the index of the highest score along the class dimension.

[0191]

[0192] Step 3.5: Loss Function and Optimization. A weighted cross-entropy loss function is used to mitigate the class imbalance problem. Let the number of samples in a batch be... The total number of categories is The model for the first Each sample outputs an unnormalized logits vector. The corresponding softmax probability is:

[0193]

[0194] Let the real label be Category weight is The weighted cross-entropy loss is then defined as:

[0195]

[0196] Class weight The calculation method is consistent with the code: Let the number of elements in the training set be... The number of samples in each class is The total number of samples in the training set is ,but

[0197]

[0198] This weight design assigns a larger weight to categories with fewer samples. This allows for stronger gradient updates during backpropagation; for missing classes in the training set, a penalty weight is set to 10 times the maximum weight of the classes that have already appeared.

[0199] Its update form can be represented as:

[0200]

[0201]

[0202] The learning rate is set to , , , Meanwhile, to suppress excessively large parameters and improve generalization ability, the optimizer introduces a weight decay coefficient:

[0203] To avoid the gradient explosion problem that may occur during the training of recurrent neural networks, the gradient norm is pruned during the training phase, with a threshold set to 1.0.

[0204]

[0205] An adaptive learning rate scheduler, ReduceLROnPlateau, based on validation set accuracy, is employed. Validation set accuracy is monitored during training. When it is in continuous If the learning rate does not improve within a certain number of epochs, it will be proportionally reduced to 0.5 of its original value. Furthermore, this study employs an early stopping strategy to prevent overfitting: if the accuracy on the validation set is continuously improved... If the model does not exceed the historical best value within a certain number of epochs, training will be terminated and the model parameters will be restored to the time when the accuracy on the validation set is optimal.

[0206] Step 4: Online Evolution Strategy Based on Sliding Window and Experience Replay. This step addresses the concept drift phenomenon in traffic flow data by designing an online update mechanism that integrates sliding window and experience replay. This dynamically introduces the latest road segment data while mixing in historical typical samples, balancing the model's adaptability to new traffic environments with the stability of its retention of old knowledge. A dynamic learning rate adaptive adjustment strategy based on drift perception is designed. This strategy dynamically assesses the degree of data distribution drift based on the model's prediction error for the current new window data, adaptively scaling the learning rate to achieve continuous model updates during long-term monitoring. In this embodiment, a schematic diagram of the online evolution strategy using sliding window and experience replay can be found here. Figure 3 .

[0207] Step 4.1: Sliding Window Mechanism Design. Let the initial offline training model parameters be... Set update step size during the online phase. Heaven, observation window size =3 days. At the kth update (corresponding time) When ), extract the time interval. The streaming data within is used as a new dataset The parameter evolution process is represented as follows:

[0208]

[0209] Step 4.2: Construction and Maintenance of the Experience Replay Buffer. Construct an experience memory buffer B with a capacity M = 10000. Maintain the buffer using a priority sampling strategy based on congestion level. When a new dataset... Upon arrival, samples are stratified and screened based on their actual TPI level:

[0210] Step 1: Sample Input and Replacement Rules: When a new dataset is generated... Upon arrival, samples are stratified and screened based on their actual TPI level. For normally congested samples (such as TPI-1 and TPI-2), random replacement is performed with a certain decay probability to prevent them from consuming excessive memory. Conversely, for rare, severely congested samples (such as TPI-5 and TPI-6), they are given absolute high priority and forcibly retained in the buffer. middle.

[0211] Step 2. Data Mixing Mechanism: In the... During the next update, from the buffer Mix according to the preset ratio Randomly select a subset of historical samples Constructing a hybrid evolutionary dataset :

[0212]

[0213] Among them, the mixing ratio The extent to which the model retains old knowledge was controlled.

[0214] Step 4.3: Dynamic learning rate adaptive adjustment strategy. During the online evolution phase, since the model already has good initial parameters... If a fixed, large learning rate is continued, it will not only disrupt the already converged attention mechanism weights and feature extraction layers but also cause oscillations in the training process. Conversely, if the learning rate is too small, the model cannot effectively cross the loss plateau and struggles to adapt to sudden concept drift. Therefore, a drift-aware dynamic learning rate adaptive adjustment strategy needs to be designed. The core idea is to dynamically assess the degree of drift in the road network data distribution using the model's prediction error for the current new window data, and then adaptively scale the learning rate. The specific steps are as follows:

[0215] Step 1. In the... Before the next update, first keep the current parameters. Freeze, computational model on the latest observation dataset Forward propagation average cross-entropy loss :

[0216]

[0217] Step 2. Use this loss value as a proxy for the non-stationarity of the current traffic flow to calculate the adaptive learning rate for this incremental fine-tuning. :

[0218]

[0219] in and These represent the upper and lower bounds of the learning rate allowed for online fine-tuning (typically much smaller than the learning rate used in offline training, for example, setting...). , ). The drift tolerance threshold that triggers a surge in the learning rate. The sigmoid activation function is used to smoothly map the scaling factors to... Interval. For smooth control parameters.

[0220] Step 4.4: Fine-tune the training process. Use a hybrid dataset. The model was fine-tuned, with the optimizer still using Adam and the learning rate set to the value calculated above. Other hyperparameters remain the same as in offline training. Fine-tuning epochs are set to 5-10. After each epoch, the weighted cross-entropy loss and F1 score on the validation set are calculated. If the F1 score does not improve after three consecutive epochs, the process terminates early. After fine-tuning, the model parameters are updated. Clear the new dataset and prepare for the next update.

[0221] Step 5: Online Inference and Congestion Risk Output. In the real-time operation phase, this step continuously acquires the latest ETC traffic data for the target location within the latest time window. Following the same time window division, feature encoding, and standardization process as the training phase, a real-time input sequence is generated and input into the end-to-end model. The model then outputs the traffic operation status of the target location within a preset prediction step size.

[0222] Step 5.1: Online Data Acquisition and Time Window Rolling Update. During the online operation phase, continuously access the ETC toll data stream corresponding to the target location. Merge real-time data every 5 minutes, the same time window length as in the training phase, to generate the current time window ID. When a new time window ends, fix all vehicle records within that time window as the "latest time window data". Maintain a time window queue of length 4: When a new time window arrives, add the latest window data to the right end of the queue, while removing the oldest time window data, always maintaining 4 consecutive time windows as the current inference input sequence.

[0223] Step 5.2: End-to-end online model inference. The online system loads the optimal model parameters obtained in Step 4. Set the model to evaluation mode (eval()) and disable gradient computation. For the current input tensor, first pass it through the vehicle feature encoding module to obtain the high-dimensional feature vector of each vehicle. Then perform forward propagation:

[0224] Step 1. Attention Aggregation Module: Calculate Attention Score:

[0225]

[0226] Then, the weights are obtained by normalization:

[0227]

[0228] Finally, the weighted summation yields the time window characteristics:

[0229]

[0230] Step 2. Temporal Modeling Module: Input the features of the four time windows into a three-layer bidirectional LSTM sequentially to obtain the comprehensive sequence representation. .

[0231] Step 3. Classifier: ... Output via a three-layer fully connected network The predicted category index is:

[0232]

[0233] The TPI level is obtained through index_to_tpi mapping.

[0234] Step 5.3: Congestion Risk Output and Alarm. The online inference output includes at least: the prediction time window ID (i.e., the first window after the current window) and the predicted TPI level / traffic status category. When the predicted TPI reaches the preset congestion threshold level (e.g., TPI ≥ 4), the system outputs "Congestion Risk" and triggers an alarm. The system continuously executes the above process according to the rolling time window cycle, forming a stable short-term traffic status prediction sequence.

[0235] It is hereby declared that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A highway congestion prediction method based on attention mechanism and online evolution, characterized in that, Includes the following steps: S1. Construct an end-to-end learning sample and label system for ETC; S1.1 Obtain historical traffic data from highway toll stations and ETC gantries, and clean, deduplicate, and process outliers from the historical traffic data; The historical traffic data includes vehicle identification information, vehicle passage time information, vehicle type, and travel data that can be used to estimate travel time; S1.2 The target road segment or adjacent detection point interval is segmented according to the preset time window length to form an input sequence composed of multiple consecutive time windows; S1.3 Obtain traffic operation status labels (operation status index TPI) aligned with the time window, and establish the training sample set required for end-to-end supervised learning; The traffic operation status label is based on the FCM construction improved by the entropy weight method. It constructs a highway traffic flow operation status evaluation system with flow rate and speed as the core evaluation indicators, and divides the traffic status into six categories: very smooth, smooth, relatively smooth, relatively congested, congested and severely congested. S2. Selection of gantry range and construction of ETC chain based on "time accessibility"; S3. Establish an end-to-end time series prediction model based on attention mechanism; The end-to-end time series prediction model includes a vehicle feature encoding module, an attention aggregation module for characterizing the differences in vehicle contributions, a time series modeling module for characterizing the evolution pattern across time windows, and a classifier. Through the above structure, it automatically learns the multi-level spatiotemporal correlation features of "vehicle-time window-road segment" and outputs the traffic operation status at the target prediction time. S4. Online evolution strategy based on sliding window and experience playback; The design incorporates an online update mechanism that integrates sliding windows and experience playback, dynamically introducing road segment data while mixing historical samples to balance the model's adaptability to new traffic environments with the stability of its memory of old knowledge. We design a dynamic learning rate adaptive adjustment strategy based on drift awareness. This strategy dynamically assesses the degree of data distribution drift based on the model's prediction error for the current new window data, and adaptively scales the learning rate to achieve continuous model updates during long-term monitoring. S5. Online reasoning and congestion risk output; During the real-time operation phase, the latest ETC passage data of the target location in the latest time window is continuously acquired. The real-time input sequence is generated according to the same time window division, feature encoding and standardization process as in the training phase, and then input into the end-to-end time series prediction model to output the traffic operation status of the target location under the preset prediction step size.

2. The highway congestion prediction method based on attention mechanism and online evolution according to claim 1, characterized in that, In step S1.3, during the construction of traffic operation status labels: A silhouette coefficient was introduced to determine the number of clusters, which enhanced the objectivity and scientific rigor of the clustering results. The entropy weight method is used to objectively assign weights to the importance of indicators, thereby obtaining the weight coefficients of each indicator, which can reflect the contribution of different indicators to the state information.

3. The highway congestion prediction method based on attention mechanism and online evolution according to claim 2, characterized in that, Step S2 includes the following sub-steps: S2.1 Target object and basic data definition; Let the target gantry corresponding to the target road segment be... To retrieve historical ETC gantry vehicle passage records, each record must include at least: vehicle identification number. Inspection point signage Passing time and vehicle model; The original records are cleaned, including at least: duplicate record removal, timestamp correction, and anomaly removal; S2.2 Centered on the target road segment, it statistically analyzes the typical running time and propagation delay of vehicles in the road network based on historical ETC passage records; Typical runtime: in, Indicates upstream gantry To the target gantry Typical operating time of the road network; This is a quantile statistics function used to extract the q-quantiles of a travel time dataset; For all vehicles from to The historical travel time sample set; q is the preset quantile threshold; Vehicle ID passing through the target gantry The moment; For the same vehicle ID passing through the upstream gantry The moment; Propagation delay margin term : in, These are empirical coefficients, and IQR is the interquartile range. S2.3 Time Accessibility Criteria and Gantry Range Screening; Upstream accessible gantry assembly: in, For the time-reachable upstream effective gantry set; The original set of all upstream ETC gantries for the target road segment within the road network; H represents the model prediction duration; Input the total duration of the historical time series into the model; The number of consecutive time windows contained in the input sequence; A fixed duration for a single basic time window; S2.4 Associate the continuous vehicle passage records between candidate detection points according to the actual passage sequence of the vehicles to construct the ETC passage chain between adjacent gantries; S2.5 calculates travel time, travel speed, and their statistical characteristics on the ETC chain, forming a spatiotemporal correlation input representation for an end-to-end learning model. .

4. The highway congestion prediction method based on attention mechanism and online evolution according to claim 3, characterized in that, In step S3, the calculation process of each module of the end-to-end time series prediction model is as follows: I. Vehicle Feature Encoding Module: Using the features calculated in step S2.5 as input, the vehicle embedding is obtained through two fully connected layers; First layer fully connected: Second layer fully connected: In the formula, This represents the original ETC passage feature vector of vehicle v within the w-th time window of batch b. ; This is the bias vector for the first fully connected layer; This is the weight matrix for the first fully connected layer; Indicates batch normalization; It is a linear rectification activation function; The 64-dimensional intermediate hidden features of a single vehicle after passing through the first layer of the network; This is the weight matrix for the second fully connected layer; This is the bias vector for the second fully connected layer; The final output is the vehicle embedding vector; II. Attention Aggregation Module: Calculates vehicle-level attention weights for all vehicle embedding vectors within a single time window, and aggregates them to obtain the global traffic flow features for the current time window; The traffic flow features output by the continuous multi-time-window attention aggregation module are used as the input to the temporal modeling module; III. Temporal Encoding Module: A bidirectional long short-term memory (Bi-LSTM) network is used to extract temporal features from aggregated feature sequences across multiple time windows. The forward and backward latent vectors are then concatenated to obtain global temporal features. ; IV. Classifier, i.e., classification network; Temporal features are processed through a three-layer fully connected network. Mapped to multi-class probability logits, the maximum value index is used to output the corresponding congestion level label; During the model training phase, a weighted cross-entropy loss function is used to optimize the network parameters.

5. The highway congestion prediction method based on attention mechanism and online evolution according to claim 4, characterized in that, Step S4 includes the following sub-steps: S4.1 Sliding window sampling: Set the update step size and observation window, and each update extracts the recent real-time traffic flow data to form a new dataset; S4.2 Hierarchical Experience Replay: Build a fixed-capacity buffer, randomly replace low-congestion samples, and forcibly retain high-congestion samples; extract historical samples proportionally and merge them with the new dataset to obtain a mixed training set; S4.3 Drift Adaptive Learning Rate: Quantifies data drift using the model's cross-entropy loss on the new dataset, and dynamically calculates the fine-tuning learning rate for this round using the Sigmoid function; in, and These are the upper and lower bounds of the learning rate allowed for online fine-tuning; The drift tolerance threshold that triggers a surge in the learning rate; The sigmoid activation function is used to smoothly map the scaling factors to... interval; To smooth out control parameters; This represents the average cross-entropy loss of the old model on the new traffic flow dataset at the k-th update. This is the latest real-time traffic flow dataset captured by the k-th round of the sliding window; The model is based on the parameters from the previous round. For the sample The congestion prediction output; For the sample The corresponding real-world road congestion TPI label; The cross-entropy loss function; S4.4 Incremental Fine-tuning Update: The model is fine-tuned on the mixed training set using the Adam optimizer and adaptive learning rate. The training takes 5-10 rounds. If the F1 score does not improve after 3 consecutive rounds of verification, the process is terminated early. After completion, the model parameters are updated and the new dataset is cleared.

Images