Expressway congestion prediction method based on ETC data end-to-end learning

Abstract

The invention belongs to the technical field of intelligent traffic, and discloses an ETC data end-to-end learning-based highway congestion prediction method, which comprises the following steps of: directly constructing a time sequence sample by vehicle passing and travel data generated by an ETC gantry / toll station, automatically completing feature extraction and time sequence correlation modeling under a unified learning framework, and automatically extracting the feature of the vehicle passing and travel data; the congestion evolution characteristics of a vehicle layer, a time window layer and a cross-time window are learned through an end-to-end deep learning model, and direct prediction from original ETC spatio-temporal data to a future congestion state is realized. According to the method, the dependence on a congestion speed threshold, an artificial feature segment and a similarity matching rule can be reduced, and the adaptability and robustness to multi-source disturbance and abnormal scenes are improved, so that a more reliable data support is provided for highway congestion early warning and active management and control.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically relating to a highway congestion prediction method based on end-to-end learning of ETC data. Background Technology

[0002] In recent years, the rapid development of Intelligent Transportation Systems (ITS) has provided new solutions for alleviating highway congestion. ITS integrates advanced information technology, communication technology, and data processing technology to improve the efficiency, safety, and sustainability of transportation systems. As an important component of ITS, Electronic Toll Collection (ETC) has been widely adopted globally. ETC systems automatically identify vehicles and complete toll collection, improving toll collection efficiency and accumulating a wealth of traffic data, including vehicle travel time and location information. This data is highly accurate, real-time, and continuous, providing a valuable data source for traffic condition analysis and prediction.

[0003] However, although ETC data has been initially applied in areas such as traffic flow prediction, travel time estimation, and congestion detection, its potential in highway congestion prediction has not yet been fully explored. Traditional research methods typically rely on statistical models or shallow machine learning algorithms, which often have limitations when dealing with complex traffic data, struggling to capture nonlinear relationships and spatiotemporal dependencies. Furthermore, traditional methods usually require complex feature engineering and model design, resulting in limited prediction efficiency and accuracy.

[0004] Patent document CN121148150A calculates the travel speed between adjacent detection points and constructs an average speed change map. It then extracts speed change feature segments before congestion based on a congestion speed threshold, and matches real-time speed changes with historical feature segments to output a congestion risk coefficient. While this modular congestion prediction method is intuitive and relatively simple to implement, it is heavily reliant on threshold settings and feature segment extraction rules, making it difficult to fully characterize the nonlinear evolution of traffic conditions. Furthermore, when real-time traffic conditions differ significantly from historical sample patterns or when new congestion patterns emerge, the matching mechanism is prone to misjudgment.

[0005] With the rapid development of artificial intelligence technology, end-to-end learning (E2E), as an emerging machine learning method, is gradually demonstrating great potential in the field of traffic prediction. End-to-end learning can directly extract features from raw data and establish complex mapping relationships between inputs and outputs, avoiding the cumbersome feature engineering process of traditional methods, thereby improving the accuracy and robustness of predictions. Applying end-to-end learning methods to highway congestion prediction based on ETC data can not only fully utilize the rich information of ETC data and improve prediction accuracy, but also provide new theoretical support and methodologies for the field of traffic prediction.

[0006] Given this background, there is an urgent need for a highway congestion prediction method that can fully utilize the coverage advantages and spatiotemporal continuity of ETC data, reduce manual thresholds and rule dependence, and maintain good generalization ability under complex traffic conditions. Summary of the Invention

[0007] Existing modular congestion prediction methods based on threshold rules or feature segment matching suffer from several drawbacks under complex traffic conditions, such as cross-road segments and cross-time periods. These problems include difficulty in unifying thresholds and insufficient feature representation. More seriously, errors between modules accumulate and amplify at each level, leading to a significant decrease in prediction stability and generalization ability. To achieve more accurate prediction and stable early warning of highway congestion, this invention proposes a highway congestion prediction method based on end-to-end learning of ETC data. This method organically combines the spatiotemporal information of ETC gantry / toll station traffic data with a deep learning model. Through an end-to-end mapping from raw data to the final prediction result, it avoids error accumulation in intermediate stages, automatically learns congestion evolution patterns, significantly reduces manual feature design and threshold dependence, and achieves high-precision and robust prediction of future traffic conditions for target road segments.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A highway congestion prediction method based on end-to-end learning of ETC data includes the following steps:

[0010] S1. Construct an end-to-end learning sample and label system for ETC;

[0011] S2. Selection of gantry range and construction of ETC chain based on "time accessibility";

[0012] S3. Establish an end-to-end time-series prediction model based on ETC data;

[0013] S4. Model training and adaptive optimization of imbalanced samples;

[0014] S5. Online reasoning and congestion risk output.

[0015] Furthermore, step S1 includes the following sub-steps:

[0016] S1.1 Data Acquisition and State Clustering Modeling;

[0017] Historical traffic data from highway toll stations and ETC gantries is collected, and the historical traffic data is cleaned, deduplicated, and anomaly removed. A statistical period is set, traffic status indicators for the corresponding road segments are calculated, and the FCM clustering algorithm with entropy weighting is used to perform cluster analysis on traffic flow operation status. The historical traffic data includes vehicle identification information, vehicle passage time information, vehicle type, and travel data that can be used to estimate travel time.

[0018] I. Traffic condition indicators: including flow rate, speed, and density;

[0019] II. Fuzzy C-means clustering algorithm: By iteratively and alternately optimizing the cluster center and membership matrix, the objective function is continuously reduced until the objective function converges or meets the preset termination condition.

[0020] III. Number of clusters: The silhouette coefficient is introduced to determine the number of clusters;

[0021] For any sample Its profile coefficient is defined as:

[0022]

[0023] In the formula, Represents the silhouette coefficient of the i-th sample; For the sample The cohesion degree represents the average distance between a sample and other samples within the same cluster; For the sample The separation degree represents the average distance between it and its nearest neighbor samples from other clusters;

[0024] set up The cluster to which it belongs is The sample size is ,but:

[0025]

[0026] For any non-cluster ,definition:

[0027]

[0028]

[0029] The average of the contour coefficients of all samples for:

[0030]

[0031] Calculate the optimal number of clusters within the candidate cluster number set, and select the cluster number with the largest average silhouette coefficient as the optimal number of clusters. When taking the maximum, the corresponding The optimal number of clusters;

[0032] IV. Determination of indicator weights: The entropy weight method is used to objectively assign weights to the importance of indicators, thereby obtaining the weight coefficients of each indicator;

[0033] S1.2 Consistency verification of traffic state classification and mechanism;

[0034] Traffic flow status is classified into six levels based on clustering output. Combined with the macroscopic traffic flow basic map and three-phase traffic flow theory, the traffic meaning of each category is interpreted accordingly, and is represented as follows: very smooth, smooth, relatively smooth, relatively congested, congested, and severely congested.

[0035] S1.3 Threshold Extraction;

[0036] Based on the distribution characteristics of each cluster category, the corresponding flow-density threshold range is extracted; further, utilizing the inherent constraint relationship among flow, density, and speed, a road segment traffic flow density index is defined. .

[0037] Furthermore, the specific content of step S2 is as follows:

[0038] S2.1 Target object and basic data definition;

[0039] Let the target gantry corresponding to the target road segment be... To retrieve historical ETC gantry vehicle passage records, each record must include at least: vehicle identification number. Inspection point signage Passing time Vehicle model;

[0040] S2.2 Typical running time and propagation delay statistics;

[0041] Target gantry Centered on the upstream gantry that has a topological connection with it. To conduct statistics;

[0042] For any candidate upstream gantry Identify vehicles from historical data Drive to A valid sample, meaning the same vehicle exists. In terms of time sequence, it satisfies the condition of passing first. After passing ;

[0043] For each Typical runtime is obtained:

[0044]

[0045] In the formula, Indicates that the vehicle is from Drive to A sample set of travel time; Representative to The typical running time obtained after taking the q-quantile represents the time taken under normal or slightly congested conditions from... arrive The representative travel time; q is the quantile value;

[0046] Introducing a propagation delay margin term This is used to cover arrival time extensions caused by queuing waves and vehicle speed fluctuations.

[0047]

[0048] In the formula, Indicates from arrive The propagation delay margin; Empirical coefficients; IQR is the interquartile range;

[0049] S2.3 Time Accessibility Criteria and Gantry Range Screening;

[0050] gantry For the target gantry Under the current forecast settings, the criterion for "time-awareness" is: its typical runtime falls within the observable time domain of the model, i.e.

[0051]

[0052]

[0053] In the formula, H represents the forecast lead time, indicating how far in advance the forecast of the target gantry is made. Predict the state; This indicates the length of the input time window, i.e., the duration of a single input time window; L represents the length of the input sequence. Input the length of the historical time domain covered;

[0054] A lower bound is set to exclude points that are "too close, causing information redundancy," thus forming a two-sided constraint:

[0055]

[0056] In the formula, Value or That is, the length of one or two time windows;

[0057] Based on this, the set of upstream reachable gantry frames is obtained:

[0058]

[0059] S2.4 ETC chain construction based on candidate gantry set;

[0060] exist Within the scope, according to vehicle identification The vehicle passage records are correlated and sorted by timestamp to form a vehicle passage sequence:

[0061] For two consecutive vehicle passage records, if the following conditions are met: The time sequence is consistent and the time difference is within a reasonable range. Two gantries can be directly adjacent in the road network topology or belong to a predefined set of adjacency relationships. ;

[0062] This is then treated as a valid link segment, forming an ETC link edge:

[0063]

[0064] In the formula, e represents a valid ETC link segment; Indicates that the vehicle has passed through the gantry. timestamp; Indicates that the vehicle has passed through the gantry. timestamp;

[0065] Calculate the segment travel time and segment speed for each valid chain segment:

[0066]

[0067] in, Indicates vehicle ID from Drive to The actual time spent; For gantry and The physical distance between them;

[0068] S2.5 Time Window Aggregation and End-to-End Input Representation Generation;

[0069] I. With For each time window, the length is [length]. and each chain segment Summarize the statistical features to form the spatiotemporal tensor for model input;

[0070] II. Target gantry The statistical features of the corresponding chain segments of the gantry, along with their time reachability, are stacked in a "gantry dimension - feature dimension - time dimension" manner to obtain the input sequence representation of the end-to-end learning model:

[0071]

[0072] Each of them In the time window Within this time window, a tensor composed of the statistical characteristics of all chain segments within the selected gantry range represents a snapshot of the traffic state.

[0073] Furthermore, the end-to-end time series prediction model constructed in step S3 includes:

[0074] Vehicle feature encoding module: used to learn high-dimensional representations related to traffic conditions from the micro-features of ETC vehicles;

[0075] Attention aggregation module: used to distinguish the differences in contributions of different vehicles to state discrimination, and to achieve adaptive weighted fusion at the vehicle layer;

[0076] Temporal modeling module: used to learn the dynamic evolution patterns across time windows, enabling the characterization of the congestion formation and propagation process;

[0077] Time-series classification output module: Used to output the traffic operation status category at the target prediction time, thereby realizing end-to-end prediction of the short-term traffic status in the future.

[0078] Furthermore, the specific content of step S4 is as follows:

[0079] The end-to-end model is trained under supervision using historical samples. A cross-entropy loss function with class weights is used to alleviate the training bias caused by the imbalance of traffic state level samples. During training, weight decay, gradient pruning, and adaptive adjustment of the learning rate based on the performance of the validation set are combined to improve training stability. On the validation set, early stopping and optimal model parameter saving / rollback are performed based on the validation accuracy to obtain model parameters with stronger generalization ability, thereby improving the prediction stability under different road segments, different date types, and different traffic demand levels.

[0080] Furthermore, the specific content of step S5 is as follows:

[0081] During the real-time operation phase, the latest ETC passage data of the target location in the latest time window is continuously acquired. The real-time input sequence is generated according to the same time window division, feature encoding and standardization process as in the training phase, and then input into the end-to-end model to output the traffic operation status of the target location under the preset prediction step size.

[0082] Beneficial effects:

[0083] This invention proposes a highway congestion prediction method based on end-to-end learning of ETC data, using existing ETC detection equipment data. It directly constructs time-series samples from vehicle traffic and trip data generated by ETC gantries / toll stations, automatically performing feature extraction and temporal correlation modeling within a unified learning framework. Through an end-to-end deep learning model, it learns congestion evolution characteristics at the vehicle layer, time window layer, and across time windows, achieving direct prediction of future congestion states from raw ETC spatiotemporal data. This method reduces reliance on congestion speed thresholds, manual feature segments, and similarity matching rules, improving adaptability and robustness to multi-source disturbances and abnormal scenarios, thus providing more reliable data support for highway congestion early warning and proactive management.

[0084] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0085] Figure 1 This is a flowchart of a highway congestion prediction method based on end-to-end learning of ETC data according to the present invention.

[0086] Figure 2 The difference between the end-to-end model and the traditional model;

[0087] Figure 3 This is a structural diagram of an end-to-end highway congestion prediction model based on MLP-Att-BiLSTM. Detailed Implementation

[0088] To make the technical solutions, advantages, and objectives of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the protection scope of this application.

[0089] This embodiment proposes a highway congestion prediction method based on end-to-end learning of ETC data. This method uses traffic data generated by toll stations and ETC gantries as a foundation, determines the range of gantries that influence the traffic state of the target location based on "time accessibility," and constructs a continuous ETC chain for vehicles to form a stable spatiotemporal sequence input. Based on this, an end-to-end deep learning model is used to automatically extract and fuse the temporal correlation features of multiple time windows and multiple gantries, achieving a predictive output of the short-term traffic operation status of the target road segment. Compared to traditional methods that rely on speed thresholds or feature segment matching, this method reduces manual rule and threshold parameter tuning, has stronger generalization ability for cross-road segment, cross-time period, and abnormal disturbance scenarios, and can quickly output congestion risk results in real-time operation, providing a quantitative basis for congestion warning and the initiation of proactive traffic management strategies. In this embodiment, the differences between the end-to-end learning highway congestion prediction model and traditional models can be found in [reference needed]. Figure 2 .

[0090] like Figure 1 As shown, a highway congestion prediction method based on end-to-end learning of ETC data includes the following steps:

[0091] Step 1: Construct an end-to-end learning sample and label system for ETC. Acquire historical traffic data from highway toll stations and ETC gantries. This data includes at least vehicle identification information, passage time information, vehicle type, and travel data that can be used to estimate travel time. Clean, deduplicate, and remove anomalies from the historical data. Divide the target road segment or adjacent detection point intervals into samples according to a preset time window length, forming an input sequence composed of multiple consecutive time windows. Simultaneously, acquire traffic operation status labels (Traffic Operation Status Index TPI) aligned with the time windows, establishing the training sample set required for end-to-end supervised learning. The traffic operation status labels are constructed based on an improved FCM using the entropy weight method, with flow rate and speed as the core evaluation indicators to build a highway traffic flow operation status evaluation system. Traffic states are classified into six categories: very smooth, smooth, relatively smooth, relatively congested, congested, and severely congested.

[0092] The traffic operation state classification results obtained by the FCM improved by the entropy weight method are introduced as the Traffic Performance Index of Link at the highway segment level. The classification criteria serve as labels for the model. The flow-density thresholds for each traffic flow operating state are determined based on the improved FCM using the entropy weight method. The traffic flow density index (TPI) for road segments is defined using the speed and flow thresholds according to the relationship between the three parameters: flow, density, and speed.

[0093] Step 1.1: Data Acquisition and State Clustering Modeling. Collect ETC gantry vehicle passage data, clean the data, remove duplicates and handle outliers; calculate macroscopic characteristic indicators such as speed, flow and density of the corresponding road segment with a statistical period of 5 minutes, and use the FCM clustering algorithm with entropy weight method to perform cluster analysis on traffic flow operation status.

[0094] (1) Traffic condition indicators

[0095] Traffic flow is the number of vehicles passing through a specific point or cross-section of a road per unit time at a given moment. Based on the definition, it can be mathematically described as follows:

[0096]

[0097] in, for arrive Traffic flow within a specific time period; for arrive The total number of vehicles passing through the observation point within the time period; This is the statistical time window interval.

[0098] Speed ​​refers to the distance a vehicle travels per unit of time, i.e., the vehicle's rate of movement, reflecting the movement of vehicles on the road and the efficiency of traffic flow. The average speed of a vehicle on a specific road segment can be calculated by the time difference between its passage through upstream and downstream gantries; this speed is called travel speed. Furthermore, by averaging the travel speeds of all vehicles within a set observation time window, the traffic flow speed for that time period can be obtained, reflecting the overall traffic conditions of that road segment. The mathematical definition is as follows:

[0099]

[0100] in, To calculate the average speed of the i-th vehicle within the statistical time window; Let m be the length of the m-th road segment; The time when the i-th vehicle enters the statistical road segment; The time when the i-th vehicle leaves the statistical road segment; for arrive Average vehicle speed over a given time period.

[0101] Traffic density refers to the number of vehicles contained within a unit length of a road segment at a given moment. According to traffic flow theory, traffic density can be calculated based on the relationships between parameters.

[0102]

[0103] in, express Traffic density at any given moment, i.e., per time window interval Calculate the traffic flow density once.

[0104] (2) Fuzzy C-means clustering algorithm

[0105] Fuzzy C-Means (FCM) clustering, as an unsupervised data classification method, can divide a dataset into... The method identifies several fuzzy categories and calculates the cluster center for each category. It iteratively optimizes the cluster centers and membership matrix alternately, causing the objective function to continuously decrease until it converges or meets a preset termination condition. Let the sample set be... ,in Indicates the first For each sample, the objective function of FCM is defined as:

[0106]

[0107] in, The membership matrix, Indicates the first The nth sample pair The membership degree of each cluster center, satisfying: . For the sample With cluster center The distance is usually expressed in Euclidean form, i.e.: m is the fuzzy weighting coefficient, used to adjust the degree of fuzziness in clustering, and is generally taken as... .

[0108] During initialization, the number of clusters needs to be predetermined during the FCM solution process. Maximum number of iterations Convergence threshold and the initial membership matrix In the first In the next iteration, the cluster centers are first updated based on the current membership matrix. Cluster center The update formula is:

[0109]

[0110] Subsequently, the membership degree of each sample to each category is updated based on the new cluster centers. The update formula is as follows:

[0111]

[0112]

[0113] Iteration stops when the maximum number of iterations is reached or the termination condition is met early. Common termination conditions can be expressed as:

[0114]

[0115] (3) Determining the number of clusters

[0116] Due to the initial cluster centers and the number of clusters The settings will have a significant impact on FCM results, while empirical selection The results are highly subjective. To enhance the objectivity and scientific rigor of the clustering results, the silhouette coefficient is introduced. The number of clusters is determined. The silhouette coefficient evaluates cluster quality by simultaneously measuring the tightness of samples within clusters and the separation between clusters; its value ranges from [value missing]. The closer the value is to 1, the better the clustering effect. For any sample Its profile coefficient is defined as:

[0117]

[0118] in For the sample The cohesion degree represents the average distance between a sample and other samples within the same cluster; For the sample The separation degree represents the average distance between a sample and its nearest neighbor samples. Let... The cluster to which it belongs is The sample size is ,but:

[0119]

[0120] For any non-cluster ,definition:

[0121]

[0122]

[0123] The average of the silhouette coefficients for all samples is:

[0124]

[0125] Calculate the optimal number of clusters within the candidate cluster number set, and select the cluster number with the largest average silhouette coefficient as the optimal number of clusters. When taking the maximum, the corresponding This represents the optimal number of clusters.

[0126] (4) Determination of indicator weights

[0127] In the classification of traffic flow operation states on highways, indicators such as speed and flow rate have varying abilities to represent traffic states. To reflect the different contributions of different indicators to state information, this paper uses the entropy weight method to objectively assign weights to the importance of indicators, thereby obtaining the weight coefficients of each indicator. Let there be a total of Individual indicators There are samples, and the original data matrix is... ,in For the first The sample at the th The values ​​for each indicator are taken. First, the data is dimensionless, resulting in... Methods such as range standardization were employed, and the first step was calculated. The first indicator The proportion of each sample:

[0128]

[0129] Thus, the first The entropy value of each indicator:

[0130]

[0131] And define the difference coefficient:

[0132]

[0133] The final indicator weights are:

[0134]

[0135] in, Indicates the first Information entropy of each indicator Indicates the first The first indicator The proportion of each sample These are the weighting coefficients for the corresponding indicators. Using the entropy weighting results to construct weighted features or weighted distances can more reasonably reflect the differences in the contributions of indicators such as speed and flow to the classification of traffic flow operation states during the clustering process.

[0136] Step 1.2: Consistency Verification of Traffic State Classification and Mechanism. Traffic flow states are classified into six levels based on clustering output. Combining the macroscopic traffic flow basic diagram and three-phase traffic flow theory, the meaning of each category is interpreted accordingly, representing them as: Very Smooth, Smooth, Slightly Smooth, Slightly Congested, Congested, and Severely Congested.

[0137] Step 1.3: Threshold Extraction. Based on the distribution characteristics of each cluster category, extract the corresponding flow-density threshold range; further utilize the inherent constraint relationship among flow, density, and speed to combine the speed threshold and flow threshold to characterize the congestion level and state changes of the road segment.

[0138] Step 2: Selection of Gantry Range and Construction of ETC Chain Based on "Time Accessibility". Centered on the target road segment, based on historical ETC passage records, the typical travel time and propagation delay of vehicles in the road network are statistically analyzed to construct a "time accessibility" criterion reflecting the upstream and downstream influence range. Under the conditions of preset prediction lead and time window length, upstream gantries / toll stations that have a significant impact on the traffic state of the target location within the time accessibility range are selected as candidate detection points. Furthermore, the continuous vehicle passage records between candidate detection points are associated according to the actual vehicle passage order to construct an ETC passage chain (ETC chain) between adjacent gantries. The travel time, travel speed and their statistical characteristics are calculated on the ETC chain to form a spatiotemporal correlation input representation for the end-to-end learning model.

[0139] Step 2.1: Define the target object and basic data. Let the target gantry corresponding to the target road segment be... To retrieve historical ETC gantry vehicle passage records, each record must include at least: vehicle identification number. Inspection point signage Passing time Vehicle model. The original records should be cleaned, including at least: duplicate record removal, timestamp correction, and removal of obvious anomalies.

[0140] Step 2.2: Typical running time and propagation delay statistics. (Based on the target gantry) Centered on the upstream gantry that has a topological connection with it. To conduct statistics.

[0141] For any candidate upstream gantry Identify vehicles from historical data Drive to A valid sample, meaning the same vehicle exists. In terms of time sequence, it satisfies the condition of passing first. After passing Calculate the travel time for each sample:

[0142] For each Typical runtime is obtained:

[0143]

[0144] in, Desirable To adapt to the propagation and fluctuations of congestion, a propagation time delay margin term can be further introduced to reflect the propagation characteristics of traffic disturbances on the target cross section. To cover arrival time spread caused by queuing waves and vehicle speed fluctuations, it is set as follows:

[0145]

[0146] in, The coefficient is an empirical coefficient, and IQR is the interquartile range; it can also be directly replaced by the historical quantile difference.

[0147] Step 2.3: Time Accessibility Criteria and Gantry Range Screening. Let the forecast lead time be... For example, in advance Predict the target state within minutes, with an input time window length of [time window length]. (For example (minutes), input sequence length is (common (One time window). The length of the historical time domain covered by the input is:

[0148]

[0149] Define gantry For the target gantry Under the current forecast settings, the criterion for "time-awareness" is: its typical runtime falls within the observable time domain of the model, i.e.

[0150]

[0151] When it is necessary to further control the number of gantry frames and avoid introducing weakly correlated noise from excessively distant gantry frames, a lower bound can be set simultaneously to exclude points that are "too close, causing information redundancy," thus forming a two-sided constraint:

[0152]

[0153] in, Desirable or .

[0154] Based on this, the set of upstream reachable gantry frames is obtained:

[0155]

[0156] Step 2.4: ETC chain construction based on candidate gantry set. Within the scope, according to vehicle identification The vehicle passage records are correlated and sorted by timestamp to form a vehicle passage sequence:

[0157]

[0158] For two consecutive vehicle passage records, if the following conditions are met:

[0159] ;

[0160] The time sequence is consistent and the time difference is within a reasonable range. This is used to exclude situations such as leaving the highway midway or making abnormal stops;

[0161] Two gantries can be directly adjacent in the road network topology or belong to a predefined set of adjacency relationships. ;

[0162] This is then treated as a valid link segment, forming an ETC link edge:

[0163]

[0164] Calculate the chain segment travel time and chain segment speed for each valid chain segment:

[0165]

[0166] in, This refers to the distance between gantry frames.

[0167] Step 2.5: Time window aggregation and end-to-end input representation generation. For each time window, the length is [length]. and each chain segment Summarize statistical features to form the spatiotemporal tensor of the model input. Then, use the target gantry. The statistical features of the corresponding chain segments of the gantry, along with their time reachability, are stacked in a "gantry dimension - feature dimension - time dimension" manner to obtain the input sequence representation of the end-to-end learning model:

[0168]

[0169] Each of them All are composed of chain segment features spliced ​​together within the selected gantry range.

[0170] Step 3: Establish an end-to-end time-series prediction model based on ETC data. An end-to-end learning network is constructed to map the input sequence to future congestion states. The model includes a vehicle feature encoding module, an attention aggregation module to characterize differences in vehicle contributions, a time-series modeling module to characterize cross-time window evolution patterns, and a classifier. Through this structure, the model automatically learns multi-level spatiotemporal correlation features of "vehicle-time window-road segment / interval," outputting the traffic operation status at the target prediction time, thus predicting short-term traffic conditions. In this embodiment, the end-to-end learning highway congestion prediction model structure can be referenced. Figure 3 .

[0171] Step 3.1: Input, output and training sample organization.

[0172] Step 1: Time Window and Forecast Settings. Set the time window length to... (In this embodiment, the timeframe can be 5 minutes). ETC gantry / toll station vehicle passage records are merged into a discrete time window index (corresponding to frequency_count5 in the code) based on timestamps. Assume the input sequence contains... The prediction lead is calculated using 4 consecutive time windows (window_size=4 in the code). The first time window (predict_ahead=1 in the code, meaning predicting the traffic status of the next window). Using the first... If a time window is used as the current moment, then the model input uses ETC information, predicting tag acquisition Traffic operation status level corresponding to the time window.

[0173] Step 2: Vehicle-level Feature Construction and Fixed Dimensionality. Extract vehicle-level feature vectors for each vehicle passage record within each time window. This embodiment uses 4-dimensional features: vehicle type / model code: vehicletype_x, vehicle count or statistically related count field: count, time difference between adjacent records: time_difference, and time feature: time. These 4 dimensions are combined to form the vehicle feature vector:

[0174]

[0175] in For indexing time windows, This refers to the vehicle sequence number within that time window.

[0176] To keep the network input dimension fixed, each time window is set to retain a maximum of [number missing]. Vehicles (max_vehicles=500 in the code): The number of vehicles within a time window. At that time, vehicle records are sampled using a fixed random seed; when When, it is filled with a zero vector to form The vehicle feature matrix.

[0177] Therefore, the input tensor for a single sample is:

[0178]

[0179] Input in batches during training:

[0180]

[0181] in This is the batch size (batch_size=16 in the code).

[0182] Step 3: Feature Standardization. To eliminate the influence of dimensions and improve convergence speed, this embodiment performs standardization processing on vehicle features. Specifically: only all vehicle feature samples within the training set window are used to fit the standardizer (using Standard Scaler in the code); the same standardizer parameters are used to transform the input features for both the training and test sets; and the zero vectors are kept zero or consistent with the standardization result.

[0183] Step 4: Label Mapping. The "Running State Index" from the state data table is used as the supervision label. To ensure training / testing consistency, a unified mapping is established across the entire dataset: This is achieved by taking all occurrences of the TPI level and sorting them. Define mapping Get the category index Model output The category score vector is used to predict the category. The index corresponding to the maximum value is then used to look up the TPI level.

[0184] Step 3.2: End-to-End Model Structure (Enhanced Traffic Classifier). In this embodiment, the end-to-end learning highway congestion prediction model structure can be referenced. Figure 3 Through the vehicle feature encoding module, the model can learn high-dimensional representations related to traffic conditions from the micro-features of ETC vehicles; through the attention aggregation module, the model can automatically distinguish the differences in the contributions of different vehicles to state judgment, and realize adaptive weighted fusion at the vehicle level; through the temporal modeling module, the model can learn the dynamic evolution law across time windows, and realize the characterization of the congestion formation and propagation process; finally, it outputs the traffic operation state category (TPI level) at the target prediction time, thereby realizing end-to-end prediction of the short-term traffic conditions in the future.

[0185] Let the input be The model consists of the following modules.

[0186] Step 1: Vehicle Feature Encoding Module. Objective: To map the original 4D vehicle features to a high-dimensional semantic space, extracting a more discriminative vehicle embedding representation. Implementation: For each vehicle feature vector... A 128-dimensional vector is obtained through encoding using a multi-layer fully connected network. This can be represented as:

[0187]

[0188] in It consists of a linear layer, ReLU, BatchNorm, and Dropout (in the code: 4→64→128, with BN and Dropout).

[0189] After inputting all vehicle codes, we get:

[0190]

[0191] This module enables nonlinear representation learning at the "vehicle layer," allowing the model to uncover the implicit relationship between vehicle micro-features and congestion status without relying on manually set thresholds / rules.

[0192] Step 2: Attention Aggregation Module (Characterizing Vehicle Contribution Differences). Purpose: Different vehicles contribute differently to the road segment's operational status within the same time window. An attention mechanism is introduced to adaptively weight vehicle embeddings, highlighting key vehicle / key feature combinations. (Time window...) Inner vehicle embedding vector Calculate and normalize the attention score:

[0193] Attention score:

[0194]

[0195] Softmax normalization yields the weights:

[0196]

[0197] The weighted summation yields the aggregate representation of this time window:

[0198]

[0199] Will The aggregated representation of the time windows is stacked in chronological order to obtain the time window sequence features:

[0200]

[0201] Attention weight This is used to reflect the contribution of vehicles to the judgment of traffic conditions in the current time window, and to achieve adaptive filtering and fusion of information at the "vehicle-time window" level.

[0202] Step 3: Time-series modeling module, learning the evolution patterns across time windows. Objective: Traffic conditions exhibit significant time dependence (formation, diffusion, and dissipation processes), requiring modeling of multi-time-window sequences to capture congestion evolution patterns. Implementation: Using a bidirectional LSTM... Encode the hidden state (hidden_size=256, num_layers=3, bidirectional=True in the code) and output the hidden state at each time step:

[0203]

[0204] The bidirectional output concatenation results in a dimension of .

[0205] To predict the state at future times, this embodiment uses the output of the last time step as the sequence synthesis representation:

[0206]

[0207] Taking the output of the last time step is equivalent to using the historical information of the entire input window to encode the current "traffic situation" and then predict it. The state at any given moment.

[0208] Step 4: Time-series classification output module, mapping to future states. Purpose: To map the comprehensive representation of the sequence to the traffic operation state category at the predicted future time.

[0209] The classifier consists of a multi-layer fully connected network (in the code: 512→256→128→C, including Batch Normalization and Dropout), and outputs logits:

[0210]

[0211] in This represents the number of TPI rank categories. The predicted categories are:

[0212]

[0213] Step 4: Model Training and Imbalanced Sample Adaptive Optimization. The end-to-end model is trained under supervision using historical samples. A cross-entropy loss function with class weights is employed to mitigate training bias caused by imbalanced traffic state level samples. During training, weight decay, gradient pruning, and adaptive learning rate adjustment based on validation set performance are combined to improve training stability. On the validation set, early stopping and optimal model parameter saving / rollback are performed based on validation accuracy to obtain model parameters with stronger generalization ability, thereby improving prediction stability under different road segments, different date types, and different traffic demand levels.

[0214] Step 4.1: Class distribution statistics in training data. Before model training begins, iterate through the training data loader (train_loader) to obtain the set of class labels for all training samples. ,in , The number of traffic state categories (determined by the output dimension of the last layer of the model).

[0215] Statistics for each category Number of samples:

[0216]

[0217] It also outputs the training set class distribution, which is used to determine the degree of imbalance and subsequent weight construction.

[0218] Step 4.2: Construct class weights to resolve imbalances and protect against missing classes.

[0219] Based on the above category counts, construct the category weight vector. This is used to enhance the contribution of minority classes to the training objective.

[0220] Step 1: Identify the categories appearing in the training set ( Category weights are calculated as follows (consistent with the code):

[0221]

[0222] in The total number of training samples, For the number of categories, For the first Number of samples in each class.

[0223] Step 2: Identify missing categories in the training set ( To avoid division by zero and maintain stability during training, a protective assignment method is used: the maximum value among the existing class weights is taken. And assign higher weights to missing categories:

[0224]

[0225] Step 4.3: Weighted Cross-Entropy Loss Function and Supervised Training Objective. The model is trained under supervised conditions using a cross-entropy loss function with class weights. Assume the model... The output logits are The real category is Then the weighted cross-entropy is:

[0226]

[0227] in For batch size, This represents the weight corresponding to the true category. By introducing... This allows less congestion level samples to generate a larger gradient contribution during backpropagation, thereby reducing the model's prediction bias for the majority class (such as the unobstructed class).

[0228] Step 4.4: Parameter optimization, training stability and regularization settings.

[0229] Optimizer: The Adam optimizer is used to iteratively update the model parameters, and a weight decay term is added to suppress overfitting (optim.Adam(..., weight_decay=1e-4) in the code). Gradient clipping: The gradient norm is clipped after each backpropagation, limiting the maximum norm to a preset threshold (max_norm=1.0 in the code) to avoid the gradient explosion problem in multi-layer temporal networks.

[0230]

[0231] in, This is the set of model parameters.

[0232] Training process monitoring: Accumulate training loss and training accuracy in each training round, and output a progress bar to display batch-level loss and accuracy in real time to monitor convergence (code uses tqdm).

[0233] Step 4.5: Validation and Evaluation, Adaptive Learning Rate Adjustment, and Optimal Model Selection. After each training round, switch the model to evaluation mode and calculate the validation loss and validation accuracy on the validation data loader test_loader. and with This serves as the basis for model selection and learning rate scheduling.

[0234] Step 1: Learning rate scheduling: An adaptive learning rate strategy based on the plateau period is adopted. When the validation accuracy does not improve significantly within several rounds, the learning rate is automatically reduced (ReduceLROnPlateau in the code, the monitoring metric is the validation accuracy, factor=0.5, patience=5) to promote more stable convergence in the later stages.

[0235] Step 2: Optimal Model Saving: If the accuracy of the current round of validation is... If the value is higher than the historical best value, update the best value and save the current model parameters (written in best_model.pth in the code), and record the best weight best_model_wts as the rollback baseline.

[0236] Step 4.6: Early Stop and Rollback to Improve Generalization Ability. To avoid overfitting and obtain model parameters with stronger generalization ability, an early stopping mechanism is set: if the validation accuracy is continuous... If the patience parameter in the code does not improve, training stops, and the model parameters are rolled back to the parameters best_model_wts at the point of best historical validation accuracy, which is then used as the final output model. This strategy can reduce performance fluctuations caused by "overfitting the training set" and improve prediction stability under different sample distributions of different date types and different demand levels.

[0237] Step 5: Online Inference and Congestion Risk Output. During the real-time operation phase, the latest ETC passage data for the target location within the latest time window is continuously acquired. A real-time input sequence is generated according to the same time window division, feature encoding, and standardization process as in the training phase, and then input into the end-to-end model. The model outputs the traffic operation status of the target location within a preset prediction step size.

[0238] Step 5.1: Online data acquisition and rolling time window updates.

[0239] Step 1: Real-time data stream access. During the online operation phase, continuously access the ETC passage data stream corresponding to the target location (road segment / section / toll station or gantry group). The data includes at least a vehicle type field, a timestamp field, and a field for calculating the arrival time difference, consistent with the fields used during training.

[0240] Step 2: Consistent Time Window Division. Use the same time window length as the training phase. The real-time data is merged to generate the current time window ID (corresponding to the frequency_count5 semantics in the training data). When the new time window ends, all vehicle records within that time window are fixed as the "latest time window data".

[0241] Step 3: Form a rolling input sequence. Maintain a length of [length missing]. Time window queue (window_size=4 in the code): When a new time window arrives, the latest window data is added to the right end of the queue, while the oldest time window data is removed, always maintaining continuity. Each time window serves as the current inference input sequence.

[0242] Step 5.2: End-to-end model online inference and prediction step size output.

[0243] Step 1: Model Loading and Inference Mode. The online system loads the optimal model parameters (best_model.pth) obtained from offline training, sets the model to evaluation mode (eval()), and disables gradient calculation to reduce computational overhead and ensure inference consistency.

[0244] Step 2: State level prediction. Convert the input tensor... Inputting the data into an end-to-end network yields category score vectors (logits): The category corresponding to the highest score is then selected as the predicted category. The index_to_tpi mapping established during the training phase will... Restored to a specific TPI level, as the "preset prediction step size" The future traffic operation status (predict_ahead=1 in the code).

[0245] Step 3: Multi-step prediction. When multiple prediction steps are needed (e.g., future predictions), this step is necessary. When training multiple different time windows, one of the following methods can be used: The model can be used to infer separately; or, while keeping the model structure unchanged, the output can be extended to multi-step, multi-class and jointly trained.

[0246] Step 5.3: Output and alarm of congestion risk.

[0247] Step 1: Output Content. Online inference output should include at least: the prediction time window ID (i.e., the ID of the prediction time window after the current window). Individual windows); predicting TPI level / traffic status category;

[0248] Step 2: Risk Level and Threshold Rules. A congestion risk index can be constructed based on the predicted TPI level or the "sum of congestion probabilities" to trigger an alarm. When the predicted TPI reaches a preset congestion threshold level (e.g., TPI ≥ a certain level), "Congestion Risk" is output; or when... Exceeding the threshold This will trigger an alert, and the threshold can be set based on historical statistics or business needs.

[0249] Step 3: Continuous Output and Stability. The system continuously executes the above process according to a rolling time window cycle, forming a stable short-term traffic condition prediction sequence for real-time congestion risk monitoring and decision support under different road segments, date types, and demand levels.

[0250] It is hereby declared that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A highway congestion prediction method based on end-to-end learning of ETC data, characterized in that, Includes the following steps: S1. Construct an end-to-end learning sample and label system for ETC; S2. Selection of gantry range and construction of ETC chain based on "time accessibility"; S3. Establish an end-to-end time-series prediction model based on ETC data; S4. Model training and adaptive optimization of imbalanced samples; S5. Online reasoning and congestion risk output.

2. The highway congestion prediction method based on end-to-end learning of ETC data according to claim 1, characterized in that, Step S1 includes the following sub-steps: S1.1 Data Acquisition and State Clustering Modeling; Historical traffic data from highway toll stations and ETC gantries is collected, and the historical traffic data is cleaned, deduplicated, and anomaly removed. A statistical period is set, traffic status indicators for the corresponding road segments are calculated, and the FCM clustering algorithm with entropy weighting is used to perform cluster analysis on traffic flow operation status. The historical traffic data includes vehicle identification information, vehicle passage time information, vehicle type, and travel data that can be used to estimate travel time. I. Traffic condition indicators: including flow rate, speed, and density; II. Fuzzy C-means clustering algorithm: By iteratively and alternately optimizing the cluster center and membership matrix, the objective function is continuously reduced until the objective function converges or meets the preset termination condition. III. Number of clusters: The silhouette coefficient is introduced to determine the number of clusters; For any sample Its profile coefficient is defined as: In the formula, Represents the silhouette coefficient of the i-th sample; For the sample The cohesion degree represents the average distance between a sample and other samples within the same cluster; For the sample The separation degree represents the average distance between it and its nearest neighbor samples from other clusters; set up The cluster to which it belongs is The sample size is ,but: For any non-cluster ,definition: The average of the contour coefficients of all samples for: Calculate the optimal number of clusters within the candidate cluster number set, and select the cluster number with the largest average silhouette coefficient as the optimal number of clusters. When taking the maximum, the corresponding The optimal number of clusters; IV. Determination of indicator weights: The entropy weight method is used to objectively assign weights to the importance of indicators, thereby obtaining the weight coefficients of each indicator; S1.2 Consistency verification of traffic state classification and mechanism; Traffic flow status is classified into six levels based on clustering output. Combined with the macroscopic traffic flow basic map and three-phase traffic flow theory, the traffic meaning of each category is interpreted accordingly, and is represented as follows: very smooth, smooth, relatively smooth, relatively congested, congested, and severely congested. S1.3 Threshold Extraction; Based on the distribution characteristics of each cluster category, the corresponding flow-density threshold range is extracted; further, utilizing the inherent constraint relationship among flow, density, and speed, a road segment traffic flow density index is defined. .

3. The highway congestion prediction method based on end-to-end learning of ETC data according to claim 2, characterized in that, The specific content of step S2 is as follows: S2.1 Target object and basic data definition; Let the target gantry corresponding to the target road segment be... To retrieve historical ETC gantry vehicle passage records, each record must include at least: vehicle identification number. Inspection point signage Passing time Vehicle model; S2.2 Typical running time and propagation delay statistics; Target gantry Centered on the upstream gantry that has a topological connection with it. To conduct statistics; For any candidate upstream gantry Identify vehicles from historical data Drive to A valid sample, meaning the same vehicle exists. In terms of time sequence, it satisfies the condition of passing first. After passing ; For each Typical runtime is obtained: In the formula, Indicates that the vehicle is from Drive to A sample set of travel time; Representative to The typical running time obtained after taking the q-quantile represents the time taken under normal or slightly congested conditions from... arrive The representative travel time; q is the quantile value; Introducing a propagation delay margin term This is used to cover arrival time extensions caused by queuing waves and vehicle speed fluctuations. In the formula, Indicates from arrive The propagation delay margin; Empirical coefficients; IQR is the interquartile range; S2.3 Time Accessibility Criteria and Gantry Range Screening; gantry For the target gantry Under the current forecast settings, the criterion for "time-reachable" is: its typical runtime falls within the observable time domain of the model, i.e. In the formula, H represents the forecast lead time, indicating how far in advance the forecast of the target gantry is made. Predict the state; This indicates the length of the input time window, i.e., the duration of a single input time window; L represents the length of the input sequence. Input the length of the historical time domain covered; A lower bound is set to exclude points that are "too close to cause information redundancy," forming a two-sided constraint: In the formula, Value or That is, the length of one or two time windows; Based on this, the set of upstream reachable gantry frames is obtained: S2.4 ETC chain construction based on candidate gantry set; exist Within the scope, according to vehicle identification The vehicle passage records are correlated and sorted by timestamp to form a vehicle passage sequence: For two consecutive vehicle passage records, if the following conditions are met: The time sequence is consistent and the time difference is within a reasonable range. Two gantries can be directly adjacent in the road network topology or belong to a predefined set of adjacency relationships. ; This is then treated as a valid link segment, forming an ETC link edge: In the formula, e represents a valid ETC link segment; Indicates that the vehicle has passed through the gantry. timestamp; Indicates that the vehicle has passed through the gantry. timestamp; Calculate the segment travel time and segment speed for each valid chain segment: in, Indicates vehicle ID from Drive to The actual time spent; For gantry and The physical distance between them; S2.5 Time Window Aggregation and End-to-End Input Representation Generation; I. With For each time window, the length is [length]. and each chain segment Summarize the statistical features to form the spatiotemporal tensor for model input; II. Target gantry The statistical features of the chain segments corresponding to the gantry that can be reached in time are stacked in the order of "gantry dimension - feature dimension - time dimension" to obtain the input sequence representation of the end-to-end learning model: Each of them In the time window Within this time window, a tensor composed of the statistical characteristics of all chain segments within the selected gantry range represents a snapshot of the traffic state.

4. The highway congestion prediction method based on end-to-end learning of ETC data according to claim 3, characterized in that, The end-to-end time series prediction model constructed in step S3 include: Vehicle feature encoding module: used to learn high-dimensional representations related to traffic conditions from the micro-features of ETC vehicles; Attention aggregation module: used to distinguish the differences in contributions of different vehicles to state discrimination, and to achieve adaptive weighted fusion at the vehicle layer; Temporal modeling module: used to learn the dynamic evolution patterns across time windows, enabling the characterization of the congestion formation and propagation process; Time-series classification output module: Used to output the traffic operation status category at the target prediction time, thereby realizing end-to-end prediction of the short-term traffic status in the future.

5. The highway congestion prediction method based on end-to-end learning of ETC data according to claim 4, characterized in that, The specific content of step S4 is as follows: The end-to-end model is trained under supervision using historical samples. A cross-entropy loss function with class weights is used to alleviate the training bias caused by the imbalance of traffic state level samples. During the training process, weight decay, gradient clipping, and adaptive adjustment of the learning rate based on the performance of the validation set are combined to improve training stability. Early stopping and optimal model parameter saving / rollback are performed on the validation set based on the validation accuracy criterion to obtain model parameters with stronger generalization ability, thereby improving the prediction stability under different road segments, different date types and different traffic demand levels.

6. The highway congestion prediction method based on end-to-end learning of ETC data according to claim 5, characterized in that, The specific content of step S5 is as follows: During the real-time operation phase, the latest ETC passage data of the target location in the latest time window is continuously acquired. The real-time input sequence is generated according to the same time window division, feature encoding and standardization process as in the training phase, and then input into the end-to-end model to output the traffic operation status of the target location under the preset prediction step size.

Images