A traffic congestion prediction method and device based on contrast learning and knowledge distillation

Abstract

A traffic congestion prediction method and device based on contrast learning and knowledge distillation belong to the technical field of traffic congestion prediction. The method comprises the following steps: step S1, preprocessing the original traffic data set to obtain traffic feature time series data; step S2, constructing a traffic congestion prediction model; step S3, constructing a multi-task loss function; step S4, training the traffic congestion prediction model using the traffic feature time series data, and optimizing the model parameters by minimizing the multi-task loss function; and step S5, inputting a to-be-predicted traffic data set into the trained traffic congestion prediction model to output the predicted congestion level of each node. The contrast learning module is used to maximize the hidden space difference between samples to enhance the robustness of the features, alleviate the problem of single generation mode of the traditional variational autoencoder, and improve the prediction accuracy in complex scenes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of traffic congestion prediction, and particularly relates to a traffic congestion prediction method and device based on contrast learning and knowledge distillation. BACKGROUND

[0002] With the development of economy and the acceleration of urbanization process, various traffic modes are rapidly popularized, the number of motor vehicles is increasing rapidly, and the pressure on urban traffic system is increasingly serious. Traffic congestion has become a core challenge for sustainable development of cities.

[0003] Traffic congestion prediction is one of the core tasks of intelligent transportation systems. The existing method based on spatio-temporal graph neural network (STGNN) (ST-FiT) realizes prediction by modeling spatial adjacency relationship and time dependence, but the diversity of generated model is insufficient, and the generalization ability of the model is also limited.

[0004] Specifically, in the time data enhancement module of the model, the traditional variational autoencoder (VAE) is prone to "mode collapse" when generating complex spatio-temporal feature data due to its inherent training target and reasoning mechanism. That is, the model will only generate a few most common or simplest traffic patterns, ignoring the various occasional, sudden or complex congestion evolution patterns existing in the real road network, resulting in insufficient diversity of generated data and inability to effectively enhance the generalization ability of the model.

[0005] The backbone network (i.e. the core STGNN) of the model only relies on the final prediction result as a supervision signal. Such "end-to-end" single supervision signal is essentially a "weak supervision", which cannot provide fine-grained guidance for feature learning of the intermediate layers of the model, resulting in limited ability of the model to capture complex spatio-temporal dependence, especially difficult to distinguish congestion patterns with similar appearance but different causes (such as accident congestion and normal peak congestion). SUMMARY

[0006] In order to solve the technical problem that the existing method based on spatio-temporal graph neural network (STGNN) for traffic congestion prediction realizes prediction by modeling spatial adjacency relationship and time dependence, but the diversity of generated model is insufficient, and the generalization ability of the model is also limited, the application proposes a traffic congestion prediction method and device based on contrast learning and knowledge distillation.

[0007] The first aspect of the application provides a traffic congestion prediction method based on contrast learning and knowledge distillation, comprising the following steps:

[0008] S1: preprocessing the original traffic data set to obtain traffic feature time series data, wherein the traffic feature time series data contains traffic indicators and congestion levels divided based on traffic density;

[0009] S2: constructing a traffic congestion prediction model, which comprises the following steps in sequence:

[0010] a time feature generator configured to perform time feature enhancement on the traffic feature time series data, comprising a variational autoencoder and a contrastive learning module, wherein the contrastive learning module is configured to generate positive samples by time series perturbation, and utilize features of different nodes or time periods as negative samples to maximize the difference in the latent space;

[0011] a spatial topology learning module configured to perform spatial feature enhancement on the time feature enhanced data to obtain spatio-temporal feature enhanced data;

[0012] a backbone spatio-temporal graph neural network configured to output congestion level prediction values based on the spatio-temporal feature enhanced data, wherein the backbone spatio-temporal graph neural network adopts a self-distillation mechanism, comprising a teacher model and a student model, and utilizes an exponential moving average to update parameters of the teacher model;

[0013] S3: constructing a multi-task loss function;

[0014] S4: training the traffic congestion prediction model using the traffic feature time series data, and optimizing model parameters by minimizing the multi-task loss function;

[0015] S5: inputting a to-be-predicted traffic data set into the trained traffic congestion prediction model to output predicted congestion levels of each node at each sampling time point.

[0016] Further, the multi-task loss function has an expression as follows:

[0017]

[0018]

[0019] wherein, , , is a weight coefficient, and is an original supervised loss function, is a distillation loss function, is a combination of a VAE loss function and a contrastive learning loss function, is a VAE loss function, is a contrastive learning loss function.

[0020] Further, the preprocessing of step S1 comprises:

[0021] extracting traffic flow and average speed as traffic indicators from the original traffic data set; if the original traffic data set includes average occupancy, the average occupancy is also extracted;

[0022] replace zero values in the extracted traffic indicators with the average of non-zero values;

[0023] calculate traffic density;

[0024] normalize the traffic density to the range [0, 1];

[0025] based on the normalized traffic density, divide congestion levels using linear mapping method;

[0026] combine the extracted traffic indicators with the congestion levels to form the traffic feature time series data.

[0027] Further, the calculating traffic density comprises:

[0028] if the original traffic data does not include average occupancy, calculate the traffic density using the following formula : ;

[0029] wherein, is traffic flow, is average speed;

[0030] if the original traffic data includes average occupancy, calculate the traffic density using the following formula :

[0031]

[0032] wherein, , is normalized average occupancy, is a preset weight coefficient.

[0033] Further, the normalizing the traffic density to the range [0, 1] comprises:

[0034] calculate the minimum value and the maximum value of the traffic density array ;

[0035] if is equal to , return an array with all elements being 0, which has the same shape as ;

[0036] otherwise, linearly normalize using the following formula:

[0037] ;

[0038] wherein, denotes the normalized result, is the array a certain sample value in the set.

[0039] Further, the congestion level is divided by a continuous congestion index converted from the normalized traffic density by a linear mapping method, and the congestion index divides the traffic congestion condition into four congestion levels of free, slight congestion, moderate congestion and severe congestion.

[0040] Further, the conversion of the normalized traffic density into the continuous congestion index by the linear mapping method is specifically:

[0041] The calculation of the normalized traffic density is , , The third quantile;

[0042] The traffic density value is mapped into the congestion index according to the following rules:

[0043] If , the congestion index ;

[0044] If , the congestion index ;

[0045] If , the congestion index ;

[0046] If , the congestion index .

[0047] Another aspect of the present application provides a traffic congestion prediction device based on contrast learning and knowledge distillation, comprising the following modules:

[0048] A preprocessing module configured to preprocess an original traffic data set to obtain traffic feature time series data, wherein the traffic feature time series data includes traffic indicators and congestion levels divided based on traffic density;

[0049] A model construction module configured to construct a traffic congestion prediction model, which comprises in sequence:

[0050] A time feature generator for time feature enhancement of the traffic feature time series data, which includes a variational autoencoder and a contrast learning module, wherein the contrast learning module is configured to generate positive samples by time series perturbation and use features of different nodes or time periods as negative samples to maximize the difference in hidden space;

[0051] A spatial topology learning module for spatial feature enhancement of the time feature enhanced data to obtain time-space feature enhanced data;

[0052] The backbone spatio-temporal graph neural network is used for outputting congestion level prediction values based on spatio-temporal feature enhanced data, wherein the backbone spatio-temporal graph neural network adopts a self-distillation mechanism, includes a teacher model and a student model, and updates parameters of the teacher model by using an exponential moving average;

[0053] and is configured to construct a multi-task loss function and add a dynamic weight adjustment strategy in the multi-task loss function.

[0054] The model training module is configured to train the traffic congestion prediction model by using the traffic feature time series data, and optimize model parameters by minimizing the multi-task loss function.

[0055] The prediction module is configured to input a to-be-predicted traffic data set into the trained traffic congestion prediction model, and output predicted congestion levels of each node at each sampling time point.

[0056] The method has the following beneficial effects:

[0057] 1. Feature diversity and robustness are synergistically improved.

[0058] The application introduces a contrast learning module into the variational autoencoder (VAE) to construct an internal mechanism of "generation-discrimination" cooperation. The variational autoencoder (VAE) is responsible for learning the basic distribution of data to ensure the generation quality, and the contrast learning module maximizes the difference between positive and negative samples in the hidden space, explicitly encourages the model to mine and retain non-typical time series patterns that are easily ignored by the variational autoencoder (VAE). This internal cooperative mechanism effectively avoids the mode collapse problem of the variational autoencoder (VAE) in principle, and the generated enhanced data is not only diverse, but also has better feature discrimination.

[0059] 2. The model generalization ability and knowledge extraction ability are enhanced.

[0060] The self-distillation mechanism introduced in the spatio-temporal graph neural network (STGNN) is essentially a dynamic evolving "teacher-student mutual promotion" learning system. The teacher model smoothes the parameters of the student model by exponential moving average (EMA), providing a more stable and informative "soft label" supervision signal. The soft label contains relative relationship information between categories (congestion levels), and compared with single supervision using only "hard label", it can guide the student model to learn the decision boundary between categories and the internal law of data, thereby improving the generalization ability of the model in complex traffic scenarios.

[0061] The traffic congestion prediction method and device based on contrast learning and knowledge distillation provided by the application are suitable for traffic congestion prediction. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 A flowchart of a traffic congestion prediction method based on contrast learning and knowledge distillation in an embodiment of the present application is shown in FIG. 1.

[0063] Figure 2 A structural diagram of a traffic congestion prediction model in an embodiment of the present application is shown in FIG. 2.

[0064] Figure 3 A structural diagram of a time feature generator (including a variational autoencoder and a contrast learning module) in an embodiment of the present application is shown in FIG. 3.

[0065] Figure 4 A structural diagram of a backbone spatio-temporal graph neural network (introducing a self-distillation mechanism) in an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0066] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0067] Embodiment 1:

[0068] As shown in FIG. 5, the present embodiment provides a technical solution: Figure 1 and Figure 2 A traffic congestion prediction method based on contrast learning and knowledge distillation, comprising the following steps:

[0069] Step S1: Preprocessing the original traffic data set to obtain traffic feature time series data, wherein the traffic feature time series data includes traffic indicators and congestion levels based on traffic density division.

[0070] The original traffic data is the traffic indicators collected by each node (basic observation point or sensor location in the traffic network) at each sampling time point, which at least includes two traffic indicators: traffic flow and average speed. It can also include an additional traffic indicator: average occupancy rate.

[0071] In the spatio-temporal graph neural network (STGNN) model, the entire urban road network is abstracted as a graph, which is composed of nodes and edges. The nodes in the graph represent observation points or sensors, and the edges represent the connection relationship between nodes, such as whether two sensors are on the same road, adjacent, or associated according to road connectivity.

[0072] In the spatio-temporal graph neural network (STGNN) model, the entire urban road network is abstracted as a graph, which is composed of nodes and edges. The nodes in the graph represent observation points or sensors, and the edges represent the connection relationship between nodes, such as whether two sensors are on the same road, adjacent, or associated according to road connectivity.

[0073] Traffic flow refers to the number of vehicles passing through a node per unit of time (e.g., per hour). Average speed refers to the average speed at which vehicles pass through a road. Average occupancy refers to the proportion of time that a road space is occupied. Among them, the average occupancy provides a stable and direct measurement of road space utilization and congestion, and using it to correct the traffic density can significantly improve the accuracy of the model in perceiving the real traffic state, thereby making more reliable predictions.

[0074] Step S2: constructing a traffic congestion prediction model, which includes the following steps in sequence:

[0075] A time feature generator is configured to perform time feature enhancement on the traffic feature time series data formed in step S1. The time feature generator includes a variational autoencoder (VAE) and a contrastive learning module. The contrastive learning module is configured to maximize the difference in the latent space by generating positive samples through time series perturbation and using features of different nodes or time periods as negative samples. In this way, the diversity of the generated data is enhanced. For example, the difference in the latent space can be maximized based on a normalized temperature parameterized cross-entropy loss (NT-Xent loss), which is a classical loss function commonly used in the field of contrastive learning.

[0076] A spatial topology learning module is configured to perform spatial feature enhancement on the time feature enhanced data to obtain spatio-temporal feature enhanced data.

[0077] A backbone spatio-temporal graph neural network (STGNN) is configured to output congestion level prediction values based on the spatio-temporal feature enhanced data. The backbone STGNN adopts a self-distillation mechanism, including a teacher model and a student model, and updates the parameters of the teacher model using an exponential moving average (EMA).

[0078] S3: constructing a multi-task loss function , the expression of which is as follows:

[0079] ;

[0080] ;

[0081] , wherein, , , is a weight coefficient, is an original supervised loss function, is a distillation loss function, is a combination of the VAE loss function and the contrastive learning loss function, is the VAE loss function, is the contrastive learning loss function.

[0082] Optionally, the original supervised loss function Mean Absolute Error (MAE), Distillation loss function KL divergence.

[0083] Distillation loss function KL divergence is used to calculate the difference between the student model output and the teacher model soft label, and the formula is:

[0084] ;

[0085] wherein, temperature parameter.

[0086] Step S4: training the traffic congestion prediction model using the traffic feature time series data, and optimizing the model parameters by minimizing the multi-task loss function.

[0087] Step S5: inputting the to-be-predicted traffic data set into the trained traffic congestion prediction model, and outputting the predicted congestion levels of each node at each sampling time point.

[0088] The core innovation of the embodiment is the construction strategy of the traffic congestion prediction model in step S2. Specifically embodied in:

[0089] First, an improved time feature generator is used.

[0090] The existing time feature generator only includes a variational autoencoder (VAE). In the present embodiment, the time feature generator is improved by adding a contrastive learning module, and a contrastive learning loss (function) is added in the latent space to force the generated time series data to align with the real data in the time series mode, avoiding generating invalid or deviating from the real distribution data.

[0091] As an existing part of the time feature generator, the variational autoencoder (VAE) is used to learn the latent distribution of the traffic data and generate synthetic samples. The input data is mapped to the mean and variance of the latent space through the encoder, and then the latent variables are generated by reparameterization sampling. Finally, the decoder is used to reconstruct the sample similar to the original input data but with a certain randomness. During training, the model parameters are optimized by minimizing the reconstruction error and KL divergence, so that the generated samples not only retain the main features of the data but also have diversity.

[0092] In combination with reference Figure 3 As shown in the figure, the specific construction process (including the training method) of the improved time feature generator is as follows:

[0093] Initialization phase: initialize the variational autoencoder (including the encoder and the decoder) and the contrastive learning module (including the shared encoder and the projection head). Ensure that the encoder of the variational autoencoder (VAE) and the feature extraction network of the contrastive learning module share the spatio-temporal graph convolutional layer parameters.

[0094] Variational Autoencoder (VAE) pre-training phase: Train a Variational Autoencoder (VAE) using raw traffic data, learn the latent distribution of the data by minimizing reconstruction error and KL divergence. The trained Variational Autoencoder (VAE) can generate synthetic samples consistent with the true data distribution, providing more diverse training data for contrastive learning.

[0095] Joint training phase, specifically including:

[0096] Data generation: Variational Autoencoder (VAE) generates synthetic samples from latent space and forms training set with original samples.

[0097] Data augmentation: Apply various data augmentation strategies to original samples and Variational Autoencoder (VAE) generated samples to generate positive sample pairs and negative sample pairs required for contrastive learning.

[0098] Feature extraction and contrastive learning: Input augmented samples into shared encoder for feature extraction, then map to contrastive space through projection head, calculate contrastive learning loss and update model parameters.

[0099] Variational Autoencoder (VAE) optimization: Use features extracted during contrastive learning to guide further optimization of Variational Autoencoder (VAE) through reconstruction consistency loss, making it generate more samples that meet the needs of contrastive learning.

[0100] Dynamic weight adjustment phase: Adjust the weights of Variational Autoencoder (VAE) loss, contrastive learning loss and reconstruction consistency loss dynamically during training. In the early stage of training, focus on Variational Autoencoder (VAE) learning the basic distribution of data, weight bias towards Variational Autoencoder (VAE) loss. In the later stage of training, increase the weight of contrastive learning, strengthen the robustness of feature representation, and ensure the interpretability of features through reconstruction consistency loss.

[0101] Iterative optimization phase: Repeat the above joint training process, Variational Autoencoder (VAE) continuously generates better synthetic samples, contrastive learning module continuously optimizes feature representation, forming a closed loop of mutual promotion; with training iterations, the model gradually learns feature representation that conforms to data distribution and is robust to perturbations, improving the accuracy and stability of traffic flow prediction.

[0102] Optionally, the contrastive learning module adopts SimCLR architecture, including:

[0103] Randomly perturb the input time window (such as node dropout, noise injection) to generate positive samples.

[0104] Take features of different nodes or non-adjacent time periods as negative samples.

[0105] Alternatively, in a time feature generator, a time series data is subjected to two random perturbations, resulting in two similar versions, which constitute a positive sample pair. Other samples in the same batch of training data are treated as negative samples.

[0106] Optionally, the contrastive learning module uses the NT-Xent loss function:

[0107] ;

[0108] in, Original features For positive sample features, For temperature parameters, For batch size, This represents the contrastive learning loss function.

[0109] The latent space contrastive loss is calculated using the NT-Xent loss function. In the latent space (Embedding vector space) obtained after feature extraction, the feature representation is optimized by calculating the similarity difference between positive and negative sample pairs.

[0110] Second, the spatial topology learning module.

[0111] The introduced spatial topology learning module uses a graph convolutional network (GCN) layer to learn the road connection relationships between nodes, obtain a spatial topology feature matrix, and concatenate it with the temporally enhanced data to obtain spatiotemporally enhanced data.

[0112] The spatial topology learning module is used for:

[0113] Combine the node's time-series sliding window data with the initial adjacency matrix. As input;

[0114] Using a multilayer perceptron (MLP) encoder, the time-series sliding window data of each node is converted into a hidden representation;

[0115] Through another multilayer perceptron (MLP), each pair of nodes , The hidden representation is mapped to a scalar Finally, a scalar matrix is ​​obtained. Using this matrix Parameterize the Bernoulli distribution between node pairs;

[0116] By employing the Gumbel-Softmax reparameterization technique, a refined adjacency matrix is ​​constructed by sampling from the Bernoulli distribution. To characterize the learned spatial topology, i.e. ,in is a temperature parameter;

[0117] To enhance the sparsity of the spatial topology, the threshold is applied to the transformation , i.e. , where is a temperature, so that the possibility of generating an edge between nodes and is reduced.

[0118] Finally, the module outputs a refined adjacency matrix , which can be used for subsequent prediction tasks of the backbone spatio-temporal graph neural network (STGNN).

[0119] Third, the improved backbone spatio-temporal graph neural network (STGNN).

[0120] The existing backbone spatio-temporal graph neural network (STGNN) does not contain a self-distillation mechanism. The present embodiment improves the backbone spatio-temporal graph neural network (STGNN) and introduces a self-distillation mechanism. The backbone core network therein is the existing part, and the improved part includes a teacher model-student model structure and a multi-task loss function part.

[0121] In combination with reference Figure 4 , the construction process of the improved backbone spatio-temporal graph neural network (STGNN) is as follows:

[0122] 1. Model initialization stage, specifically including the following processes:

[0123] Create a double model structure: initialize two completely identical spatio-temporal graph neural network (STGNN) models as a teacher model and a student model, respectively. Both have the same architecture (such as multiple layers of spatio-temporal convolution blocks, output layers), but the parameters are independent.

[0124] Parameter synchronization: in some implementations, the student model inherits the initial parameters (such as pre-training weights) of the teacher model to accelerate convergence.

[0125] 2. Teacher model training stage, specifically including the following processes:

[0126] Data loading and preprocessing: obtain historical traffic data (such as vehicle speed, traffic flow) from a traffic sensor network; construct a spatio-temporal graph structure, with nodes representing sensors and edges representing road connection relationships; standardize and perform sliding window processing on the data to generate training samples. In the present embodiment, data enhanced with spatio-temporal features is used as training samples.

[0127] Forward Propagation: The teacher model receives input data, extracts spatial and temporal features through spatio-temporal convolution layers; the output layer generates traffic flow prediction at future time steps.

[0128] Loss Calculation and Parameter Update: Calculate the loss (e.g., RMSE, MAE) between predicted values and true values; update the teacher model parameters through backpropagation.

[0129] 3. Teacher Model Knowledge Extraction Phase, including the following processes:

[0130] Fixed Teacher Parameters: After training, freeze all parameters of the teacher model to ensure stable output during knowledge transfer.

[0131] Generate Soft Labels (Soft Targets): The teacher model performs forward propagation on the training data again to generate prediction results; apply temperature scaling to the prediction results to soften the probability distribution and expose more relative relationships between classes. The formula for generating soft labels is as follows:

[0132] ;

[0133] Where T is the temperature When T is larger, the distribution is smoother, When T is smaller, it degenerates to the standard .

[0134] 4. Student Model Distillation Training Phase, including the following processes:

[0135] Data Preparation: Use the same training data as the teacher model; the student model receives the same input and generates its own prediction (hard prediction).

[0136] Multi-task Loss Calculation (for ease of understanding, do not consider the contrast learning loss for the time being):

[0137] Task Loss (i.e., the original supervised loss function): The loss between the student's hard prediction and the true label (e.g., MSE), ensuring that the student learns the basic task;

[0138] Distillation Loss (Distillation Loss): The loss between the student's soft prediction (generated by the same temperature T) and the teacher's soft label (e.g., KL divergence), guiding the student to learn the teacher's generalization ability;

[0139] Combine the two losses with weights to get the total loss, the formula is as follows:

[0140] ;

[0141] Where For distillation weight, usually increases with the training process.

[0142] In this embodiment, because the time feature generator adds a contrast learning module and introduces a contrast learning loss (function) in the latent space, the multi-task loss function, i.e., the total loss is calculated as follows:

[0143] ;

[0144] ;

[0145] wherein, , , is a weight coefficient, is an original supervised loss function, is a distillation loss function, is a combination of the VAE loss function and the contrast learning loss function, is the VAE loss function, is the contrast learning loss function;

[0146] Student model parameter update: only update the parameters of the student model through back propagation, so that it can fit the real label and the teacher's soft knowledge at the same time.

[0147] 5. Dynamic adjustment of distillation parameter stage, including the following contents:

[0148] Temperature annealing: a higher temperature T is used in the early stage of training to soften the teacher output distribution and encourage the student to learn more extensive knowledge; the temperature is reduced in the later stage of training to focus on the teacher's confident prediction and improve the student's accuracy.

[0149] Distillation weight scheduling: in the early stage of model training, the original supervised loss weight ( ) is set to 1.0, which dominates and prompts the student model to focus on learning the basic mode corresponding to the original supervised loss function, ensuring the mastery of the basic task. The distillation loss weight ( ) is initially set to 0.5, and the contrast learning loss weight is set to 0.1, which assists the original supervised loss in training.

[0150] As the training progresses, after a warm-up round, linear decay begins. Specifically, as the training round increases, the distillation weight is updated by the formula , from 1.0 to 0.1. In this process, the relative influence of distillation loss and contrastive learning loss gradually increases. Distillation loss softens the teacher model output distribution, measures the difference between student model and teacher model output using KL divergence, and helps the student model learn complex decision logic; contrastive learning loss maximizes the similarity of positive samples and minimizes the similarity of negative samples, thereby enhancing the model's ability to identify similar samples. Therefore, as the decay of , the student model can gradually absorb the advanced knowledge of the teacher model, realizing a smooth transition from basic learning to knowledge refinement.

[0151] 6. Teacher model update, specifically including the following process:

[0152] Periodic parameter inheritance: In some implementations, every certain training period, the parameters of the student model are copied back to the teacher model.

[0153] Form a closed-loop self-learning: The teacher model continuously evolves by absorbing the latest knowledge of the student, forming an iterative improvement process.

[0154] Finally, the inference stage, only using the student model for prediction, the teacher model no longer participates; since the student model has learned both the true label and the teacher's generalization knowledge, its performance is usually better than that of a model trained alone.

[0155] In this embodiment, the student model and the teacher model share the main network structure, and the teacher model is updated from the student model parameters through EMA, the formula is:

[0156]

[0157] wherein, is the EMA decay coefficient, and EMA is the exponential moving average.

[0158] It can be seen that the traffic congestion prediction method based on contrastive learning and knowledge distillation in this embodiment solves the shortcomings of the existing ST-FiT model through the following technical solutions:

[0159] According to the traffic indicators in the data set, the traffic density is calculated, and the congestion level is divided based on the traffic density.

[0160] A contrastive learning module is introduced into the variational autoencoder (VAE), positive samples are generated by applying perturbation to the time window, and the features of different nodes or time periods are used as negative samples, the NT-Xent loss is maximized based on the difference in the hidden space, and the diversity of generated data is enhanced.

[0161] The backbone spatio-temporal graph neural network (STGNN) with a self-distillation optimization mechanism is adopted, a teacher-student model architecture and an exponential moving average (EMA) are used to update the parameters of the teacher model, and the original loss and the distillation loss are combined to improve the prediction accuracy.

[0162] Through the combination of contrastive learning and self-distillation, the prediction accuracy is significantly improved, and the method is suitable for traffic congestion prediction scenarios.

[0163] In summary, the traffic congestion prediction method based on contrastive learning and knowledge distillation in the embodiment has the following advantages:

[0164] 1. Feature diversity and robustness are improved.

[0165] The application introduces a contrastive learning module into the variational autoencoder (VAE) to build an internal mechanism of "generation-discrimination" cooperation. The variational autoencoder (VAE) is responsible for learning the basic distribution of the data to ensure the generation quality, while the contrastive learning module maximizes the difference between positive and negative samples in the hidden space, explicitly encouraging the model to mine and retain those non-typical time series patterns that are easily ignored by the variational autoencoder (VAE). This internal cooperative mechanism effectively avoids the mode collapse problem of the variational autoencoder (VAE) in principle, and the generated enhanced data is not only diverse, but also has better feature discrimination.

[0166] 2. Model generalization ability and knowledge extraction ability are enhanced.

[0167] The self-distillation mechanism introduced in the spatio-temporal graph neural network (STGNN) is essentially a dynamic evolving "teacher-student mutual promotion" learning system. The teacher model smooths the parameters of the student model through exponential moving average (EMA), providing a more stable and informative "soft label" supervision signal. This soft label contains relative relationship information between categories (congestion levels), which can guide the student model to learn the decision boundaries between categories and the internal rules of the data, thereby improving the generalization ability of the model in complex traffic scenarios that have not been seen before.

[0168] 3. Synergistic optimization effect of multi-task loss.

[0169] The multi-task loss function designed in the application is not a simple weighted sum. Through the proposed dynamic weight adjustment strategy, the loss function guides the model to optimize in stages with emphasis during the training process: in the early stage, focus on mastering the basic prediction task (high ), in the middle stage, start to absorb the generalization knowledge of the teacher model (improve ), and in the later stage, strengthen the feature learning of disturbance robustness (improve ). This orderly and dynamic optimization process ensures that the model ultimately achieves the optimal balance between accuracy, generalization, and robustness in the learning path.

[0170] In summary, by jointly optimizing contrastive learning and self-distillation, the application significantly improves the accuracy and robustness of traffic congestion prediction, making it suitable for real-time prediction scenarios in intelligent transportation systems.

[0171] Embodiment 2:

[0172] As mentioned in Embodiment 1, the original traffic data set includes at least two traffic indicators: traffic flow and average speed, and may also include average occupancy.

[0173] Based on Embodiment 1, the preprocessing in step S1 includes:

[0174] Extract traffic flow and average speed from the original traffic data set as traffic indicators; if the original traffic data set includes average occupancy, extract it together.

[0175] Replace zero values in the extracted traffic indicators with the average of non-zero values.

[0176] Calculate traffic density.

[0177] Normalize the traffic density to the range [0, 1].

[0178] Based on the normalized traffic density, use linear mapping method to divide congestion levels.

[0179] Combine the extracted traffic indicators and congestion levels to form traffic feature time series data.

[0180] Embodiment 3:

[0181] Based on Embodiment 2, calculating traffic density includes:

[0182] If the original traffic data set does not include average occupancy, use the following formula to calculate traffic density:

[0183] ;

[0184] where, is the traffic density, is the traffic flow, is the average speed.

[0185] If the original traffic data set includes average occupancy, use the following steps to calculate traffic density:

[0186] The average occupancy rate is normalized, and then weighted sum with the initial density calculated based on traffic flow and average speed to get the final traffic density. The specific formula is as follows: ;

[0187] ;

[0188] wherein, is the traffic density, is the initial density, is the normalized average occupancy rate, is the weight coefficient.

[0189] In this embodiment, the weight coefficient is 0.7 by default, which can be adjusted according to actual conditions.

[0190] Example 4:

[0191] On the basis of example 2, the traffic density is normalized to the range [0, 1] including:

[0192] The minimum value and the maximum value of the traffic density array are calculated.

[0193] If is equal to , an array with the same shape as and all elements are 0 is returned.

[0194] Otherwise, linear normalization is performed using the following formula:

[0195] ;

[0196] wherein, denotes the normalized result, is a sample value in the array .

[0197] Example 5:

[0198] On the basis of example 2, the congestion level is a continuous congestion index converted from the normalized traffic density by linear mapping method. The congestion index divides the traffic congestion condition into four congestion levels: smooth, slight congestion, moderate congestion, and severe congestion.

[0199] In this embodiment, the specific process of dividing the congestion level by linear mapping method is as follows:

[0200] The normalized traffic density is calculated as , , Quartiles, the normalized traffic density is divided into 4 intervals: [0, ]、( , ]、( , ]、( , 1]。

[0201] The 4 intervals are linearly mapped to the equidistant subintervals in the range of [0, 1] respectively.

[0202] Let the normalized traffic density be :

[0203] If , then The corresponding congestion index , the interval [0, p25] corresponds to the congestion index range [0, 0.25], corresponding to the first congestion level: smooth and unobstructed.

[0204] If , then The corresponding congestion index , the interval (p25, p50] corresponds to the congestion index range (0.25, 0.5], corresponding to the second congestion level: slight congestion.

[0205] If , then The corresponding congestion index , the interval (p50, p75] corresponds to the congestion index range (0.5, 0.75], corresponding to the third congestion level: moderate congestion.

[0206] If , then The corresponding congestion index , the interval (p75, 1] corresponds to the congestion index range (0.75, 1], corresponding to the fourth congestion level: severe congestion.

[0207] Embodiment 6:

[0208] As shown in Figure 4 , the embodiment provides a technical scheme:

[0209] A traffic congestion prediction device based on contrast learning and knowledge distillation, comprising the following modules:

[0210] A preprocessing module configured to preprocess an original traffic data set to obtain traffic feature time series data, wherein the traffic feature time series data includes traffic indicators and congestion levels based on traffic density division;

[0211] The model construction module is configured to construct a traffic congestion prediction model, and sequentially comprises:

[0212] The time feature generator is configured to perform time feature enhancement on the traffic feature time series data, and comprises a variational autoencoder (VAE) and a contrastive learning module.

[0213] The spatial topology learning module is configured to perform spatial feature enhancement on the data after time feature enhancement to obtain data after spatio-temporal feature enhancement.

[0214] The backbone spatio-temporal graph neural network (STGNN) is configured to output congestion level prediction values based on the data after spatio-temporal feature enhancement.

[0215] The multi-task loss function is configured to be constructed , and an expression thereof is:

[0216] ;

[0217] ;

[0218] wherein, 、 、 are weight coefficients, is an original supervised loss function, is a distillation loss function, is a combination of a VAE loss function and a contrastive learning loss function, is the VAE loss function, is the contrastive learning loss function.

[0219] The model training module is configured to train the traffic congestion prediction model using the traffic feature time series data, and optimize the model parameters by minimizing the multi-task loss function.

[0220] The prediction module is configured to input a to-be-predicted traffic data set into the trained traffic congestion prediction model, and output predicted congestion levels of each node at each sampling time point.

[0221] The traffic congestion prediction device in the embodiment can be used to implement any one of the traffic congestion prediction methods in Embodiments 1 to 5, and further implementation details can be referred to the descriptions in the above embodiments, which will not be described herein.

Claims

1. A traffic congestion prediction method based on contrastive learning and knowledge distillation, characterized in that, Includes the following steps: S1: Preprocess the original traffic dataset to obtain traffic feature time series data, wherein the traffic feature time series data includes traffic indicators and congestion levels based on traffic density. S2: Construct a traffic congestion prediction model, which includes the following steps: A time feature generator is used to enhance the time features of the traffic feature time series data. It includes a variational autoencoder and a contrastive learning module. The contrastive learning module is configured to generate positive samples by time series perturbation and use the features of different nodes or time periods as negative samples to maximize the latent space difference. The spatial topology learning module is used to enhance the spatial features of the data after temporal feature enhancement, resulting in data with enhanced spatiotemporal features. A backbone spatiotemporal graph neural network is used to output congestion level predictions based on data enhanced with spatiotemporal features. The backbone spatiotemporal graph neural network adopts a self-distillation mechanism, including a teacher model and a student model, and uses exponential moving average to update the parameters of the teacher model. S3: Construct a multi-task loss function; S4: Train the traffic congestion prediction model using the traffic feature time series data, and optimize the model parameters by minimizing the multi-task loss function; S5: Input the traffic dataset to be predicted into the trained traffic congestion prediction model, and output the predicted congestion level of each node at each sampling time point.

2. The traffic congestion prediction method based on contrastive learning and knowledge distillation according to claim 1, characterized in that, Multi-task loss function The expression is: ； ； in, , , Here are the weighting coefficients, which are... The original supervised loss function is... Let be the distillation loss function. This is a combination of the VAE loss function and the contrastive learning loss function. The loss function for VAE is... To compare the learning loss function.

3. The traffic congestion prediction method based on contrastive learning and knowledge distillation according to claim 2, characterized in that, The preprocessing in step S1 includes: Traffic flow and average speed are extracted as traffic indicators from the original traffic dataset; if the original traffic dataset includes average occupancy, it is also extracted. Replace the zero values ​​in the extracted traffic indicators with the average of the non-zero values; Calculate traffic density; Traffic density is normalized to the range [0, 1]. Based on the normalized traffic density, a linear mapping method is used to classify congestion levels. The extracted traffic indicators are combined with the congestion level to form the time-series data of traffic characteristics.

4. The traffic congestion prediction method based on contrastive learning and knowledge distillation according to claim 3, characterized in that, The calculation of traffic density includes: If the original traffic data does not include average occupancy, traffic density is calculated using the following formula. : ; in, For traffic flow, Average speed; If the original traffic dataset includes average occupancy, then traffic density is calculated using the following formula. : in, , The normalized average market share These are the preset weighting coefficients.

5. The traffic congestion prediction method based on contrastive learning and knowledge distillation according to claim 4, characterized in that, The normalization of traffic density to the [0,1] range includes: Calculate the traffic density array minimum value and maximum value ; like and If they are equal, return a value equal to... An array of the same shape, where all elements are 0; Otherwise, use the following formula for linear normalization: ； in, This represents the result after normalization. For array A sample value in the dataset.

6. The traffic congestion prediction method based on contrastive learning and knowledge distillation according to claim 5, characterized in that, The congestion level is defined by a continuous congestion index, which is derived from the normalized traffic density through a linear mapping method. The congestion index classifies traffic conditions into four levels: unimpeded, slightly congested, moderately congested, and severely congested.

7. The traffic congestion prediction method based on contrastive learning and knowledge distillation according to claim 6, characterized in that, The normalized traffic density is converted into a continuous congestion index using a linear mapping method, specifically as follows: Calculate the normalized traffic density , , Third place; Traffic density value Mapped to a congestion index according to the following rules: like The congestion index is... ; like The congestion index is... ; like The congestion index is... ; like The congestion index is... .

8. A traffic congestion prediction device based on contrastive learning and knowledge distillation, characterized in that, Includes the following modules: Preprocessing module: configured to preprocess the original traffic dataset to obtain traffic feature time series data, wherein the traffic feature time series data includes traffic indicators and congestion levels based on traffic density; Model building module: Configured to build traffic congestion prediction models, including: A time feature generator is used to enhance the time features of the traffic feature time series data. It includes a variational autoencoder and a contrastive learning module. The contrastive learning module is configured to generate positive samples by time series perturbation and use the features of different nodes or time periods as negative samples to maximize the latent space difference. Spatial topology learning module: used to perform spatial feature enhancement on data that has been enhanced with temporal features, to obtain data with enhanced spatiotemporal features; The backbone spatiotemporal graph neural network is used to output congestion level prediction values ​​based on data enhanced with spatiotemporal features. The backbone spatiotemporal graph neural network adopts a self-distillation mechanism, including a teacher model and a student model, and uses exponential moving average to update the parameters of the teacher model. And it is configured to construct a multi-task loss function, and to add a dynamic weight adjustment strategy to the multi-task loss function; Model training module: configured to train the traffic congestion prediction model using the traffic feature time series data, and optimize the model parameters by minimizing the multi-task loss function; Prediction module: It is configured to input the traffic dataset to be predicted into the trained traffic congestion prediction model and output the predicted congestion level of each node at each sampling time point.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-7.

Images