Traffic robust knowledge distillation training prediction method and device, equipment and storage medium

Abstract

The application discloses a traffic robust knowledge distillation training prediction method and device, equipment and a storage medium, the method comprises the following steps: acquiring traffic flow historical observation data, dividing the traffic flow historical observation data into source domain training data and out-of-distribution verification data, and constructing a knowledge distillation architecture comprising a teacher model and a student model; according to the source domain training data and the out-of-distribution verification data, performing meta-learning double-layer optimization training on the knowledge distillation architecture, using the source domain training data in the inner loop to guide the student model to imitate the teacher model, using the out-of-distribution verification data in the outer loop to evaluate the generalization performance of the student model and correct the parameters of the teacher model in reverse, until a robust target student model is obtained; according to the target student model, receiving real-time input traffic flow data and performing forward reasoning, and outputting a prediction result of future traffic conditions, which can significantly improve the generalization ability and accuracy of traffic prediction, and realize efficient and robust prediction in complex traffic environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a robust knowledge distillation training prediction method, apparatus, equipment, and storage medium for transportation. Background Technology

[0002] Real-time traffic flow prediction is the core foundation of Intelligent Transportation Systems (ITS). However, deploying it on edge devices faces a key dilemma: deep models (such as Spatio-Temporal Graph Neural Networks (STGNNs)) are too large for resource-constrained hardware, while lightweight models (such as Multi-Layer Perceptrons (MLPs)) often have extremely poor generalization ability under non-stationary distribution shifts (such as the recurring concept drift between weekday commutes and irregular weekends). Existing knowledge distillation methods exacerbate this problem by forcing student models to blindly mimic the biases of teacher models in the source domain, ultimately leading to negative transfer in out-of-distribution (OOD) scenarios.

[0003] Over the past decade, deep learning methods have achieved remarkable results in the field of traffic flow prediction. Early works such as Diffusion Convolutional Recurrent Neural Network (DCRNN) and Spatio-Temporal Graph Convolutional Networks (STGCN) introduced Graph Neural Networks (GNN) to model spatial dependencies and combined recurrent or convolutional layers to capture temporal dynamics.

[0004] In recent years, advanced methods such as Graph WaveNet, D2STGNN, and STAEformer based on Transformer have further improved prediction accuracy by modeling complex dynamic correlations.

[0005] However, deploying these complex models in real-world scenarios faces two major bottlenecks. First, there is low computational efficiency; current mainstream models typically contain a large number of graph convolutions and attention mechanisms, resulting in excessive computational overhead and memory consumption. Real-time traffic flow prediction (RT-TFP) often needs to run on resource-constrained edge devices (such as intelligent traffic lights). When highly complex spatiotemporal graph neural networks (STGNNs) are executed on such hardware, inference latency reduces the system's ability to respond quickly to sudden changes in traffic flow, which contradicts the core objective of real-time traffic flow prediction. Although recent lightweight baseline models such as Spatial-Temporal Identity (STID) and Time-series Dense Encoder (TiDE) use simple multilayer perceptrons (MLPs) to accelerate inference, they generally lack the ability to capture complex dynamic heterogeneity compared to graph neural network models. Summary of the Invention

[0006] The main objective of this invention is to provide a traffic robust knowledge distillation training prediction method, apparatus, device, and storage medium, aiming to solve the technical problems of decreased generalization performance and insufficient robustness of existing traffic prediction models when facing non-stationary distribution shifts.

[0007] In a first aspect, the present invention provides a traffic robust knowledge distillation training and prediction method, the traffic robust knowledge distillation training and prediction method comprising the following steps: Historical traffic flow observation data is acquired, and the historical traffic flow observation data is divided into source domain training data and out-of-distribution validation data. A knowledge distillation architecture containing teacher model and student model is constructed to provide a data foundation and network structure for model optimization. Based on the source domain training data and the out-of-distribution validation data, the knowledge distillation architecture is subjected to meta-learning two-layer optimization training. In the inner loop, the source domain training data is used to guide the student model to imitate the teacher model. In the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model until convergence is obtained to obtain a robust target student model. The target student model receives real-time traffic flow data and performs forward inference to output a prediction of future traffic conditions.

[0008] Optionally, the step of acquiring historical traffic flow observation data involves dividing the historical traffic flow observation data into source domain training data and out-of-distribution validation data, and constructing a knowledge distillation architecture that includes a teacher model and a student model to provide a data foundation and network structure for model optimization, including: Obtain the raw dataset containing historical traffic flow observation sequences and road network structure data; The original dataset is divided into source domain training data that follows the independent and identically distributed hypothesis and out-of-distribution validation data that simulates concept drift scenarios; Simultaneously, a knowledge distillation architecture containing a teacher model and a student model is initialized and constructed. The student model has a built-in content-aware temporal pooling encoder and a spatial decoupling encoder. The content-aware temporal pooling encoder is used to extract traffic dynamic features from the time dimension and weaken the influence of time and location offset. The spatial decoupling encoder is used to separate the static attributes of nodes from the traffic dynamic rules from the spatial dimension to realize spatial heterogeneity modeling. The content-aware temporal pooling encoder and the spatial decoupling encoder provide network structure support for two-layer optimized distillation training based on the meta-learning framework.

[0009] Optionally, the step of performing meta-learning two-layer optimization training on the knowledge distillation architecture based on the source domain training data and the out-of-distribution validation data, wherein the inner loop uses the source domain training data to guide the student model to imitate the teacher model, and the outer loop uses the out-of-distribution validation data to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model, until convergence is obtained to obtain a robust target student model, includes: Based on the source domain training data and the distributed out-of-distribution validation data, the knowledge distillation architecture is subjected to meta-learning two-layer optimization training to construct an iterative process that includes inner loop optimization and outer loop correction. In the inner loop, the teacher model parameters are fixed, and the distillation loss is calculated using the source domain training data. The student model parameters are updated using gradient descent to minimize the divergence between the student model output and the teacher model output, thus obtaining temporary student model parameters. In the outer loop, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function. The gradient with respect to the teacher model parameters is calculated based on the meta-loss function. The implicit function theorem is used to avoid unfolding the complete training trajectory, and the teacher model parameters are back-corrected to remove domain-specific noise. Repeat the inner loop optimization and outer loop correction until the meta-loss function converges to obtain a robust target student model.

[0010] Optionally, in the inner loop, the teacher model parameters are fixed, the distillation loss is calculated using the source domain training data, and the student model parameters are updated via gradient descent to minimize the divergence between the student model output and the teacher model output, resulting in temporary student model parameters, including: In the inner loop, a subset of source domain data is sampled from the source domain training data; With the teacher model parameters fixed, a temporary student model is constructed on the source domain data subset using the following formula to mimic the objective function of the current teacher model:

[0011] in, These are temporary student model parameters. For student model parameters, The loss function for the inner loop training is... A subset of the source domain data. For teacher model parameters; The parameters of the temporary student model are approximated using gradient descent, and the update formula is:

[0012] in, These are temporary student model parameters. For student model parameters, For learning rate, For gradient operators, For balance coefficient, For the loss of true labels, A subset of the source domain data. For teacher model parameters, This is due to knowledge distillation loss.

[0013] Optionally, in the outer loop, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function. Based on the meta-loss function, the gradient with respect to the teacher model parameters is calculated. The implicit function theorem is used to avoid unfolding the complete training trajectory, and the teacher model parameters are back-corrected to remove domain-specific noise. This includes: In the outer loop, the performance of the temporary student model parameters is evaluated on the out-of-distribution validation set representing the distribution shift, and the meta-loss is calculated, which characterizes the error of the student model in the distribution shift scenario. The gradient of the meta-loss with respect to the teacher model parameters is calculated using the following formula as a correction signal:

[0014] in, For the meta-loss function, For teacher model parameters, These are temporary student model parameters. For the elementary gradient, For the loss, the partial gradient with respect to the student parameters, It is a Jacobian matrix; Based on the implicit function theorem, the parameters of the temporary student model are assumed to converge to a stationary point, and the stationary point satisfies the following constraints:

[0015] in, For stationary point constraint functions, These are the temporary student model parameters after convergence. For teacher model parameters, For the gradient operator with respect to the parameters of the student model, Let be the distillation loss function. This is a condition for a stationary point; Based on the implicit function theorem, the Jacobian matrix is ​​determined by differentiating the constraint equations with respect to the teacher model parameters:

[0016]

[0017] in, The Hessian matrix represents the distillation loss. The notation for the Hessian matrix is ​​as follows: The Jacobian matrix of the optimal student parameters with respect to the teacher parameters. Let be the mixing second-order partial derivative matrix of distillation loss. The symbol for the mixed partial derivative matrix; The teacher model is updated to simultaneously minimize both the out-of-distribution error of the student model and its own true label error:

[0018]

[0019] in, For teacher model parameters, For the teacher model learning rate, For the elementary gradient, For balance coefficient, The gradient is the loss value for the true label. For the loss of true labels, The gradient of the meta-loss with respect to the temporary student model parameters, It is the inverse of the Hessian matrix. The mixed second-order partial derivative moment is the loss due to distillation.

[0020] Optionally, the step of repeatedly performing the inner loop optimization and outer loop correction until the meta-loss function converges to obtain a robust target student model includes: Repeat the iterative process of inner loop optimization and outer loop correction. In each iteration, the combined loss is calculated using a subset of the source domain training data to update the student model parameters, and the meta-gradient is calculated using the out-of-distribution validation set to correct the teacher model parameters. Continuously monitor the numerical changes of the meta-loss function until the meta-loss function converges to a stable state; After training, the final updated student model parameters are saved as a robust target student model. The formula for updating the student model parameters at the end of a single iteration is expressed as:

[0021] in, For student model parameters, For the student model learning rate, For the gradient operator with respect to the parameters of the student model, For the combined loss function, A subset of the source domain data. These are the parameters for the teacher model.

[0022] Optionally, the step of receiving real-time input traffic flow data based on the target student model and performing forward inference to output a prediction result of future traffic conditions includes: Receive real-time traffic flow data based on the target student model; The real-time traffic flow data is input into the student model as a node input sequence. The dynamic attention score, based on the input content rather than the time step index, is calculated using a lightweight scoring function parameterized by a learnable vector and a projection matrix, as follows:

[0023]

[0024]

[0025]

[0026] in, For dynamic attention scores, For learnable vectors, It is the transpose symbol. The hyperbolic tangent activation function is used. For the projection matrix, Input features for the nodes, For bias terms, Input a sequence for the node. For node indexing, The time step index represents the time step from the 1st time step to the Tth time step. For a single time step feature vector, Let be a real matrix space. For sequence length, For feature dimensions; Attention weights are obtained by softmax normalization along the temporal dimension based on the dynamic attention scores:

[0027] in, Attention weights It is an exponential function. For dynamic attention scores, For summation index variables, For sequence length, For the first Dynamic attention score at each time step; The robust node representation calculated using the attention weights is a weighted aggregation result:

[0028] in, For robust node representation, Attention weights Input features for the nodes, For node indexing, For sequence length, For time step index; Introducing a learnable spatial context encoder to generate spatial location codes

[0029]

[0030] in, Encoding spatial location, For a learnable spatial context encoder, For node indexing, for 3D real vector space, The total number of nodes. For location encoding dimension, It is a real matrix space; The robust node representation is fused with the spatial location encoding and adjacency encoding to obtain the comprehensive node state:

[0031] in, For the overall node status, For robust node representation, Encoding spatial location, For adjacency coding, Let be a real vector space. For time-series feature dimensions, For location encoding dimension, For adjacency encoding dimensions; A lightweight multilayer perceptron with weights shared across all nodes is used as a function fitter to perform forward inference, generating predictions of future traffic conditions using the following formula:

[0032] in, Predict the output for the student model. To share multilayer perceptron functions, For the overall node status, This is the first layer weight matrix. This is the weight matrix for the second layer. For activation function, This is the first layer bias vector. This is the bias vector for the second layer.

[0033] Secondly, to achieve the above objectives, the present invention also proposes a traffic robust knowledge distillation training and prediction device, the traffic robust knowledge distillation training and prediction device comprising: An architecture building module is used to acquire historical traffic flow observation data, divide the historical traffic flow observation data into source domain training data and out-of-distribution validation data, and construct a knowledge distillation architecture that includes teacher models and student models, providing a data foundation and network structure for model optimization. The two-layer optimization training module is used to perform meta-learning two-layer optimization training on the knowledge distillation architecture based on the source domain training data and the out-of-distribution validation data. In the inner loop, the source domain training data is used to guide the student model to imitate the teacher model. In the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model until convergence is obtained to obtain a robust target student model. The inference and prediction module is used to receive real-time traffic flow data based on the target student model, perform forward inference, and output the prediction results of future traffic conditions.

[0034] Thirdly, to achieve the above objectives, the present invention also proposes a traffic robust knowledge distillation training and prediction device, the traffic robust knowledge distillation training and prediction device comprising: a memory, a processor, and a traffic robust knowledge distillation training and prediction program stored in the memory and executable on the processor, the traffic robust knowledge distillation training and prediction program being configured to implement the steps of the traffic robust knowledge distillation training and prediction method as described above.

[0035] Fourthly, to achieve the above objectives, the present invention also proposes a storage medium storing a traffic robust knowledge distillation training and prediction program, wherein the traffic robust knowledge distillation training and prediction program, when executed by a processor, implements the steps of the traffic robust knowledge distillation training and prediction method as described above.

[0036] The proposed traffic robust knowledge distillation training and prediction method acquires historical traffic flow observation data, divides this data into source domain training data and out-of-distribution validation data, and constructs a knowledge distillation architecture including a teacher model and a student model, providing a data foundation and network structure for model optimization. Based on the source domain training data and the out-of-distribution validation data, a meta-learning two-layer optimization training is performed on the knowledge distillation architecture. In the inner loop, the source domain training data guides the student model to mimic the teacher model; in the outer loop, the out-of-distribution validation data evaluates the generalization performance of the student model and back-corrects the teacher model parameters, until convergence is achieved to obtain the robust target. The target student model receives real-time traffic flow data and performs forward inference to output predictions of future traffic conditions. It can construct a knowledge distillation architecture by dividing the source domain training data and out-of-distribution validation data, and combine it with a meta-learning two-layer optimization training mechanism. It uses out-of-distribution validation data to evaluate the generalization performance of the student model in the outer loop and back-corrects the parameters of the teacher model. This effectively removes domain-specific noise that is only effective in the source domain but harmful to the target domain, enabling the student model to learn robust traffic dynamics with invariance. As a result, it significantly improves the generalization ability and accuracy of traffic prediction when facing non-stationary distribution offset scenarios, and achieves efficient and robust prediction in complex traffic environments. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the traffic robust knowledge distillation training prediction method of the present invention; Figure 3 This is a flowchart illustrating the second embodiment of the traffic robust knowledge distillation training prediction method of the present invention; Figure 4 This is a flowchart illustrating the third embodiment of the traffic robust knowledge distillation training prediction method of the present invention; Figure 5 This is a functional block diagram of the first embodiment of the traffic robust knowledge distillation training prediction device of the present invention.

[0038] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0039] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0040] The solution of this invention mainly involves: acquiring historical traffic flow observation data, dividing the historical traffic flow observation data into source domain training data and out-of-distribution validation data, constructing a knowledge distillation architecture including a teacher model and a student model, providing a data foundation and network structure for model optimization; performing meta-learning two-layer optimization training on the knowledge distillation architecture based on the source domain training data and the out-of-distribution validation data, using the source domain training data in the inner loop to guide the student model to imitate the teacher model, and using the out-of-distribution validation data in the outer loop to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model, until convergence to obtain a robust target student model; and receiving real-time input traffic flow data based on the target student model. By processing traffic data and performing forward inference, the system outputs predictions of future traffic conditions. It can construct a knowledge distillation architecture by dividing source domain training data and out-of-distribution validation data, and combine it with a meta-learning two-layer optimization training mechanism. The out-of-distribution validation data is used to evaluate the generalization performance of the student model in the outer loop and back-correct the parameters of the teacher model. This effectively eliminates domain-specific noise that is only effective in the source domain but harmful to the target domain, enabling the student model to learn robust traffic dynamics with invariance. As a result, it significantly improves the generalization ability and accuracy of traffic prediction when facing non-stationary distribution offset scenarios, and achieves efficient and robust prediction in complex traffic environments. This solves the technical problems of decreased generalization performance and insufficient robustness of existing traffic prediction models when facing non-stationary distribution offsets.

[0041] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0042] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0043] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0044] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating device, a network communication module, a user interface module, and a traffic robust knowledge distillation training and prediction program.

[0045] The device of the present invention calls the traffic robust knowledge distillation training and prediction program stored in the memory 1005 through the processor 1001, and performs the operations in the traffic robust knowledge distillation training and prediction method embodiment described below.

[0046] Based on the above hardware structure, an embodiment of the traffic robust knowledge distillation training and prediction method of the present invention is proposed.

[0047] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the traffic robust knowledge distillation training prediction method of the present invention.

[0048] In a first embodiment, the traffic robust knowledge distillation training prediction method includes the following steps: Step S10: Obtain historical traffic flow observation data, divide the historical traffic flow observation data into source domain training data and out-of-distribution validation data, and construct a knowledge distillation architecture that includes teacher models and student models to provide a data foundation and network structure for model optimization.

[0049] It should be noted that by collecting historical traffic observation data and dividing it into a source domain dataset for basic training and an out-of-distribution validation set for evaluating generalization ability, and by building a knowledge distillation framework for teacher models to guide student models, the necessary data input environment and model structure foundation are laid for the subsequent meta-learning optimization process.

[0050] Step S20: Based on the source domain training data and the out-of-distribution validation data, perform meta-learning two-layer optimization training on the knowledge distillation architecture. In the inner loop, the source domain training data is used to guide the student model to imitate the teacher model. In the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model until convergence is obtained to obtain a robust target student model.

[0051] It should be understood that by using the two-layer optimization strategy of meta-learning, the student model can realize the knowledge imitation of the teacher model based on the source domain data in the inner loop, while the out-of-distribution data is used in the outer loop to evaluate the generalization performance of the student model and correct the parameters of the teacher model in reverse. Through this collaborative iteration of inner and outer loops, it is ensured that the teacher model transmits robust general knowledge, and finally converges to obtain the target student model that can adapt to the distribution shift.

[0052] Step S30: Receive real-time traffic flow data based on the target student model and perform forward inference to output the prediction results of future traffic conditions.

[0053] Understandably, by using the robust target student model obtained through meta-learning optimization, real-time traffic flow data is used as input samples to perform forward propagation calculations, thereby directly generating predicted values ​​for future traffic conditions and completing the final transformation from model training to actual prediction services.

[0054] This embodiment, through the above-described scheme, acquires historical traffic flow observation data, divides this data into source domain training data and out-of-distribution validation data, and constructs a knowledge distillation architecture containing a teacher model and a student model, providing a data foundation and network structure for model optimization. Based on the source domain training data and the out-of-distribution validation data, a meta-learning two-layer optimization training is performed on the knowledge distillation architecture. In the inner loop, the source domain training data is used to guide the student model to mimic the teacher model; in the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the teacher model parameters, until convergence yields a robust target student model. The target student model receives real-time traffic flow data and performs forward inference to output predictions of future traffic conditions. It can construct a knowledge distillation architecture by dividing source domain training data and out-of-distribution validation data, and combine it with a meta-learning two-layer optimization training mechanism. It uses out-of-distribution validation data to evaluate the generalization performance of the student model in the outer loop and back-corrects the parameters of the teacher model. This effectively removes domain-specific noise that is only effective in the source domain but harmful to the target domain, enabling the student model to learn robust traffic dynamics with invariance. As a result, it significantly improves the generalization ability and accuracy of traffic prediction when facing non-stationary distribution offset scenarios, and achieves efficient and robust prediction in complex traffic environments.

[0055] Furthermore, Figure 3 This is a flowchart illustrating the second embodiment of the traffic robust knowledge distillation training prediction method of the present invention, as shown below. Figure 3 As shown, based on the first embodiment, a second embodiment of the traffic robust knowledge distillation training prediction method of the present invention is proposed. In this embodiment, step S10 specifically includes the following steps: Step S11: Obtain the original dataset containing historical traffic flow observation sequences and road network structure data.

[0056] It should be noted that collecting historical traffic flow observation records covering the time dimension and road network topology information covering the spatial dimension forms an undivided raw data set, providing a complete spatiotemporal information input foundation for subsequent data domain division and model construction.

[0057] Step S12: Divide the original dataset into source domain training data that follows the independent and identically distributed hypothesis and out-of-distribution validation data that simulates the concept drift scenario.

[0058] Understandably, dividing the original dataset into source domain training data that conforms to the independent and identically distributed hypothesis and out-of-distribution validation data used to simulate concept drift scenarios can construct data subsets with different statistical properties.

[0059] Step S13: Simultaneously initialize and construct a knowledge distillation architecture containing a teacher model and a student model. The student model has a built-in content-aware temporal pooling encoder and a spatial decoupling encoder. The content-aware temporal pooling encoder is used to extract traffic dynamic features from the time dimension and weaken the influence of time and location offset. The spatial decoupling encoder is used to separate the static attributes of nodes from the traffic dynamic rules from the spatial dimension to realize spatial heterogeneity modeling.

[0060] It should be understood that by establishing a knowledge distillation system that includes teacher and student models, and integrating a content-aware temporal pooling encoder and a spatial decoupling encoder into the student model, static attributes and dynamic regularity features are processed through these two encoders, providing specific network structure support for subsequent knowledge distillation and model optimization.

[0061] Step S14: The content-aware temporal pooling encoder and the spatial decoupling encoder provide network structure support for the two-layer optimized distillation training based on the meta-learning framework.

[0062] It is understandable that, through the content-aware temporal pooling encoder and the spatial decoupling encoder, and through their specific feature extraction and decoupling mechanisms, the necessary network structure foundation is provided for the two-layer optimized distillation training process based on the meta-learning framework, ensuring that the knowledge imitation of the inner loop and the robustness correction of the outer loop can be successfully executed at the effective feature representation level.

[0063] This embodiment, through the above-described scheme, acquires a raw dataset containing historical traffic flow observation sequences and road network structure data; divides the raw dataset into source domain training data following the independent and identically distributed hypothesis and out-of-distribution validation data simulating concept drift scenarios; simultaneously, it initializes and constructs a knowledge distillation architecture containing a teacher model and a student model. The student model incorporates a content-aware temporal pooling encoder and a spatial decoupling encoder. The content-aware temporal pooling encoder is used to extract traffic dynamic features from the time dimension and weaken the influence of time-location offset, while the spatial decoupling encoder is used to separate node static attributes from traffic dynamic patterns from the spatial dimension, achieving spatial decoupling. Qualitative modeling; through the content-aware temporal pooling encoder and the spatial decoupling encoder, a network structure support is provided for the two-layer optimized distillation training based on the meta-learning framework. It can construct a differentiated data environment by dividing the source domain training data and the out-of-distribution validation data, and effectively separate static node attributes and dynamic traffic patterns by combining the content-aware temporal pooling encoder and the spatial decoupling encoder. This provides a solid data foundation and network structure support for the two-layer optimized distillation training based on the meta-learning framework, enabling the model to focus on learning general traffic dynamic features with invariance, thereby significantly improving the generalization performance and prediction robustness in non-stationary distribution offset scenarios.

[0064] Furthermore, Figure 4 This is a flowchart illustrating the third embodiment of the traffic robust knowledge distillation training prediction method of the present invention, as shown below. Figure 4 As shown, based on the first embodiment, a third embodiment of the traffic robust knowledge distillation training prediction method of the present invention is proposed. In this embodiment, step S20 specifically includes the following steps: Step S21: Based on the source domain training data and the distributed external validation data, perform meta-learning two-layer optimization training on the knowledge distillation architecture to construct an iterative process that includes inner loop optimization and outer loop correction.

[0065] It should be noted that the knowledge distillation architecture driven by source domain training data and out-of-distribution validation data is optimized in a meta-learning two-layer manner. By constructing an iterative process that combines inner loop optimization and outer loop correction, a collaborative training strategy is achieved to transfer basic knowledge in the source domain and perform generalization evaluation and parameter correction in out-of-distribution scenarios.

[0066] Step S22: In the inner loop, fix the teacher model parameters, calculate the distillation loss using the source domain training data, and update the student model parameters through gradient descent to minimize the divergence between the student model output and the teacher model output, thereby obtaining temporary student model parameters.

[0067] Understandably, in the inner loop optimization phase, keeping the teacher model parameters fixed, calculating the distillation loss using the source domain training data, and updating the student model parameters using the gradient descent algorithm can minimize the difference between the student model output and the teacher model output, thereby obtaining temporary student model parameters and completing the transfer of basic knowledge.

[0068] Furthermore, step S22 specifically includes the following steps: In the inner loop, a subset of source domain data is sampled from the source domain training data; With the teacher model parameters fixed, a temporary student model is constructed on the source domain data subset using the following formula to mimic the objective function of the current teacher model:

[0069] in, These are temporary student model parameters. For student model parameters, The loss function for the inner loop training is... A subset of the source domain data. For teacher model parameters; The parameters of the temporary student model are approximated using gradient descent, and the update formula is:

[0070] in, These are temporary student model parameters. For student model parameters, For learning rate, For gradient operators, For balance coefficient, For the loss of true labels, A subset of the source domain data. For teacher model parameters, This is due to knowledge distillation loss.

[0071] It should be noted that by sampling a subset of source domain data from the source domain training data, and under the premise of fixing the teacher model parameters, a temporary student model is constructed to mimic the objective function of the current teacher model. The student model parameters are then approximately solved and updated through a gradient descent step, thereby obtaining the temporary student model parameters used for outer loop evaluation.

[0072] Step S23: In the outer loop, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function. The gradient with respect to the teacher model parameters is calculated based on the meta-loss function. The implicit function theorem is used to avoid unfolding the complete training trajectory. The teacher model parameters are then back-corrected to remove domain-specific noise.

[0073] It should be understood that in the outer loop correction phase, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function, and the gradient with respect to the teacher model parameters is calculated based on the meta-loss function. In this process, the implicit function theorem is used to avoid unfolding the complete inner loop training trajectory in order to improve computational efficiency. Finally, the calculated gradient is used to back-correct the teacher model parameters, which can eliminate domain-specific noise that is only effective in the source domain but harmful to the target domain, ensuring that the knowledge transmitted by the teacher model has cross-domain robustness and achieving the training effect of co-optimization of inner and outer loops.

[0074] Furthermore, step S23 specifically includes the following steps: In the outer loop, the performance of the temporary student model parameters is evaluated on the out-of-distribution validation set representing the distribution shift, and the meta-loss is calculated, which characterizes the error of the student model in the distribution shift scenario. The gradient of the meta-loss with respect to the teacher model parameters is calculated using the following formula as a correction signal:

[0075] in, For the meta-loss function, For teacher model parameters, These are temporary student model parameters. For the elementary gradient, For the loss, the partial gradient with respect to the student parameters, It is a Jacobian matrix; Based on the implicit function theorem, the parameters of the temporary student model are assumed to converge to a stationary point, and the stationary point satisfies the following constraints:

[0076] in, For stationary point constraint functions, These are the temporary student model parameters after convergence. For teacher model parameters, For the gradient operator with respect to the parameters of the student model, Let be the distillation loss function. This is a condition for a stationary point; Based on the implicit function theorem, the Jacobian matrix is ​​determined by differentiating the constraint equations with respect to the teacher model parameters:

[0077]

[0078] in, The Hessian matrix represents the distillation loss. The notation for the Hessian matrix is ​​as follows: The Jacobian matrix of the optimal student parameters with respect to the teacher parameters. Let be the mixing second-order partial derivative matrix of distillation loss. The symbol for the mixed partial derivative matrix; The teacher model is updated to simultaneously minimize both the out-of-distribution error of the student model and its own true label error:

[0079]

[0080] in, For teacher model parameters, For the teacher model learning rate, For the elementary gradient, For balance coefficient, The gradient is the loss value for the true label. For the loss of true labels, The gradient of the meta-loss with respect to the temporary student model parameters, It is the inverse of the Hessian matrix. The mixed second-order partial derivative moment is the loss due to distillation.

[0081] Understandably, the performance of the temporary student model parameters is evaluated on the out-of-distribution validation set representing the distribution offset, and a meta-loss is constructed. The gradient of the meta-loss with respect to the teacher model parameters is calculated using the chain rule. Based on the implicit function theorem, it is assumed that the temporary student model parameters converge to a stationary point. The Jacobian matrix is ​​determined by taking the derivative of the constraint equation with respect to the teacher model parameters using the Hessian matrix, thereby avoiding the unfolding of the complete training trajectory and reducing computational complexity. Finally, the teacher model parameters are updated by combining the meta-gradient and the gradient of the true label loss, so that it simultaneously minimizes the out-of-distribution error of the student model and its own true label error.

[0082] Step S24: Repeat the inner loop optimization and outer loop correction until the meta-loss function converges to obtain a robust target student model.

[0083] It should be understood that by repeatedly performing the coordinated operation of inner loop optimization and outer loop correction, the model parameters are continuously adjusted until the meta-loss function reaches a stable convergence state, indicating that the model has the ability to cope with distribution shifts, thus ending the training process and obtaining a robust target student model, providing the final model foundation for subsequent practical prediction applications.

[0084] Furthermore, step S24 specifically includes the following steps: Repeat the iterative process of inner loop optimization and outer loop correction. In each iteration, the combined loss is calculated using a subset of the source domain training data to update the student model parameters, and the meta-gradient is calculated using the out-of-distribution validation set to correct the teacher model parameters. Continuously monitor the numerical changes of the meta-loss function until the meta-loss function converges to a stable state; After training, the final updated student model parameters are saved as a robust target student model. The formula for updating the student model parameters at the end of a single iteration is expressed as:

[0085] in, For student model parameters, For the student model learning rate, For the gradient operator with respect to the parameters of the student model, For the combined loss function, A subset of the source domain data. These are the parameters for the teacher model.

[0086] It should be noted that by repeatedly executing the coordinated process of inner loop optimization and outer loop correction, the combined loss is calculated using a subset of the source domain training data to update the student model parameters, and the meta-gradient is calculated using the out-of-distribution validation set to correct the teacher model parameters. At the same time, the numerical change of the meta-loss function is continuously monitored until it converges to a stable state, indicating that the model training is complete. Finally, the updated student model parameters are saved as a robust target student model, where the update of the student model parameters follows a descent formula based on the combined loss gradient.

[0087] Accordingly, step S30 specifically includes the following steps: Receive real-time traffic flow data based on the target student model; The real-time traffic flow data is input into the student model as a node input sequence. The dynamic attention score, based on the input content rather than the time step index, is calculated using a lightweight scoring function parameterized by a learnable vector and a projection matrix, as follows:

[0088]

[0089]

[0090]

[0091] in, For dynamic attention scores, For learnable vectors, It is the transpose symbol. The hyperbolic tangent activation function is used. For the projection matrix, Input features for the nodes, For bias terms, Input a sequence for the node. For node indexing, The time step index represents the time step from the 1st time step to the Tth time step. For a single time step feature vector, Let be a real matrix space. For sequence length, For feature dimensions; Attention weights are obtained by softmax normalization along the temporal dimension based on the dynamic attention scores:

[0092] in, Attention weights It is an exponential function. For dynamic attention scores, For summation index variables, For sequence length, For the first Dynamic attention score at each time step; The robust node representation calculated using the attention weights is a weighted aggregation result:

[0093] in, For robust node representation, Attention weights Input features for the nodes, For node indexing, For sequence length, For time step index; Introducing a learnable spatial context encoder to generate spatial location codes

[0094]

[0095] in, Encoding spatial location, For a learnable spatial context encoder, For node indexing, for 3D real vector space, The total number of nodes. For location encoding dimension, It is a real matrix space; The robust node representation is fused with the spatial location encoding and adjacency encoding to obtain the comprehensive node state:

[0096] in, For the overall node status, For robust node representation, Encoding spatial location, For adjacency coding, Let be a real vector space. For time-series feature dimensions, For location encoding dimension, For adjacency encoding dimensions; A lightweight multilayer perceptron with weights shared across all nodes is used as a function fitter to perform forward inference, generating predictions of future traffic conditions using the following formula:

[0097] in, Predict the output for the student model. To share multilayer perceptron functions, For the overall node status, This is the first layer weight matrix. This is the weight matrix for the second layer. For activation function, This is the first layer bias vector. This is the bias vector for the second layer.

[0098] It should be noted that real-time traffic flow data is used as the node input sequence. A lightweight scoring function parameterized by learnable vectors and projection matrices is used to calculate dynamic attention scores based on the input content. After softmax normalization, attention weights are obtained and weighted aggregation is performed to obtain robust node representations. Subsequently, a learnable spatial context encoder is introduced to generate spatial location codes. The robust node representations, spatial location codes, and adjacency codes are fused to obtain the comprehensive node state. Finally, a lightweight multilayer perceptron with weights shared across all nodes is used to perform forward inference to generate predictions of future traffic conditions.

[0099] In practice, traditional knowledge distillation methods assume that the static teacher model is the true benchmark; however, teacher models trained on historical data often overfit the source domain bias, leading to negative transfer when student models are deployed in out-of-distribution scenarios.

[0100] To address the aforementioned issues, this embodiment proposes MetaDistill. This MLP framework aims to filter out domain-specific noise and distill spatiotemporally invariant features into a lightweight model. The framework includes the following modules: 1. Meta-correction mechanism based on bi-layer optimization: Treat the distillation process as a dynamic game and use out-of-distribution feedback to correct the teacher model bias; 2. Robust temporal pooling encoder: While extracting key temporal events, it ignores jitter interference caused by concept drift; 3. Spatial Decoupling Encoder: Separates and models static node attributes from dynamic traffic patterns; 4. Shared Invariant MLP: A lightweight shared module used to approximate general traffic physics laws.

[0101] Standard knowledge distillation achieves learning by minimizing the divergence between the distributions of the student model and the teacher model; however, if the teacher model... The bias towards the training distribution (source domain, such as weekdays) forces student models to inherit these biases, resulting in a performance degradation on the test distribution (target domain, such as weekends).

[0102] Distillation is remodeled as a bilevel optimization problem to achieve cross-domain correction; the core idea is to evaluate the generalization ability of the student model on an independent out-of-distribution test set Quiz and use the feedback to update the teacher model.

[0103] 1) Internal circulation In the inner loop, the temporary student model Small batches of source domain data (from (Sampling) mimics the current teacher model:

[0104] Approximate solution using gradient descent steps:

[0105] At this stage, It may overfit source domain-specific traffic patterns.

[0106] 2) External circulation (distributed external correction) Subsequently, in the Quiz set representing the distribution offset (Assessment on the transition date) If the teacher provides "robust" knowledge, then exist The above should perform well; otherwise, there will be no loss. It will be very high; the loss is significant for... The gradient is the correction signal:

[0107] For efficient calculation Without needing to unfold the complete training trajectory, this embodiment employs the Implicit Function Theorem (IFT); assuming... converge to a stationary point :

[0108] Regarding the above formula Differentiation yields:

[0109] Therefore, the Jacobian matrix is... The derivation of the meta-gradient is as follows:

[0110] 3) Scalable approximation This embodiment uses the conjugate gradient method to approximate the Hessian inverse product. Only the Hessian vector product needs to be calculated.

[0111] in, For Hessian matrix, For vectors, The gradient of the loss function. These are temporary student model parameters. It is a scalar with minute perturbations. This is the loss function.

[0112] 4) Teacher correction The teacher model is updated to simultaneously minimize both the out-of-distribution error of the student model and its own true label error:

[0113] in, It acts as a regularization term, removing features that are only effective for the source domain but harmful to the target domain.

[0114]

[0115] Thus, robust meta-distillation has been completed in one iteration.

[0116] Standard sequence models (such as RNNs and Transformers) rely heavily on explicit positional encodings (PEs) to strictly preserve the temporal order of the input.

[0117] These methods work well on stationary data, but this embodiment argues that when the distribution shifts, the hard-coded temporal position becomes a weakness; for example, the shift of traffic peaks from weekdays (t=8) to weekends (t=10) is a kind of temporal jitter, and models that rely too much on fixed time indices (t) will be unable to capture such shifted patterns.

[0118] To address this, this embodiment proposes a content-aware temporal pooling mechanism that actively discards explicit temporal coding; the student model no longer learns "when" an event occurs, but focuses on "what" the event is (such as the starting point of congestion), thereby achieving invariance to temporal shifts.

[0119] For the node input sequence Dynamic attention scores are calculated based on input content rather than time step indices. A lightweight scoring function is used, derived from learnable vectors. With projection matrix Parameterization:

[0120] Attention weight We obtain the following by normalizing along the time dimension using softmax:

[0121] Ultimately, robust node representation For the weighted aggregation result:

[0122] A common problem with pooling methods is the loss of sequence order information.

[0123] In this framework, this problem is circumvented through knowledge distillation; although the student model does not use explicit positional encoding, the teacher model (such as D2STGNN) has sophisticated capabilities for modeling temporal dynamics and causal relationships.

[0124] During the distillation process, the teacher model provides supervision over the sequence results.

[0125] To minimize distillation losses, the student model automatically adjusts its attention mechanism to highlight causally relevant time steps (such as congestion precursors) without regard to their absolute position.

[0126] This enables the student model to learn offset-invariant dynamic patterns: regardless of whether congestion occurs at 8 a.m. or 10 a.m., the model can identify its pattern, significantly improving its out-of-distribution generalization ability.

[0127] A major challenge in robust traffic prediction is spatial heterogeneity: different nodes (such as highways and residential areas) follow different distributions. Directly embedding such static differences into dynamic learners can impair generalization ability; decoupling static node identities from dynamic traffic patterns is also necessary.

[0128] Introducing a learnable spatial context encoder :

[0129] Overall node status Robust timing characteristics Spatial location coding The result of fusion:

[0130] This design enables the model to utilize specific location information. At the same time, traffic dynamics are handled using a sharing mechanism. .

[0131] To ensure that the student model learns general traffic patterns rather than overfitting to specific nodes, the model uses a lightweight shared MLP as the function fitter:

[0132] The key lies in the weight. In all Shared across nodes.

[0133] By forcing all nodes to pass through the same MLP, the model is regularized to learn a single, robust transition function applicable to the entire graph, rather than memory-based learning. Set an independent mode.

[0134] This embodiment employs the above-described scheme, using the source domain training data and the out-of-distribution validation data to perform meta-learning two-layer optimization training on the knowledge distillation architecture, constructing an iterative process including inner loop optimization and outer loop correction. In the inner loop, the teacher model parameters are fixed, the distillation loss is calculated using the source domain training data, and the student model parameters are updated using gradient descent to minimize the divergence between the student model output and the teacher model output, obtaining temporary student model parameters. In the outer loop, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function, and the gradient with respect to the teacher model parameters is calculated based on the meta-loss function. The implicit function theorem is used to avoid unfolding the complete training trajectory, and the teacher model parameters are back-corrected to remove domain-specific noise. The inner loop optimization and outer loop are repeatedly executed. The process continues until the meta-loss function converges, resulting in a robust target student model. This model performs a two-layer meta-learning optimization training on the knowledge distillation architecture using source domain training data and out-of-distribution validation data. It constructs an iterative process of inner-loop optimization and outer-loop correction. In the inner loop, the student model effectively imitates the teacher model's knowledge. In the outer loop, the generalization performance is evaluated based on out-of-distribution validation data, and the teacher model parameters are corrected in reverse. The implicit function theorem is used to avoid unfolding the complete training trajectory, thus reducing computational complexity. By iterating repeatedly until the meta-loss function converges, domain-specific noise that is only effective in the source domain but harmful to the target domain is effectively eliminated. This prompts the teacher model to transmit robust knowledge with invariance, thereby obtaining a robust target student model. This significantly improves the model's generalization ability and prediction accuracy in non-stationary distribution shift scenarios.

[0135] Accordingly, the present invention further provides a traffic robust knowledge distillation training prediction device.

[0136] Reference Figure 5 , Figure 5 This is a functional block diagram of the first embodiment of the traffic robust knowledge distillation training prediction device of the present invention.

[0137] In a first embodiment of the traffic robust knowledge distillation training and prediction device of the present invention, the traffic robust knowledge distillation training and prediction device includes: Architecture building module 10 is used to acquire historical traffic flow observation data, divide the historical traffic flow observation data into source domain training data and out-of-distribution validation data, and construct a knowledge distillation architecture containing teacher model and student model to provide data foundation and network structure for model optimization.

[0138] The two-layer optimization training module 20 is used to perform meta-learning two-layer optimization training on the knowledge distillation architecture based on the source domain training data and the out-of-distribution validation data. In the inner loop, the source domain training data is used to guide the student model to imitate the teacher model. In the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model until convergence is obtained to obtain a robust target student model.

[0139] The inference and prediction module 30 is used to receive real-time traffic flow data based on the target student model, perform forward inference, and output the prediction results of future traffic conditions.

[0140] The steps for implementing each functional module of the traffic robust knowledge distillation training and prediction device can be referred to in the various embodiments of the traffic robust knowledge distillation training and prediction method of the present invention, and will not be repeated here.

[0141] Furthermore, this embodiment of the invention also proposes a storage medium storing a traffic robust knowledge distillation training and prediction program, which, when executed by a processor, performs the operations described in the above-described traffic robust knowledge distillation training and prediction method embodiment.

[0142] Those skilled in the art will understand that all or part of the steps in the methods described above can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium is a computer-readable storage medium, including: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program code.

[0143] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0144] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0145] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A robust knowledge distillation training prediction method for traffic, characterized in that, The traffic robust knowledge distillation training prediction method includes: Historical traffic flow observation data is acquired, and the historical traffic flow observation data is divided into source domain training data and out-of-distribution validation data. A knowledge distillation architecture containing teacher model and student model is constructed to provide a data foundation and network structure for model optimization. Based on the source domain training data and the out-of-distribution validation data, the knowledge distillation architecture is subjected to meta-learning two-layer optimization training. In the inner loop, the source domain training data is used to guide the student model to imitate the teacher model. In the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model until convergence is obtained to obtain a robust target student model. The target student model receives real-time traffic flow data and performs forward inference to output a prediction of future traffic conditions.

2. The traffic robust knowledge distillation training prediction method as described in claim 1, characterized in that, The process of acquiring historical traffic flow observation data involves dividing this data into source domain training data and out-of-distribution validation data, and constructing a knowledge distillation architecture that includes teacher and student models. This provides a data foundation and network structure for model optimization, including: Obtain the raw dataset containing historical traffic flow observation sequences and road network structure data; The original dataset is divided into source domain training data that follows the independent and identically distributed hypothesis and out-of-distribution validation data that simulates concept drift scenarios; Simultaneously, a knowledge distillation architecture containing a teacher model and a student model is initialized. The student model incorporates a content-aware temporal pooling encoder and a spatial decoupling encoder. The content-aware temporal pooling encoder is used to extract traffic dynamic features from the time dimension and weaken the influence of time and location offset. The spatial decoupling encoder is used to separate the static attributes of nodes from the traffic dynamics from the spatial dimension, realizing spatial heterogeneity modeling. Through the content-aware temporal pooling encoder and the spatial decoupling encoder, a network structure support is provided for the two-layer optimized distillation training based on the meta-learning framework.

3. The traffic robust knowledge distillation training prediction method as described in claim 1, characterized in that, The step of performing meta-learning two-layer optimization training on the knowledge distillation architecture based on the source domain training data and the out-of-distribution validation data includes: guiding the student model to mimic the teacher model using the source domain training data in the inner loop, and evaluating the generalization performance of the student model and back-correcting the teacher model parameters using the out-of-distribution validation data in the outer loop, until convergence to obtain a robust target student model. Based on the source domain training data and the distributed out-of-distribution validation data, the knowledge distillation architecture is subjected to meta-learning two-layer optimization training to construct an iterative process that includes inner loop optimization and outer loop correction. In the inner loop, the teacher model parameters are fixed, the distillation loss is calculated using the source domain training data, and the student model parameters are updated by gradient descent to minimize the divergence between the student model output and the teacher model output, thus obtaining temporary student model parameters. In the outer loop, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function. The gradient with respect to the teacher model parameters is calculated based on the meta-loss function. The implicit function theorem is used to avoid unfolding the complete training trajectory. The teacher model parameters are then corrected in reverse to remove domain-specific noise. Repeat the inner loop optimization and outer loop correction until the meta-loss function converges to obtain a robust target student model.

4. The traffic robust knowledge distillation training prediction method as described in claim 3, characterized in that, In the inner loop, the teacher model parameters are fixed, the distillation loss is calculated using the source domain training data, and the student model parameters are updated using gradient descent to minimize the divergence between the student model output and the teacher model output, resulting in temporary student model parameters, including: In the inner loop, a subset of source domain data is sampled from the source domain training data; With the teacher model parameters fixed, a temporary student model is constructed on the source domain data subset using the following formula to mimic the objective function of the current teacher model: in, These are temporary student model parameters. For student model parameters, The loss function for the inner loop training is... A subset of the source domain data. For teacher model parameters; The parameters of the temporary student model are approximated using gradient descent, and the update formula is: in, These are temporary student model parameters. For student model parameters, For learning rate, For gradient operators, For balance coefficient, For the loss of true labels, A subset of the source domain data. For teacher model parameters, This is due to knowledge distillation loss.

5. The traffic robust knowledge distillation training prediction method as described in claim 3, characterized in that, In the outer loop, the temporary student model parameters are applied to the out-of-distribution validation data to construct a meta-loss function. Based on this meta-loss function, the gradient with respect to the teacher model parameters is calculated. The implicit function theorem is used to avoid unfolding the complete training trajectory, and the teacher model parameters are back-corrected to remove domain-specific noise. This includes: In the outer loop, the performance of the temporary student model parameters is evaluated on the out-of-distribution validation set representing the distribution shift, and the meta-loss is calculated, which characterizes the error of the student model in the distribution shift scenario. The gradient of the meta-loss with respect to the teacher model parameters is calculated using the following formula as a correction signal: in, For the meta-loss function, For teacher model parameters, These are temporary student model parameters. For the elementary gradient, For the loss, the partial gradient with respect to the student parameters, It is a Jacobian matrix; Based on the implicit function theorem, the parameters of the temporary student model are assumed to converge to a stationary point, and the stationary point satisfies the following constraints: in, For stationary point constraint functions, These are the temporary student model parameters after convergence. For teacher model parameters, For the gradient operator with respect to the parameters of the student model, Let be the distillation loss function. This is a condition for a stationary point; Based on the implicit function theorem, the Jacobian matrix is ​​determined by differentiating the constraint equations with respect to the teacher model parameters: in, The Hessian matrix represents the distillation loss. The notation for the Hessian matrix is ​​as follows: The Jacobian matrix of the optimal student parameters with respect to the teacher parameters. Let be the mixing second-order partial derivative matrix of distillation loss. The symbol for the mixed partial derivative matrix; The teacher model is updated to simultaneously minimize both the out-of-distribution error of the student model and its own true label error: in, For teacher model parameters, For the teacher model learning rate, For the elementary gradient, For balance coefficient, The gradient is the loss value for the true label. For the loss of true labels, The gradient of the meta-loss with respect to the temporary student model parameters, It is the inverse of the Hessian matrix. The mixed second-order partial derivative moment is the loss due to distillation.

6. The traffic robust knowledge distillation training prediction method as described in claim 3, characterized in that, The process of repeatedly performing the inner loop optimization and outer loop correction until the meta-loss function converges, to obtain a robust target student model, includes: Repeat the iterative process of inner loop optimization and outer loop correction. In each iteration, the combined loss is calculated using a subset of the source domain training data to update the student model parameters, and the meta-gradient is calculated using the out-of-distribution validation set to correct the teacher model parameters. Continuously monitor the numerical changes of the meta-loss function until the meta-loss function converges to a stable state; After training, the final updated student model parameters are saved as a robust target student model. The formula for updating the student model parameters at the end of a single iteration is expressed as: in, For student model parameters, For the student model learning rate, For the gradient operator with respect to the parameters of the student model, For the combined loss function, A subset of the source domain data. These are the parameters for the teacher model.

7. The traffic robust knowledge distillation training prediction method as described in claim 1, characterized in that, The step of receiving real-time traffic flow data based on the target student model, performing forward inference, and outputting a prediction result of future traffic conditions includes: Receive real-time traffic flow data based on the target student model; The real-time traffic flow data is input into the student model as a node input sequence. The dynamic attention score, based on the input content rather than the time step index, is calculated using a lightweight scoring function parameterized by a learnable vector and a projection matrix, as follows: in, For dynamic attention scores, For learnable vectors, It is the transpose symbol. The hyperbolic tangent activation function is used. For the projection matrix, Input features for the nodes, For bias terms, Input a sequence for the node. For node indexing, The time step index represents the time step from the 1st time step to the Tth time step. For a single time step feature vector, Let be a real matrix space. For sequence length, For feature dimensions; Attention weights are obtained by softmax normalization along the temporal dimension based on the dynamic attention scores: in, Attention weights It is an exponential function. For dynamic attention scores, For summation index variables, For sequence length, For the first Dynamic attention score at each time step; The robust node representation calculated using the attention weights is a weighted aggregation result: in, For robust node representation, Attention weights Input features for the nodes, For node indexing, For sequence length, For time step index; Introducing a learnable spatial context encoder to generate spatial location codes in, Encoding spatial location, For a learnable spatial context encoder, For node indexing, for 3D real vector space, The total number of nodes. For location encoding dimension, It is a real matrix space; The robust node representation is fused with the spatial location encoding and adjacency encoding to obtain the comprehensive node state: in, For the overall node status, For robust node representation, Encoding spatial location, For adjacency coding, Let be a real vector space. For time-series feature dimensions, For location encoding dimension, For adjacency encoding dimensions; A lightweight multilayer perceptron with weights shared across all nodes is used as a function fitter to perform forward inference, generating predictions of future traffic conditions using the following formula: in, Predict the output for the student model. To share multilayer perceptron functions, For the overall node status, This is the first layer weight matrix. This is the weight matrix for the second layer. For activation function, This is the first layer bias vector. This is the bias vector for the second layer.

8. A traffic robust knowledge distillation training prediction device, characterized in that, The traffic robust knowledge distillation training prediction device includes: An architecture building module is used to acquire historical traffic flow observation data, divide the historical traffic flow observation data into source domain training data and out-of-distribution validation data, and construct a knowledge distillation architecture that includes teacher models and student models, providing a data foundation and network structure for model optimization. The two-layer optimization training module is used to perform meta-learning two-layer optimization training on the knowledge distillation architecture based on the source domain training data and the out-of-distribution validation data. In the inner loop, the source domain training data is used to guide the student model to imitate the teacher model. In the outer loop, the out-of-distribution validation data is used to evaluate the generalization performance of the student model and back-correct the parameters of the teacher model until convergence is obtained to obtain a robust target student model. The inference and prediction module is used to receive real-time traffic flow data based on the target student model, perform forward inference, and output the prediction results of future traffic conditions.

9. A traffic robust knowledge distillation training and prediction device, characterized in that, The traffic robust knowledge distillation training and prediction device includes: a memory, a processor, and a traffic robust knowledge distillation training and prediction program stored in the memory and executable on the processor, the traffic robust knowledge distillation training and prediction program being configured to implement the steps of the traffic robust knowledge distillation training and prediction method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a traffic robust knowledge distillation training and prediction program, which, when executed by a processor, implements the steps of the traffic robust knowledge distillation training and prediction method as described in any one of claims 1 to 7.

Images