A spatiotemporal large language model flight delay prediction method based on local freezing map enhancement

Abstract

The application relates to a kind of space-time large language model flight delay prediction methods based on local freezing graph enhancement, belong to flight operation prediction technical field.The application constructs airport correlation network and normalized adjacency matrix, inputs space-time large language model after airport-time series and time embedding, node embedding are fused, and graph mask constraint is introduced in attention mechanism;Meanwhile, local freezing training strategy and LoRA low rank adaptation are used, only local bottom trainable parameter and low rank increment are updated, efficient fine-tuning is realized.The application trains the model by the mask weighted combination loss function of MAE and RMSE, can accurately capture the intra-day, intra-week periodicity of flight delay, and output multi-time step airport delay prediction result.The application considers space-time correlation modeling and efficient parameter training, and significantly improves the accuracy and robustness of flight delay prediction.

Description

Technical Field

[0001] This invention relates to the field of flight operation prediction technology, specifically to a method for predicting flight delays based on a spatiotemporal large language model enhanced by local frozen graphs, belonging to the field of cross-application technology of intelligent scheduling and artificial intelligence in civil aviation transportation. Background Technology

[0002] Flight delays are a core pain point in the civil aviation operation system. They not only reduce air transport efficiency and increase operating costs, but also seriously affect passenger travel experience. Accurate flight delay prediction is an important basis for civil aviation scheduling decisions. Airport networks are typical spatiotemporally complex systems. The generation and propagation of flight delays are affected by multiple factors, including the topological relationships between airports, time periodicity, and dynamic changes in flight traffic. Therefore, delay prediction requires accurate modeling of the complex spatiotemporal dependencies.

[0003] Existing flight delay prediction methods often fall into several categories. Traditional time-series models can only capture the temporal patterns of a single airport, failing to model the spatial relationships between airports. While shallow graph networks can model airport topological relationships, their ability to capture long-term periodic patterns is insufficient. Prediction methods based on spatiotemporal large language models, although considering spatiotemporal feature modeling, suffer from large model parameters, high fine-tuning training costs, and the lack of graph structure constraints on attention distribution leads to low matching between attention weight allocation and actual airport relationships, limiting prediction accuracy and generalization ability. Furthermore, existing methods lack targeted loss function design when dealing with missing flight data, making them prone to model training bias due to missing samples. Some models use full parameter training, failing to utilize the general features of pre-trained models, resulting in wasted computational resources and a high risk of overfitting.

[0004] To address the aforementioned issues, there is an urgent need for a flight delay prediction method that balances accurate spatiotemporal correlation modeling, efficient parameter training, and robust handling of missing data. This method enhances the attention mechanism of the spatiotemporal large language model through graph structure and designs a parameter training strategy with local freezing, thereby reducing training costs while improving prediction accuracy and robustness. Summary of the Invention

[0005] To address the shortcomings of existing flight delay prediction models, such as difficulty in effectively modeling complex spatial relationships between airports, inability to properly handle missing data, and low training efficiency, this invention proposes a flight delay prediction method based on a spatiotemporal large language model enhanced by local frozen graphs.

[0006] This method constructs a normalized adjacency matrix of the airport association network as the attention mechanism for a graph mask-constrained spatiotemporal large language model, achieving efficient modeling of spatial associations between airports. It designs a locally frozen Transformer encoder training strategy and combines it with LoRA low-rank adaptation to achieve efficient parameter fine-tuning. At the same time, it adopts a mask-weighted MAE and RMSE combined loss function to take into account both data missingness handling and the capture of intraday and intraweekly periodic patterns of flight delays, ultimately improving the accuracy, robustness and training efficiency of flight delay prediction.

[0007] To achieve the above-mentioned objectives, this invention provides a method for predicting flight delays based on a spatiotemporal large language model enhanced by local frozen graphs.

[0008] The spatiotemporal large language model includes a temporal embedding module, a node embedding module, a graph mask attention module, a Transformer encoder network, and a delay prediction module. The specific steps are as follows: S1. Obtain the flight delay prediction dataset and airport basic information. The prediction dataset includes at least the airports where the flights take off and arrive, the scheduled time, and the actual delay value. The airport basic information includes the airport number, the route take-off and landing relationship, and the flight frequency data.

[0009] S2, preprocess the predicted dataset: divide the time window according to the preset time granularity, remove airport nodes with a sample number lower than the preset threshold, and aggregate the target delay data and input features according to the airport-time dimension to form a standardized airport-time series.

[0010] S3. Based on the route takeoff and landing correlation and flight frequency data in the airport basic information, construct the airport association network and generate the corresponding normalized adjacency matrix. Extend the normalized adjacency matrix to the multi-head attention dimension to generate a graph mask tensor.

[0011] S4. The airport-time series is constructed as a model input sequence and an output sequence. Time features are directly calculated from flight times to capture intraday and intraweekly periodic patterns of flight delays. At the same time, a target observation mask is generated to filter out missing samples.

[0012] S5, after channel mapping, the input sequence is fused with the temporal embedding generated by the temporal embedding module and the trainable node embedding generated by the node embedding module. The fused features are then input into the Transformer encoder network of the spatiotemporal large language model. In the attention mechanism of the graph mask attention module, the normalized adjacency matrix is ​​introduced as a graph mask to correct the attention weight distribution, and the local low-level parameters of the encoder are frozen during model training.

[0013] S6 inputs the output features of the Transformer encoder network into the delay prediction module, and outputs the airport delay prediction results for multiple future time steps through a fully connected layer. During model training, LoRA low-rank adaptation is used to efficiently fine-tune the Transformer encoder network, and a mask-weighted MAE and RMSE combined loss function is designed to optimize the model parameters.

[0014] Furthermore, the time features mentioned in step S4 include normalized intraday time and weekday index, and the formula for calculating normalized intraday time is:

[0015] in, , Time buckets The start time in hours and minutes. Indicates the normalized time position within a day; weekday index These correspond to the seven days of the week and are used to reflect the weekday cyclical pattern of flight delays.

[0016] Furthermore, the temporal embedding described in step S5 is composed of trainable temporal representations, and its calculation formula is as follows:

[0017] in, , These are intraday time embedding tables and intraweek time embedding tables, both of which are trainable parameters. This represents the total number of time steps within the day. For the embedded dimension; This is the index corresponding to the normalized intraday time. The time is an integer index corresponding to the weekday index; the time embedding is broadcast to each airport node during batch training to realize the vectorized representation of time features.

[0018] Furthermore, the construction and normalization of the normalized adjacency matrix in step S3 satisfy:

[0019] in For the airport arrive One-way flight frequency, For the airport arrive One-way flight frequency, This represents the maximum sum of the frequencies of two-way flights for all airports. This indicates the airport's own self-connectivity.

[0020] Furthermore, the graph enhancement attention calculation method of the graph mask attention module in step S5 is as follows:

[0021] in, These are query tensors, key tensors, and value tensors, respectively. For the dimension of attention head, To serve as a scaling factor to avoid excessively large inner product values, The graph mask tensor generated in step S3 is used to constrain attention weights to be distributed only among adjacent nodes in the airport association network.

[0022] Furthermore, the specific process of feature fusion described in step S5 satisfies:

[0023]

[0024]

[0025] in This is a two-dimensional tensor representing the historical window features expanded along the time and channel dimensions. For time embedding, For trainable node embedding, express Convolution mapping, where concat represents channel concatenation. The input representation for the converter; The tokens are then activated by LeakyReLU and rearranged into a token sequence for input into GPT.

[0026] in, This is a two-dimensional tensor representing the historical window features expanded along the time and channel dimensions. Represents a 1×1 convolution mapping. , These are the convolution weight matrices for the channel mapping, For time embedding, For trainable node embedding, This indicates a concatenation operation along the channel dimension; After being processed by the LeakyReLU activation function, the tokens are rearranged into a sequence and input into the Transformer encoder network.

[0027] Furthermore, the Transformer encoder network employs LoRA low-rank adaptation for efficient fine-tuning, and its weight update formula is as follows:

[0028] in , r is a low-rank number; and The trainable parameter matrix and the original weight matrix Keep frozen, only update the low-rank increment matrix. The LoRA low-rank adaptation applies to the attention-related weighting modules in the transformer.

[0029] in, This is the original weight matrix of the attention-related modules in the Transformer encoder network. , The trainable parameter matrix for low-rank fitting, where r is the low-rank order and , The low-rank increment matrix; the original weight matrix Keep frozen, only update the low-rank increment matrix. This reduces the number of parameters required for model training.

[0030] Furthermore, freezing the local low-level parameters of the encoder in step S5 specifically involves: Divide all layers of the Transformer encoder network into the first... Layers and back layer, This represents the total number of layers in GPT-2. This is the preset number of thawing layers.

[0031] Forward In this layer, all parameters of the self-attention and feedforward networks are frozen, and only the layer normalization (LN) parameters and the location embedding (WPE) parameters are retained for training.

[0032] After The layer unfreezes the self-attention parameters and introduces the graph-enhanced attention with LoRA low-rank adaptation, while the feedforward network (MLP) parameters remain frozen.

[0033] Trainable parameter set In summary:

[0034] in, For the encoder's layer index, Indicates the normalized parameters of the layer. Indicates the position embedding parameter, This represents the parameters of the feedforward network.

[0035] Furthermore, the loss function for model training employs a masked weighted average absolute error. Weighted root mean square error of mask The weighted combination loss function formula is:

[0036] in, These are weighting factors used for balancing. and The contribution of losses.

[0037] The formulas for calculating the mask-weighted average absolute error and the mask-weighted root mean square error are as follows:

[0038] in, The observation mask generated in step S4, Airport At time step The delay data is considered a valid sample. This indicates missing data; For the airport At time step The predicted delay value, the predicted delay versus the actual delay, This corresponds to the actual delay value.

[0039] Compared with the prior art, the present invention has the following significant advantages: This invention introduces the spatial topological association of airports into the attention mechanism of a spatiotemporal large language model by constructing a normalized adjacency matrix of the airport association network as a graph mask. It constrains the attention weights to be distributed only between adjacent airport nodes, thus solving the problem of mismatch between the attention distribution of traditional large language models and the actual association of airports, and accurately capturing the spatial propagation pattern of flight delays.

[0040] This invention designs a local freeze training strategy for Transformer encoders and combines it with LoRA low-rank adaptation. Only the local top-level self-attention parameters are unfrozen and the low-rank increment matrix is ​​updated, which greatly reduces the number of trainable parameters, reduces the computational and time costs of model training, and at the same time preserves the general spatiotemporal features of the pre-trained model and improves the model's generalization ability.

[0041] This invention extracts intraday and intraweekly periodic time features from flight times and achieves vectorized representation of these features through trainable time embedding, accurately capturing the periodic patterns of flight delays and adapting to the time characteristics of civil aviation operations.

[0042] This invention employs a mask-weighted combined loss function of MAE and RMSE, which filters out missing samples by using a target observation mask and calculates the loss only for valid data. This solves the model training bias problem caused by missing flight data and improves the model's robustness to missing data.

[0043] The method of this invention takes into account the accurate modeling of spatial correlation and temporal periodicity between airports, as well as the efficient parameter training and robust handling of missing data, which significantly improves the accuracy and robustness of flight delay prediction. The output multi-time step delay prediction results can provide accurate data support for civil aviation scheduling decisions and are suitable for flight operation scheduling scenarios in large-scale airport networks. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0045] Figure 2 This is a structural diagram of the model of the present invention. Detailed Implementation

[0046] This invention uses a spatiotemporal large language model to predict flight delays. The model includes a time embedding module, a node embedding module, a graph mask attention module, a Transformer encoder network, and a delay prediction module. Detailed descriptions are provided below with reference to specific embodiments.

[0047] In this embodiment of the invention, a flight delay dataset from actual civil aviation operations is selected as the model training, validation, and testing data. The dataset contains flight operation information from 30 core civil aviation airports in China, spanning 12 months, and covers core information such as flight departure and arrival airports, planned times, actual departure / arrival times, and flight delay values. At the same time, basic airport information is also obtained, including unique airport numbers, the relationship between routes between airports, and daily / weekly flight frequency data for each route.

[0048] Flight arrival delay time is taken as the core prediction target of the model. The features in the prediction dataset are divided into time features, airport topology features, and flight traffic features, which serve as the basic features for model input.

[0049] Dataset preprocessing The acquired flight delay prediction dataset was standardized and preprocessed. The preset time granularity was set to 15 minutes, and the natural day was divided into 96 time buckets. The flight data was then divided into time windows according to this time granularity.

[0050] To avoid the impact of sparse nodes on model training performance, airport nodes with fewer than 1,000 valid samples per month are removed.

[0051] The delay values, flight traffic, and other characteristics of each airport are aggregated according to the airport-time two-dimensional dimension to form a standardized airport-time series with dimension . This represents the number of airports after filtering. The time window length, For feature dimensions.

[0052] All numerical features are Z-score standardized to eliminate the impact of differences in feature dimensions on the model. The standardization formula is as follows:

[0053] in, For the original value of the feature, This is the mean of the feature on the training set. denoted as the standard deviation of this feature on the training set.

[0054] Airport Association Network and Normalized Adjacency Matrix Construction: Based on the route takeoff and landing relationships and flight frequency data in the airport's basic information, an airport association network is constructed: Using airports as network nodes, if there are two-way flight routes between two airports, an edge connection is established between the corresponding nodes, and the weight of the edge is the sum of the two-way flight frequencies between the two airports.

[0055] Based on the constructed airport association network, the corresponding adjacency matrix is ​​generated and normalized. The calculation of the normalized adjacency matrix satisfies:

[0056] in, For the airport Arrive at the airport One-way flight frequency, For the airport Arrive at the airport One-way flight frequency, To find the maximum sum of two-way flight frequencies for all airports, map the flight frequencies to... Range, and set at the same time This indicates the airport's own self-connectivity.

[0057] In this embodiment of the invention, the number of attention heads in the Transformer encoder network is set to 8. The normalized adjacency matrix is ​​expanded along the attention head dimension, copied 8 times, and then concatenated to generate a dimension of... Graph mask tensor This adapts to the dimensional requirements of multi-head attention mechanisms, providing a foundation for the subsequent implementation of graph mask attention mechanisms.

[0058] The generation of time features and target observation masks constructs the standardized airport-time series as the input and output sequences of the model. Time features are directly calculated from flight schedule times to accurately capture intraday and intraweekly periodic patterns of flight delays.

[0059] The time features include normalized intraday time and weekday index, where the formula for calculating normalized intraday time is:

[0060] , Time buckets The start time in hours and minutes. Indicates the normalized time position within a day. Corresponding to 0:00, Corresponding to 11:45 PM; Weekday Index These numbers correspond to the seven days of the week, with 0 representing Sunday, 1 representing Monday, and so on up to 6 representing Saturday, reflecting the cyclical pattern of flight delays within the week.

[0061] Simultaneously, to address the issue of missing values ​​in flight data, a target observation mask is generated. To filter out missing samples: Iterate through the airport-time series delay data, if a certain airport At time step The delay data must be non-empty and a valid value; set .

[0062] If data is missing, abnormal, or invalid, set This provides a mask basis for the subsequent calculation of the mask-weighted loss function.

[0063] The generation of temporal and node embeddings: This invention uses a temporal embedding module and a node embedding module to realize the vectorized representation of temporal features and airport node features, respectively. The temporal embedding is composed of trainable temporal representations, and the calculation formula is as follows:

[0064] , These are the intraday time embedding table and the intraweek time embedding table, respectively, both of which are trainable parameters of the model. In this embodiment of the invention, the total number of intraday time steps is set. (Matching the granularity of a 15-minute time bucket), Embedded Dimensions ; For the normalized intraday time, This is the integer index corresponding to the weekday index; the time embedding is broadcast during batch training of the model, from the dimension... Expand to This aligns with the airport-time series dimension.

[0065] The node embedding module generates trainable node embeddings for each airport node. The embedding dimension is also set to 128. The airport node embedding is initialized with a random normal distribution and iteratively optimized during model training to achieve a vectorized representation of the airport's inherent topological and operational features. Node embedding is also performed via broadcasting operations from the dimension... Expand to Aligned with time embedding and airport-time series completion dimensions.

[0066] Input feature fusion and Transformer encoder input construction input the airport-time series data to the channel mapping layer, achieving feature dimension unification and multi-feature fusion. The specific fusion process satisfies:

[0067]

[0068]

[0069] in, It is a two-dimensional tensor of the airport-time series after unfolding in the time and channel dimensions; Represents a 1×1 convolution mapping. , These are the convolution weight matrices for channel mapping, set in this embodiment of the invention. , ; This represents a concatenation operation along the channel dimension, which concatenates the channel-mapped airport-time series, time embedding, and node embedding into a fused feature; This is the input representation for the final converter.

[0070] Will After processing with the LeakyReLU activation function, the sequence is rearranged into a token sequence, ultimately forming a sequence with dimension [missing value]. The feature sequences are input into the Transformer encoder network of the spatiotemporal large language model; in this embodiment of the invention, the total number of layers of the Transformer encoder network is set. With 8 attention heads and 512 hidden layer dimensions, it provides a network foundation for deep modeling of spatiotemporal features.

[0071] The graph mask attention mechanism is implemented in the Transformer encoder network's attention mechanism by introducing the generated normalized adjacency matrix as a graph mask. The graph mask attention module corrects the attention weight distribution, ensuring the model focuses only on the features of adjacent nodes in the airport network, accurately capturing the spatial propagation patterns of flight delays. The calculation method for graph-enhanced attention is as follows:

[0072] in, , , These are the query tensor, key tensor, and value tensor for the attention mechanism, respectively. In this embodiment, the dimension of the attention head is considered. , This is a scaling factor used to avoid the gradient vanishing problem caused by excessively large inner product values; The graph mask tensor generated above is used to constrain the allocation of attention weights only among adjacent nodes in the airport association network, thereby achieving a deep integration of airport spatial topology association and attention mechanism.

[0073] When training the LoRA low-rank adaptation model, the partial freeze training strategy is used on the Transformer encoder network to significantly reduce the number of trainable parameters and achieve efficient parameter fine-tuning. Divide all layers of the Transformer encoder network into the first... Layers and back layer, This represents the total number of layers in the encoder. In this embodiment, the preset number of thawing layers is set. , That is, the first 8 layers are the bottom frozen layer, and the last 4 layers are the top thawed layer.

[0074] Forward All parameters of the self-attention and feedforward networks are frozen, and only the layer normalization (LN) parameters and location embedding (WPE) parameters are retained for training.

[0075] After In this layer, the attention parameters are unfrozen, and the graph-enhanced attention and LoRA low-rank adaptation are introduced, while the feedforward network (MLP) parameters remain frozen. This locally frozen training strategy corresponds to the set of trainable parameters. It can be summarized as follows:

[0076] in, For the encoder's layer index, Indicates the normalized parameters of the layer. Indicates the position embedding parameter, This represents the parameters of the feedforward network.

[0077] Meanwhile, LoRA low-rank adaptation is introduced into the thawed self-attention module to further reduce the number of parameters required for model training. The LoRA weight update formula is as follows:

[0078] in, This is the original weight matrix of the attention-related modules in the Transformer encoder network, as shown in this embodiment. ; , For a low-rank fit, a trainable parameter matrix. It is a low-rank number and satisfies In this embodiment, the following settings are provided. , , ; The low-rank increment matrix is ​​the original weight matrix during model training. Keep frozen, only update the low-rank increment matrix. This enables efficient fine-tuning of parameters and significantly reduces the consumption of computing resources.

[0079] The model training loss function and training strategy employ a mask-weighted combined MAE and RMSE loss function to effectively handle missing values ​​in flight data. Loss is calculated only for valid samples. The loss function formula is as follows:

[0080] in, These are weighting coefficients used to balance the mask weighted average absolute error (MAE). ) and mask weighted root mean square error (RMSE) The loss contribution of ) is set in this embodiment. It balances the model's fitting accuracy for small errors with its robustness to large errors.

[0081] The formulas for calculating the mask-weighted average absolute error and the mask-weighted root mean square error are as follows:

[0082]

[0083] in, For the target observation mask generated above, Airport At time step The delay data are valid samples; For the airport At time step The predicted delay value, This corresponds to the actual delay value.

[0084] The model was trained using the Adam optimizer with a learning rate of 1e-4, a batch size of 32, and 100 training epochs.

[0085] To avoid model overfitting, an early stopping strategy is adopted: the dataset is divided into training, validation and test sets in a ratio of 7:2:1. If the loss value of the validation set does not decrease for 10 consecutive rounds, the model training is stopped and the current optimal model parameters are saved.

[0086] The flight delay prediction results are input into the delay prediction module by taking the output features of the Transformer encoder network.

[0087] This module consists of two fully connected layers.

[0088] The first fully connected layer maps the 512-dimensional features output by the encoder to 128 dimensions, using the ReLU activation function to achieve a non-linear transformation of the features.

[0089] The second fully connected layer maps the 128-dimensional features to 1 dimension, directly outputting the airport delay prediction value.

[0090] In this embodiment of the invention, the model prediction time step is set to 4, that is, the airport-time series features of the past 24 hours are input, and the airport delay prediction results for the next hour (4 15-minute time buckets) are output.

[0091] The final output dimension is The prediction matrix provides 15-minute forecasts of flight delays for each airport over the next hour.

[0092] It can provide precise data support for real-time scheduling and flight adjustment decisions at civil aviation airports.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting flight delays based on a spatiotemporal large language model enhanced with local frozen graphs, characterized in that, Includes the following steps: S1, obtain flight delay prediction dataset and airport basic information. The prediction dataset includes at least the departure and arrival airports, scheduled times, and delay values, and may further include meteorological characteristics. S2, preprocess the predicted dataset: divide the time into buckets according to a preset time granularity, filter airport nodes, and aggregate the target delay and input features according to "airport-time" to form an airport-time series; S3, Based on the airport basic information and flight take-off and landing relationships, construct an airport association network and generate a normalized adjacency matrix; S4, construct the airport-time series into an input sequence and an output sequence, and generate time features and target observation masks; S5, the input sequence is fused with temporal embedding and node embedding after channel mapping, and then input into the Transformer encoder network of the spatiotemporal large language model; wherein, the normalized adjacency matrix is ​​introduced as a graph mask in the attention mechanism, and the local low-level parameters of the encoder are frozen during model training. S6 outputs airport delay prediction results for multiple future time steps through the delay prediction module.

2. The method according to claim 1, characterized in that, In step S4, the time features are directly calculated from flight times to capture the intraday and weekly periodic patterns of flight delays: Specifically, the diurnal variation pattern of traffic flow is reflected by normalizing the intraday time: In the formula , Time buckets The start time in hours and minutes. Indicates the normalized time position within a day. This indicates the weekday index, used to reflect the weekday periodicity of flight delays.

3. The method according to claim 1, characterized in that, The adjacency matrix construction and normalization described in step S3 satisfy the following: in For the airport arrive One-way flight frequency, For the airport arrive The frequency of one-way flights is used to expand the adjacency matrix to a multi-head attention dimension, generating a corresponding mask tensor. .

4. The method according to claim 1, characterized in that, In step S5, the mask is incorporated into the self-attention calculation process to correct the distribution of attention weights; the calculation method for the graph-enhanced attention is as follows: In the formula These are query tensors, key tensors, and value tensors, respectively. For the dimension of attention head, is the scale factor.

5. The method for predicting flight delays using a locally frozen graph-enhanced spatiotemporal large language model according to claim 1, characterized in that, The input fusion in step S5 satisfies: in This is the tensor of the historical window features expanded in the time and channel dimensions. For time embedding, For trainable node embedding, express Convolution mapping, where concat represents channel concatenation. The input representation for the converter; The tokens are then activated by LeakyReLU and rearranged into a token sequence for input into GPT.

6. The method for predicting flight delays using a locally frozen graph-enhanced spatiotemporal large language model according to claim 1, characterized in that, The temporal embedding described in step S5 consists of trainable temporal representations and satisfies: in , For trainable embedding tables, This represents the total number of time steps within the day. The time embedding is used to characterize intraday and intraweek periodic changes and is broadcast to each airport node during batch training.

7. The method according to claim 1, characterized in that, The converter employs LoRA low-rank adaptation to achieve efficient fine-tuning, satisfying the following: in , r is a low-rank number; and The trainable parameter matrix and the original weight matrix Keep frozen, only update the low-rank increment matrix. The LoRA low-rank adaptation is applied to the attention-related weighting module in the transformer.

8. The method according to claim 1, characterized in that, Divide all layers of the converter into front Layers and back layer, This represents the total number of layers in GPT-2. The number of layers to be thawed; for the previous In the first layer, the self-attention and feedforward networks are all frozen, and only the layer normalization parameters and position embedding parameters are retained for training; for the second layer... The layer unfreezes the self-attention parameters and introduces the graph-enhanced attention to adapt to LoRA low-rank, while the feedforward submodule parameters remain frozen; The set of trainable parameters corresponding to this training strategy can be summarized as follows: The transformer adopts a local freeze training strategy, and the set of trainable parameters satisfies: in For layer index, This represents the total number of floors. For the number of thawed layers, Indicates the normalized parameters of the layer. Indicates the position embedding parameter, This indicates the parameters of the feedforward submodule.

9. The method according to claim 1, characterized in that, The training loss function uses a weighted combination of MAE and RMSE, satisfying: in, The weighted average absolute error of the mask. The root mean square error is weighted by the mask, and both are passed through the observation mask. The error is calculated by filtering out missing samples and only calculating the error of the truly observable data, as defined below: in To observe the mask, and They represent airports At time step The predicted delay versus the actual delay.

Images