KNN-Attention-LSTM Winter Road Surface Temperature Prediction Method Based on Stacking Integration

Abstract

The present invention discloses a KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration. After processing the data of winter road surface temperature and other meteorological factors, this method leverages the advantages of the physically constrained KNN and Attention mechanisms through feature engineering and a stacking model integration strategy, improving the accuracy of winter road surface temperature prediction and the stability of prediction results under different time periods and meteorological conditions. This method can effectively capture the dynamic changes in winter road surface temperature and overcome the limitations of traditional prediction methods when dealing with complex meteorological conditions and nonlinear time series data. The present invention can be applied to fields such as road safety, traffic management, and the establishment of deicing and anti-skid measures, providing reliable support for travel information services and having broad practical application value.

Description

Technical Field

[0001] The present invention relates to the field of meteorological applications, and in particular to a KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration. Background Art

[0002] The evolution of road surface temperature (RST) is significantly correlated with traffic risk. Under critical thermodynamic equilibrium conditions in winter, the phase transition threshold of the road surface temperature directly determines the condensation pattern of water vapor on the road surface. When the temperature drops to the critical freezing point, the road friction coefficient undergoes a step-like decrease. This phase transition triggers three cascading effects: ① The adhesion of the tire-road interface decreases by more than 40%; ② The vehicle braking performance curve exhibits nonlinear distortion; and ③ Traffic flow stability parameters exceed safety thresholds. By establishing a dynamic model combining road surface temperature with other meteorological factors, high-resolution forecasts of future road surface temperatures can be achieved.

[0003] Accurate short-term prediction of road surface temperature faces many challenges. First, road surface temperature is affected by a combination of meteorological factors, including air temperature, humidity, precipitation, wind speed, and other factors, which have complex nonlinear relationships and time lags. Second, road surface temperature has significant spatiotemporal heterogeneity, with temperature variation patterns varying significantly across different road sections and time periods. Accurate short-term prediction becomes even more difficult, especially under variable meteorological conditions. Traditional winter road surface temperature prediction methods rely heavily on physical models or simple linear regression analysis, which struggle to effectively capture the dynamic changes and nonlinear characteristics of winter road surface temperature, resulting in limited prediction accuracy. In recent years, the successful application of deep learning technology in the field of time series prediction has provided new insights for winter road surface temperature prediction. In particular, the Long Short-Term Memory Network (LSTM) and its improved models have performed exceptionally well in handling complex temporal dependencies, bringing new possibilities for improving road surface temperature prediction accuracy.

[0004] However, the standard LSTM model still has many shortcomings in practical applications, including insufficient feature processing flexibility, limited ability to integrate spatiotemporal features, and untimely response to sudden meteorological events. In particular, the prediction accuracy within the critical phase transition temperature range needs to be improved. These limitations restrict the practical application of the LSTM model in road surface temperature prediction. Summary of the Invention

[0005] The purpose of the present invention is to solve the above technical problems and provide a KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: a KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration, characterized by comprising the following steps:

[0007] Step S1: Obtain historical data from a traffic weather monitoring station, including visibility, temperature, humidity, precipitation, wind speed, wind direction, and road surface temperature;

[0008] Step S2: perform quality control on the data, including time alignment, outlier processing, and missing value filling;

[0009] Step S3: Calculate the Spearman rank correlation coefficient between winter road surface temperature and other meteorological factors, and select features with strong correlation;

[0010] Step S4: Use Stacking integration technology to build a KNN-Attention-LSTM prediction model, specifically:

[0011] Step S41: Use two basic learners, KNN-LSTM and Attention-LSTM, to learn and predict the input data respectively;

[0012] Step S42: Based on step S41, Stacking constructs a meta-learner by taking the prediction results of KNN-LSTM and Attention-LSTM as feature inputs, and uses linear regression to perform weighted summation or combination of the outputs of the two basic models.

[0013] Step S43: Use the scaler_target.inverse_transform function to denormalize the prediction result of the meta-model to obtain the final prediction result;

[0014] Step S5: Input real-time meteorological data and output the road surface temperature forecast value for the next 1-72 hours.

[0015] As a preferred embodiment of the present invention, the specific method of step S41 is:

[0016] The KNN-LSTM model is used to capture the long-term dependencies of the time series, and physical boundaries are extracted through KNN. These boundaries are added as constraints to the LSTM loss function:

[0017] ;

[0018] in, is the weight of the physical constraint, which controls the influence of the physical constraint loss;

[0019] Based on the LSTM output, the Attention-LSTM model adjusts the weight of the time step through the self-attention mechanism, learns which time steps have a greater impact on the prediction results, and enhances the model's feature capture capability through the multi-head attention mechanism:

[0020] ;

[0021] Among them, Q is the query matrix (Query), K is the key matrix (Key), and V is the value matrix (Value). is the dimension of the key. By calculating the similarity between the query and the key, the attention weight is obtained, and the weighted sum is used to obtain the final output.

[0022] For each input data sample, each base model will give a predicted value:

[0023] ;

[0024] ;

[0025] in, and They are the predicted values obtained by KNN-LSTM and Attention-LSTM models respectively.

[0026] As a preferred embodiment of the present invention, the specific method of step S42 is:

[0027] The two model prediction results of the first layer are used as input features through the meta-learner:

[0028] ;

[0029] Then, a linear regression meta-model is used to learn how to combine the predictions of these two base models:

[0030] ;

[0031] in, , , are the parameters obtained through linear regression training.

[0032] As a preferred embodiment of the present invention, the above steps also include: step S6, the system regularly evaluates the prediction performance of the model and calculates the mean absolute error (MAE) and root mean square error (RMSE) of the prediction results; when the MAE exceeds a preset threshold or the prediction accuracy decreases, the system will automatically trigger an update mechanism and use the latest collected traffic and meteorological historical data to re-select features and re-train the model.

[0033] As a preferred embodiment of the present invention, in step S6 described in 1, during each update process, the system analyzes the contribution of features to the winter road surface temperature prediction based on the Spearman rank correlation coefficient, automatically selects meteorological factors with greater contributions, eliminates features with poor prediction effects, retrains and optimizes the KNN-Attention-LSTM model and Stacking integration strategy. After the model update is completed, the system obtains minute-by-minute measured data in real time for rolling prediction.

[0034] As a preferred embodiment of the present invention, the method for performing quality control on the data in step S2 includes:

[0035] Outlier detection: Descriptive statistical analysis and box-and-whisker plots are used to identify abnormal data points, reasonable thresholds for meteorological elements are set, and data outside the range is marked as abnormal;

[0036] Outlier processing: Single-point outliers are interpolated using the mean of the previous and next moments; continuous outliers are retained and marked if they conform to meteorological laws; if they are obviously wrong, they are corrected using the sliding window mean method;

[0037] Missing value filling: For continuous missing values, linear interpolation is used. For non-continuous missing values, forward filling and backward filling are combined and the average of the two is taken.

[0038] Time alignment: Check whether the timestamps are arranged according to the set time interval. If there is a slight deviation, align them to the nearest standard time point;

[0039] Data consistency assurance: Unify units to ensure that all meteorological elements use standard units.

[0040] In summary, this application includes at least one of the following beneficial technical effects:

[0041] The present invention provides a method for predicting winter road surface temperature on highways. The method utilizes data such as visibility, air temperature, humidity, precipitation, wind direction, wind speed, and road surface temperature from highway traffic meteorological monitoring stations, as well as constructed sliding window optimization, physical constraints, Attention mechanism, and LSTM, to accurately predict winter road surface temperature and provide timely and accurate prediction results. This method provides timely and accurate prediction results for traffic safety management, deicing and anti-skid measures, and traffic command departments through innovative model design and optimized data processing strategies, significantly improving prediction performance and system practicality. The present invention integrates and optimizes multiple deep learning models, dynamically adjusts the weights of key meteorological elements through the Attention mechanism, and solves the problem of rigid feature processing of traditional LSTM; combines the KNN-LSTM hybrid architecture to synchronously capture spatiotemporal features, and overcomes the defect of insufficient spatiotemporal modeling capabilities of a single LSTM; introduces physical constraints to significantly improve the prediction accuracy of the critical zone of phase change, and significantly improves prediction accuracy and stability.

[0042] In terms of data processing and method optimization, this paper adopts a systematic strategy to improve forecast accuracy and computational efficiency. Feature selection using the Spearman rank correlation coefficient eliminates redundant variables and streamlines input data. Furthermore, the application of time series cross-validation and a rolling window strategy ensures the model's generalizability across time series, significantly reducing response time to sudden meteorological events.

[0043] This invention boasts high portability, allowing for fine-tuning based on meteorological data from different sites. It also supports the addition of new data to optimize model parameters, providing rolling forecasts of winter road surface temperatures for the next 1 to 72 hours to meet operational needs. By accurately predicting winter road surface temperatures, this invention provides a predictive data foundation for highway meteorological support and decision-making management. This allows traffic control departments to promptly identify road icing risks, optimize resource allocation, and reduce traffic accidents, significantly reducing maintenance costs, improving traffic quality, and enhancing operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following is combined with Figure 1 The present invention is described in further detail.

[0046] Reference Figure 1 This embodiment discloses a KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration, including the following steps:

[0047] Step S1: Acquisition of historical traffic and weather data:

[0048] The observation data of the M9393 traffic meteorological station west of the Longhai Railway Bridge are used. The data are stored in the database with an interval of 5 minutes. The data include: visibility, temperature, humidity, precipitation, wind speed, wind direction and road surface temperature.

[0049] Step S2: Perform data quality control on the acquired data, specifically:

[0050] Step 2.1 Data quality check

[0051] Step 2.1.1 Outlier Detection: Use descriptive statistics (mean, standard deviation, quantiles) and box-and-whisker plots to identify anomalous data points. Set reasonable thresholds for meteorological elements (e.g., temperature range [-30°C to 50°C], wind speed range [0 to 30 m / s]), and mark data outside these ranges as outliers. Receive special treatment for consecutive outliers (e.g., three or more consecutive 5-minute data points exceeding the threshold) to avoid misidentification of extreme weather events.

[0052] Step 2.1.2 Missing Value Identification: Detect missing values in the data (e.g., -999, NaN, NULL, and other invalid identifiers). Distinguish between short-term missing values (single point or a small number of discontinuous missing values) and long-term missing values (missing values at multiple consecutive time points) so that different imputation strategies can be used.

[0053] Step 2.2 Data quality control

[0054] Step 2.2.1 Time alignment: Check whether the timestamps are strictly arranged according to the set time interval (such as 5 minutes). If there is a slight deviation (such as 5 minutes and 10 seconds), align to the nearest standard time point.

[0055] Step 2.2.2 Outlier processing: Single point outlier: Use the mean interpolation of the previous and next moments (i.e. replace the mean of the previous moment with the mean of the next moment).

[0056] Continuous outliers: If they are consistent with meteorological laws (such as extreme weather), they are retained and marked; if they are obviously wrong, they are corrected using the sliding window average method.

[0057] Step 2.2.3: Fill missing values. Specifically, for continuous missing values (short-term missing values), use linear interpolation (based on fitting of previous and subsequent valid data). For non-continuous missing values (sporadic points), combine forward filling (FFill) and backward filling (BFill), taking the average of the two (if both are valid). For long-term missing values (≥3 points), use regression estimation using correlations between meteorological elements (e.g., the relationship between temperature and humidity, wind speed), or refer to data from neighboring stations for filling.

[0058] Step 2.3: Ensure data consistency: Standardize units, ensuring all meteorological elements use standard units (e.g., temperature °C, wind speed m / s). Eliminate invalid data, directly remove irreparable long-term missing data or outliers, and log them. Correct data distribution: For data that clearly does not conform to meteorological patterns (e.g., -20°C in summer), adjust based on historical data for the same period.

[0059] Step S3: Extracting meteorological features related to road surface temperature:

[0060] The Spearman rank correlation coefficient was used to analyze the correlation between meteorological characteristics and winter road surface temperature. Features with low correlation with the target variable (winter road surface temperature) were removed to improve model prediction accuracy. Ultimately, the following features were selected as model inputs: air temperature, humidity, precipitation, wind speed, wind direction, and road surface temperature.

[0061] Step S4: KNN-Attention-LSTM integrated model construction: Based on the selected features, the KNN-Attention-LSTM prediction model is constructed using the Stacking integration method:

[0062] First layer: Use two basic learners, KNN-LSTM and Attention-LSTM, to learn and predict the input data respectively.

[0063] The KNN-LSTM model combines the KNN (K-Nearest Neighbors) algorithm with the LSTM model to capture long-term dependencies in time series. It also extracts physical boundaries (constraints) through KNN and adds these boundaries as constraints to the LSTM loss function:

[0064] ;

[0065] in, is the weight of the physical constraint, which controls the influence of the physical constraint loss.

[0066] Based on the LSTM output, the Attention-LSTM model adjusts the weight of the time step through the self-attention mechanism, learns which time steps have a greater impact on the prediction results, and enhances the model's feature capture capability through the multi-head attention mechanism:

[0067] ;

[0068] Among them, Q is the query matrix (Query), K is the key matrix (Key), and V is the value matrix (Value). is the dimension of the key. By calculating the similarity between the query and the key, the attention weight is obtained, and the weighted sum is used to obtain the final output.

[0069] For each input data sample, each base model in the first layer will give a prediction value:

[0070] ;

[0071] ;

[0072] in, and They are the predicted values obtained by KNN-LSTM and Attention-LSTM models respectively.

[0073] Second layer: Based on the first layer, Stacking constructs a meta-learner by taking the prediction results of KNN-LSTM and Attention-LSTM as feature inputs, and uses linear regression to perform weighted summation or combination of the outputs of the two basic models.

[0074] The task of this meta-model is to learn how to combine the predictions of multiple models in the first layer to obtain a more accurate final prediction.

[0075] The meta-learner in the second layer takes the predictions of the two models in the first layer as input features:

[0076] ;

[0077] Then, a linear regression meta-model is used to learn how to combine the predictions of these two base models:

[0078] ;

[0079] in, , , are the parameters obtained through linear regression training.

[0080] Denormalization: Since the predicted output is normalized, it is necessary to use the scaler_target.inverse_transform function to denormalize the meta-model's prediction results to obtain the final prediction results:

[0081] ;

[0082] Step S5: Model Performance Analysis and Application: In the actual application of the road surface temperature prediction model, the system inputs real-time minute-by-minute meteorological monitoring data, including key characteristics such as temperature, humidity, precipitation, and wind speed. Using a KNN-Attention-LSTM model based on stacking integration, the system can predict road surface temperature values for the next 1 to 72 hours and output the prediction results. This provides a scientific basis for road icing risk warnings, the deployment of de-icing and anti-skid measures, and decision-making by traffic management departments, while also providing a reliable reference for public travel information services.

[0083] In terms of model training and updating, the system regularly evaluates the model's predictive performance, calculating metrics such as the mean absolute error (MAE) and root mean square error (RMSE) of the prediction results. When the MAE exceeds a preset threshold (0.6) or prediction accuracy declines, the system automatically triggers an update mechanism, reselecting features and retraining the model using the latest historical traffic and meteorological data. During each update, the system analyzes the contribution of features to winter road surface temperature prediction using the Spearman rank correlation coefficient, automatically selecting meteorological factors with significant contributions (such as temperature and humidity), removing features with poor predictive performance, and retraining and optimizing the KNN-Attention-LSTM model and stacking integration strategy. After the model update is complete, the system acquires real-time, minute-by-minute measured data for rolling predictions, ensuring the timeliness and accuracy of prediction results to meet the dynamic needs of road safety management.

[0084] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration, characterized in that: The following steps are involved: Step S1: Obtain historical data from a traffic weather monitoring station, including visibility, temperature, humidity, precipitation, wind speed, wind direction, and road surface temperature; Step S2: perform quality control on the data, including time alignment, outlier processing, and missing value filling; Step S3: Calculate the Spearman rank correlation coefficient between winter road surface temperature and other meteorological factors, and select features with strong correlation; Step S4: Use Stacking integration technology to build a KNN-Attention-LSTM prediction model, specifically: Step S41: Use two basic learners, KNN-LSTM and Attention-LSTM, to learn and predict the input data respectively; Step S42: Based on step S41, Stacking constructs a meta-learner by taking the prediction results of KNN-LSTM and Attention-LSTM as feature inputs, and uses linear regression to perform weighted summation or combination of the outputs of the two basic models. Step S43: Use the scaler_target.inverse_transform function to denormalize the prediction result of the meta-model to obtain the final prediction result; Step S5: Input real-time meteorological data and output the road surface temperature forecast value for the next 1-72 hours.

2. The KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration according to claim 1 is characterized in that: The specific method of step S41 is: The KNN-LSTM model is used to capture the long-term dependencies of the time series, and physical boundaries are extracted through KNN. These boundaries are added as constraints to the LSTM loss function: L total =MSE(y Pred ,y true )+λ*MSE(y Pred ,y constraint ); Among them, λ is the weight of the physical constraint, which controls the influence of the physical constraint loss; Based on the LSTM output, the Attention-LSTM model adjusts the weight of the time step through the self-attention mechanism to learn which time steps have a greater impact on the prediction results; the multi-head attention mechanism enhances the model's feature capture capability. Where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the key. By calculating the similarity between the query and the key, the attention weight is obtained, and the weighted sum is used to obtain the final output. For each input data sample, each base model will give a predicted value: y klstm =kNN-LSTM(X); y alstm =Attention-LSTM(X); Among them, y klstm and y alstm They are the predicted values obtained by KNN-LSTM and Attention-LSTM models respectively.

3. The KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration according to claim 2 is characterized in that: The specific method of step S42 is: The two model prediction results of the first layer are used as input features through the meta-learner: X meta =[and klstm ,and alstm ]; Then, a linear regression meta-model is used to learn how to combine the predictions of these two base models: Among them, β0, β1, and β2 are parameters obtained through linear regression training.

4. The KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration according to claim 1 is characterized in that: It also includes: step S6, the system regularly evaluates the prediction performance of the model and calculates the mean absolute error (MAE) and root mean square error (RMSE) of the prediction results; when the MAE exceeds the preset threshold or the prediction accuracy decreases, the system will automatically trigger the update mechanism and use the latest collected traffic meteorological historical data to re-select features and re-train the model.

5. The KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration according to claim 4 is characterized in that: In step S6, during each update, the system analyzes the contribution of features to the winter road surface temperature prediction based on the Spearman rank correlation coefficient, automatically selects meteorological factors with greater contributions, eliminates features with poor prediction effects, retrains and optimizes the KNN-Attention-LSTM model and Stacking integration strategy. After the model update is completed, the system obtains minute-by-minute measured data in real time for rolling prediction.

6. The KNN-Attention-LSTM winter road surface temperature prediction method based on stacking integration according to claim 4 is characterized by: The method for performing quality control on the data in step S2 includes: Outlier detection: Descriptive statistical analysis and box-and-whisker plots are used to identify abnormal data points, reasonable thresholds for meteorological elements are set, and data outside the range is marked as abnormal; Outlier processing: Single-point outliers are interpolated using the mean of the previous and next moments; continuous outliers are retained and marked if they conform to meteorological laws; if they are obviously wrong, they are corrected using the sliding window mean method; Missing value filling: For continuous missing values, linear interpolation is used. For non-continuous missing values, forward filling and backward filling are combined and the average of the two is taken. Time alignment: Check whether the timestamps are arranged according to the set time interval. If there is a slight deviation, align them to the nearest standard time point; Data consistency assurance: Unify units to ensure that all meteorological elements use standard units.

Images