A pavement flatness prediction method based on a pre-trained TabPFN model

By combining a pre-trained TabPFN model with high-order quantile mapping and SHAP attribution analysis, the stability and accuracy problems of road surface smoothness prediction under sparse monitoring of basic highways were solved, and efficient prediction was achieved under conditions of small sample size and missing features.

CN123069883APending Publication Date: 2026-11-10HUAZHONG UNIV OF SCI & TECH +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611248623.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-18
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing road surface evenness prediction methods suffer from problems such as overfitting under small sample conditions, missing key features, and insufficient cross-regional generalization ability when monitoring sparsely on basic highways, resulting in insufficient prediction stability and accuracy.

Method used

A pre-trained TabPFN model is adopted. It is pre-trained offline on synthetic causal data, and a single forward propagation is performed using a context learning mechanism. Combined with high-order quantile mapping and SHAP attribution analysis, it achieves stable prediction of road surface smoothness and is adaptable to small sample and feature missing conditions.

Benefits of technology

It improves prediction stability and accuracy under conditions of small sample size and missing features, provides interpretable prediction results, reduces the risk of overfitting, and improves cross-regional applicability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123069883A_ABST
    Figure CN123069883A_ABST
Patent Text Reader

Abstract

The application belongs to the technical field of road engineering digitization maintenance, and discloses a road surface flatness prediction method based on a pre-trained TabPFN model. The method comprises the following steps: obtaining road surface service state data as original physical characteristics; performing quantile transformation on the original physical characteristics, and constructing a context prompt set and a query set to adapt the model input; loading the pre-trained TabPFN model and freezing the model parameters; performing inference based on the context samples, calculating the correlation between the to-be-tested samples and the historical samples through a self-attention mechanism, and performing weighted aggregation, and outputting a posterior predictive distribution; outputting an IRI prediction value and a confidence interval according to the posterior predictive distribution; performing SHAP feature interpretation on the prediction result, analyzing the contribution of each input feature, and outputting the road surface flatness prediction and maintenance auxiliary decision result. The application has good prediction stability and generalization ability under the conditions of small samples and feature loss.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the technical field of digital maintenance of road engineering, and more specifically, relates to a method for predicting road surface smoothness based on a pre-trained TabPFN model. Background Technology

[0002] The International Roughness Index (IRI) is a core indicator for evaluating road surface quality and driving comfort. Accurate prediction of road surface roughness plays an important role in decision support for formulating scientific highway maintenance plans and optimizing the allocation of maintenance funds.

[0003] Existing road surface roughness prediction methods can be broadly categorized into the following types based on their technical approaches: Traditional regression analysis methods fit the evolution pattern of IRI (Integrated Road Surface Roughness) using parameters such as road surface age and traffic load, but they struggle to characterize the complex degradation process under the combined effects of multiple factors; Classical machine learning methods (such as artificial neural networks, support vector machines, random forests, and gradient boosting tree variants like XGBoost and CatBoost) can achieve high accuracy when data quality is good and samples are sufficient, and have become the mainstream methods; Deep learning methods (such as convolutional neural networks and long short-term memory networks) can automatically extract high-dimensional abstract features and exhibit good performance in scenarios where monitoring data is continuous and of high quality.

[0004] However, in the actual management and maintenance of grassroots highways, due to limitations in testing funds, equipment conditions, and monitoring capabilities, historical observation data on pavement performance generally suffer from the following problems: First, the monitoring frequency is low and the sample size is limited. The grassroots road network covers a wide area and has long routes; limited funding and equipment make it difficult to conduct high-frequency, full-coverage pavement performance testing along the entire route. Historical IRI observation records are sparse for some road sections, resulting in a very limited number of effective samples for modeling. Under small sample conditions, traditional machine learning and deep learning models are prone to overfitting during training, leading to a significant decrease in model generalization ability; while ensemble learning models have some robustness, their predictive stability is still difficult to guarantee in extremely sparse data scenarios.

[0005] Second, key features are missing. Due to factors such as sensor malfunction, missing data records, or limitations in detection conditions, some key features are often missing in historical observation data. Most existing prediction methods require complete input features, and missing values ​​usually need to be imputed or deleted. This not only increases the workload of data preprocessing but may also introduce additional biases, affecting the reliability of the prediction results.

[0006] Third, the model lacks cross-regional generalization ability. Significant differences exist in road structure design, traffic load characteristics, and climate conditions across different regions, making it difficult for existing models trained on specific datasets to be effectively transferred to other regions. When applied to new road sections with different data distributions, prediction accuracy and stability often decline significantly.

[0007] In recent years, Tabular Prior-data Fitted Network (TabPFN) has attracted attention as an emerging method for modeling tabular data. TabPFN embeds general prior knowledge into the model parameters through offline pre-training on a large amount of synthetic causal data, enabling prediction during the inference phase without gradient updates based on downstream data. However, existing TabPFN technologies are mainly geared towards prediction of general tabular data. When applied to pavement smoothness prediction under sparse monitoring conditions on basic highways, further solutions are needed to address the issues of adapting to pavement physical features and stable inference under conditions of missing features.

[0008] Therefore, it is necessary to provide a road surface smoothness prediction method that can maintain good prediction stability under conditions of small sample size and missing features, so as to make up for the shortcomings of existing technologies in data-sparse scenarios. Summary of the Invention

[0009] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a road surface evenness prediction method based on a pre-trained TabPFN model. The aim is to utilize the prior knowledge of a model pre-trained offline on synthetic causal data, and to complete inference during a single forward propagation through a context learning mechanism. This enables road surface evenness prediction without requiring gradient updates for downstream sparse samples, thereby reducing the risk of overfitting under small sample conditions. This solves the technical problem of insufficient prediction stability in existing technologies under conditions of small samples, low monitoring frequency, and missing key features.

[0010] To achieve the above objectives, according to one aspect of the present invention, a method for predicting road surface evenness based on a pre-trained TabPFN model is provided, comprising the following steps: S1: Obtain multi-dimensional pavement service status data under sparse working conditions as the original physical features. The pavement service status data includes asphalt surface layer thickness, cumulative equivalent axle load application times, annual average temperature, annual precipitation, annual freezing index, and historically observed International Roughness Index (IRI) records. Among them, pavement service status data with observed IRI values ​​are used as historical samples, and the IRI of the corresponding time period of the road segment to be predicted is used as the prediction target. Some missing features are allowed in the pavement service status data. S2: Perform prior distribution alignment processing on the original physical features obtained in step S1. Based on quantile transformation, use high-order quantile mapping to transform the long-tailed physical quantities to the feature space that the TabPFN model depends on. Reconstruct the aligned two-dimensional structured table data into a high-dimensional input tensor that adapts to the Transformer architecture. Encode historical samples into a context hint set sequence. Encode the feature vector of the road segment to be predicted into a query set sequence. The IRI in the query set sequence is the item to be predicted. S3: Load the locally deployed network pre-trained weight file, which contains optimized Transformer encoder parameters and multi-head attention parameters. Merge the context cue set sequence and query set sequence generated in step S2 and input them into the TabPFN model. Keep the model parameters frozen and do not perform gradient backpropagation or weight update. S4: Based on the context hint set sequence and query set sequence constructed in step S2, inference is performed using the pre-trained TabPFN model loaded in step S3. The association relationship between each test sample in the query set sequence and all historical samples in the hint set sequence is calculated. Based on the association relationship, the predictable items in the hint set are weighted and aggregated. The posterior prediction distribution result of the test sample is directly approximated and solved in a single forward propagation process. S5: Analyze the posterior prediction distribution results output in step S4, extract the posterior expected value as the deterministic prediction value of road surface smoothness, and generate a confidence interval that reflects the uncertainty of the model to provide risk reference for subsequent maintenance decisions. S6: The SHAP attribution algorithm is used to interpret the features of the road surface evenness prediction value obtained in step S5, calculate the marginal contribution of each original physical feature to the current prediction result, and output a structured maintenance auxiliary decision result containing the evenness prediction value, confidence interval and feature attribution weight.

[0011] Preferably, step S2 specifically includes the following steps: S21: For the original physical features that are not missing, perform quantile transformation based on the empirical distribution of each feature to map the cumulative equivalent axle load number and annual total precipitation physical quantities that have a long tail or skewed distribution to the feature space that is compatible with the prior of the pre-trained TabPFN model. S22: Construct a context cue set from the transformed features of historical road segment samples and their observed IRI values, construct a query set from the transformed features of the road segment to be predicted, and set the IRI corresponding to the query set as the item to be predicted.

[0012] Preferably, the prior distribution alignment process in step S2 uses a higher-order quantile mapping to transform the physical quantity from the original distribution to the feature space on which the model's underlying layer depends, so as to eliminate the influence of different physical quantities and distribution forms on model inference.

[0013] Preferably, the expression for the posterior prediction distribution in step S4 is:

[0014] In the formula, This represents a set of historical observation samples containing historical road segment characteristics and corresponding measured IRI values. This represents the input features of the sample to be tested. This represents the IRI value of the road segment to be predicted. This represents the pre-trained TabPFN model.

[0015] Preferably, during the inference process in step S4, the pre-trained TabPFN model performs weighted aggregation of the items to be predicted in the cue set based on the association between the test sample and historical samples. Its self-attention calculation process is expressed as follows:

[0016] in, The query representation of the sample to be tested. Key representations of historical observation samples, The values ​​representing historical observation samples are... express transpose, The dimension represented by the key. To perform weighted aggregation of historical sample information based on weights, This is used to normalize the association score into attention weights.

[0017] Preferably, the feature attribution weights in step S6 are used to quantify the dynamic coupling relationship between disaster-causing factors and resistance factors, wherein the annual freezing index and the average annual temperature have a positive damaging effect, and the asphalt surface layer thickness has a negative protective effect.

[0018] Preferably, the road surface service status data in step S1 is obtained by retrieving historical records or by portable testing equipment, and the missing features include data missing due to missed detection or sensor failure.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0020] In summary, compared with the prior art, the road surface evenness prediction method based on the pre-trained TabPFN model provided by this invention has the following advantages: 1. Good stability in small sample inference. By utilizing prior knowledge embedded in the model parameters through offline pre-training and combining it with a contextual cue set composed of historical samples for single forward inference, there is no need to update gradients for downstream sparse samples, which can reduce the risk of overfitting caused by repeated training of traditional models under small sample conditions.

[0021] 2. Predictive capability remains even with missing features. For road service features with missed detections, sensor malfunctions, or missing records, the model can combine other non-missing features with historical sample information from the contextual cue set to perform inference, reducing additional biases that may be introduced by conventional imputation or sample deletion processes.

[0022] 3. Good data distribution adaptability. By performing high-order quantile mapping on the road surface physical features that exhibit long-tailed or skewed distributions, the difference between the original road surface data distribution and the prior feature space of the pre-trained model is reduced, which helps to improve the inference stability of the model under different road surface service data conditions.

[0023] 4. The prediction results are interpretable. SHAP attribution analysis is used to calculate the contribution of various pavement structure, traffic load, and climate characteristics to the IRI prediction results, providing a reference for analyzing the impact of different factors on pavement smoothness prediction results. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating the overall implementation of the method of the present invention; Figure 2 This is a comparison chart of the IRI prediction determination coefficients of different algorithms under small sample conditions in the embodiments of the present invention as a function of sample size; Figure 3 This is a comparison chart of the root mean square error of IRI prediction under the condition of increasing feature missing rate in different algorithms in the embodiments of the present invention, which is used to illustrate the prediction ability of the method of the present invention under the condition of incomplete features. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0026] Please see Figure 1 The present invention provides a method for predicting road surface smoothness based on a pre-trained TabPFN model, comprising the following steps: S1: Obtain multi-dimensional pavement service status data under sparse working conditions as the original physical features. The pavement service status data includes asphalt surface layer thickness, cumulative equivalent axle load application times, annual average temperature, annual precipitation, annual freezing index, and historically observed International Roughness Index (IRI) records. Among them, pavement service status data with observed IRI values ​​are used as historical samples, pavement service status data without IRI values ​​are samples to be tested, and the IRI of the corresponding time period of the road segment to be predicted is used as the prediction target. Some missing features are allowed in the pavement service status data. Specifically, restricted working condition data construction and feature selection are performed. Based on the pre-analysis of pavement performance causality, five core covariates and one target variable are selected. The target variable and original physical features used in this embodiment are shown in Table 1. A highly sparse feature matrix containing missing values ​​is constructed through a feature masking mechanism. Specific variables include: the International Roughness Index (IRI) as the target variable, and asphalt pavement thickness, cumulative equivalent axle load application times, annual average temperature, annual freezing index, and annual total precipitation as the original physical features. For road segments to be predicted with historical detection records, the most recently observed IRI before the prediction time is used as the pavement historical state information; samples of historical road segments with observed IRI values ​​are used to form the subsequent contextual cue set, and samples of road segments to be predicted are used to form the query set. For feature values ​​with missing values, their original missing state is retained, without mean imputation or sample deletion.

[0027] Table 1. Target variables and input features used in this embodiment.

[0028] S2: Perform prior distribution alignment processing on the original physical features obtained in step S1. Based on quantile transformation, use high-order quantile mapping to transform the long-tailed physical quantities to the feature space that the TabPFN model depends on. Reconstruct the aligned two-dimensional structured table data into a high-dimensional input tensor that adapts to the Transformer architecture. Encode historical samples into a context hint set sequence. Encode the feature vector of the road segment to be predicted into a query set sequence. The IRI in the query set sequence is the item to be predicted. Specifically, step S2 includes the following sub-steps: S21: For the original physical features that are not missing, perform quantile transformation based on the empirical distribution of each feature to map the cumulative equivalent axle load number and annual total precipitation physical quantities that have a long tail or skewed distribution to the feature space that is compatible with the prior of the pre-trained TabPFN model. S22: Construct a context cue set from the transformed features of historical road segment samples and their observed IRI values, construct a query set from the transformed features of the road segment to be predicted, and set the IRI corresponding to the query set as the item to be predicted.

[0029] S3: Load the locally deployed network pre-trained weight file, which contains optimized Transformer encoder parameters and multi-head attention parameters. Merge the context cue set sequence and query set sequence generated in step S2 and input them into the TabPFN model. Keep the model parameters frozen and do not perform gradient backpropagation or weight update. S4: Based on the context hint set sequence and query set sequence constructed in step S2, inference is performed using the pre-trained TabPFN model loaded in step S3. The association relationship between each test sample in the query set sequence and all historical samples in the hint set sequence is calculated. Based on the association relationship, the predictable items in the hint set are weighted and aggregated. The posterior prediction distribution result of the test sample is directly approximated and solved in a single forward propagation process. Specifically, preferably, the expression for the posterior prediction distribution in step S4 is:

[0030] In the formula, This represents a set of historical observation samples containing historical road segment characteristics and corresponding measured IRI values. This represents the input features of the sample to be tested. This represents the IRI value of the road segment to be predicted. This represents the pre-trained TabPFN model.

[0031] Specifically, in the inference process of step S4, the pre-trained TabPFN model performs weighted aggregation of the items to be predicted in the cue set based on the association between the test sample and historical samples. Its self-attention calculation process is expressed as follows:

[0032] in, The query representation of the sample to be tested. Key representations of historical observation samples, The values ​​representing historical observation samples are... express transpose, The dimension represented by the key. To perform weighted aggregation of historical sample information based on weights, This is used to normalize the association score into attention weights.

[0033] S5: Analyze the posterior prediction distribution output in step S4, extract the posterior expected value as the deterministic prediction value of road surface smoothness, and generate a confidence interval that reflects the uncertainty of the model to provide risk reference for subsequent maintenance decisions. S6: The SHAP attribution algorithm is used to interpret the features of the road surface evenness prediction value obtained in step S5, calculate the marginal contribution of each original physical feature to the current prediction result, and output a structured maintenance auxiliary decision result containing the evenness prediction value, confidence interval and feature attribution weight.

[0034] This embodiment uses the pavement performance benchmark library from the LTPP program as the basic data source. To simulate the actual sparse monitoring conditions of the base road network, controlled downsampling and random masking are performed on the raw data to construct test subsets with different sample numbers and different proportions of missing features, in order to verify the applicability of the method of this invention under data-constrained conditions.

[0035] In this embodiment, a pre-trained TabPFN model is selected as the road surface evenness prediction model. The pre-trained weight file is downloaded from the official open-source repository and deployed locally. The transformed features, context hint set sequence, and query set sequence obtained in step S2 are input into the pre-trained TabPFN model to obtain the road surface evenness prediction results for the corresponding samples. For comparison and verification, two gradient boosting tree algorithms, XGBoost and CatBoost, are selected as the comparison model, and their hyperparameters are optimized on the corresponding training set.

[0036] This embodiment presents the prediction process of TabPFN under conditions of small sample size and high missing rate, as shown in Table 2 below. The algorithm first performs random sampling and missing mask injection on the training set, then calls the TabPFN classifier and regressor respectively for gradient-free update inference, and uses metrics such as Brier Score, log loss, and coefficient of determination to evaluate the prediction performance. Finally, it uses the SHAP value to perform physical attribution of the model's decisions.

[0037]

[0038] Please see Figure 2 This graph compares the coefficients of determination for IRI predictions using different algorithms under small sample conditions, illustrating the prediction stability of the method presented in this invention under small sample conditions. In the continuous smoothness prediction task, the coefficients of determination of the method presented in this invention remain relatively stable with the sample size under small sample conditions, indicating that it can reduce the risk of prediction instability under small sample conditions.

[0039] Please see Figure 3 This chart compares the changes in the root mean square error (RMSE) of IRI predictions using different algorithms under increasing feature missing rate conditions, illustrating the predictive ability of the method presented in this invention under incomplete feature conditions. Under the test condition of increasing feature missing rate, the RMSE of the method presented in this invention increases relatively slowly, indicating that it still possesses predictive ability in scenarios with incomplete features.

[0040] Feature attribution analysis based on SHAP shows that the annual freezing index and average annual temperature have a significant impact on the prediction results, followed by cumulative axle load. Asphalt pavement thickness exhibits a certain protective effect, while high freeze-thaw cycles, high temperatures, and heavy traffic show a positive damaging effect. These results indicate that the prediction results of the method in this invention are in good agreement with general knowledge of pavement engineering. This agreement suggests that although TabPFN uses only a small number of samples for inference, its internal feature weight allocation does not deviate from known physical laws.

[0041] In summary, this invention provides a road surface evenness prediction method based on a pre-trained TabPFN model. The pre-trained TabPFN model is a basic model that has completed offline training. The technical improvement of this invention lies in applying the model to road surface evenness prediction and combining it with the original physical features, the construction of contextual hint sets and query sets, and inference under the condition of missing features to achieve adaptation. It can be applied to road surface evenness prediction under small sample and feature missing conditions, and can provide a reference for the pavement performance prediction and maintenance auxiliary decision-making of basic highways under sparse monitoring conditions.

[0042] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for predicting road surface smoothness based on a pre-trained TabPFN model, characterized in that: Includes the following steps: S1: Obtain multi-dimensional pavement service status data under sparse working conditions as the original physical features. The pavement service status data includes asphalt surface layer thickness, cumulative equivalent axle load application times, annual average temperature, annual precipitation, annual freezing index, and historically observed International Roughness Index (IRI) records. Among them, pavement service status data with observed IRI values ​​are used as historical samples, and the IRI of the corresponding time period of the road segment to be predicted is used as the prediction target. Some missing features are allowed in the pavement service status data. S2: Perform prior distribution alignment processing on the original physical features obtained in step S1. Based on quantile transformation, use high-order quantile mapping to transform the long-tailed physical quantities to the feature space that the TabPFN model depends on. Reconstruct the aligned two-dimensional structured table data into a high-dimensional input tensor that adapts to the Transformer architecture. Encode historical samples into a context hint set sequence. Encode the feature vector of the road segment to be predicted into a query set sequence. The IRI in the query set sequence is the item to be predicted. S3: Load the locally deployed network pre-trained weight file, which contains optimized Transformer encoder parameters and multi-head attention parameters. Merge the context cue set sequence and query set sequence generated in step S2 and input them into the TabPFN model. Keep the model parameters frozen and do not perform gradient backpropagation or weight update. S4: Based on the context hint set sequence and query set sequence constructed in step S2, inference is performed using the pre-trained TabPFN model loaded in step S3. The association relationship between each test sample in the query set sequence and all historical samples in the hint set sequence is calculated. Based on the association relationship, the predictable items in the hint set are weighted and aggregated. The posterior prediction distribution result of the test sample is directly approximated and solved in a single forward propagation process. S5: Analyze the posterior prediction distribution results output in step S4, extract the posterior expected value as the deterministic prediction value of road surface smoothness, and generate a confidence interval that reflects the uncertainty of the model to provide risk reference for subsequent maintenance decisions. S6: The SHAP attribution algorithm is used to interpret the features of the road surface evenness prediction value obtained in step S5, calculate the marginal contribution of each original physical feature to the current prediction result, and output a structured maintenance auxiliary decision result containing the evenness prediction value, confidence interval and feature attribution weight.

2. The method as described in claim 1, characterized in that: Step S2 specifically includes the following steps: S21: For the original physical features that are not missing, perform quantile transformation based on the empirical distribution of each feature to map the cumulative equivalent axle load number and annual total precipitation physical quantities that have a long tail or skewed distribution to the feature space that is compatible with the prior of the pre-trained TabPFN model. S22: Construct a context cue set from the transformed features of historical road segment samples and their observed IRI values, construct a query set from the transformed features of the road segment to be predicted, and set the IRI corresponding to the query set as the item to be predicted.

3. The method as described in claim 1, characterized in that: The prior distribution alignment process described in step S2 uses high-order quantile mapping to transform the physical quantity from the original distribution to the feature space on which the model depends, so as to eliminate the influence of different physical quantities and distribution forms on model inference.

4. The method as described in claim 1, characterized in that: The expression for the posterior prediction distribution mentioned in step S4 is: In the formula, This represents a set of historical observation samples containing historical road segment characteristics and corresponding measured IRI values. This represents the input features of the sample to be tested. This represents the IRI value of the road segment to be predicted. This represents the pre-trained TabPFN model.

5. The method as described in claim 1, characterized in that: In the inference process of step S4, the pre-trained TabPFN model performs weighted aggregation of the items to be predicted in the cue set based on the association between the test sample and historical samples. Its self-attention calculation process is expressed as follows: in, The query representation of the sample to be tested. Key representation of historical observation samples, The values ​​representing historical observation samples are... express transpose, The dimension represented by the key. To perform weighted aggregation of historical sample information based on weights, This is used to normalize the association score into attention weights.

6. The method as claimed in claim 1, characterized in that: The feature attribution weights mentioned in step S6 are used to quantify the dynamic coupling relationship between disaster-causing factors and resistance factors. Among them, the annual freezing index and the average annual temperature have a positive damaging effect, while the asphalt surface layer thickness has a negative protective effect.

7. The method as claimed in claim 1, characterized in that: The road surface service status data mentioned in step S1 is obtained by retrieving historical records or by portable detection equipment. The missing features include data loss due to missed detection or sensor failure.

8. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the method as claimed in any one of claims 1 to 7.