Rural road condition index data quality repairing method and system based on stacking ensemble learning

Abstract

The application discloses a kind of rural road condition index data quality repair method and system based on Stacking integrated learning, comprising: rural road condition index data is washed to anomaly, and data missing mode is divided;For the data of source missing mode, construct filling feature set according to disease internal correlation and pavement decay law;For the data of sub-index missing mode, construct filling feature set according to pavement decay law;For filling feature set, variable selection is carried out using Lasso regression algorithm, and the repair combination after dimension reduction is obtained;Based on Stacking integrated model, according to the repair combination after dimension reduction, rural road automatic detection index data is filled.The application proposes a scientific and reasonable data filling method for the problem of missing road condition index data caused by data collection and data cleaning process, which can substantially improve data quality and provide solid technical and data support for the technical condition assessment, maintenance management and other work of low-grade rural roads.

Description

Technical Field

[0001] This invention belongs to the field of rural road maintenance and data quality restoration technology, specifically involving a method and system for restoring the data quality of rural road condition indicators based on Stacking ensemble learning. Background Technology

[0002] In recent years, as a fundamental task supporting the assessment of road surface technical conditions and the formulation of maintenance decisions, the construction of automated rapid detection systems for rural road conditions has been accelerated in various provinces. The accumulated data resources of automated detection of rural road conditions are enormous, and the value of these data assets urgently needs to be explored and utilized.

[0003] However, the quality issues of the vast and diverse rural road condition monitoring data restrict the effective application of this data, and its accuracy is crucial to the rationality of rural road maintenance decisions. Before making maintenance decisions, rural road condition indicator data that does not meet accuracy and validity requirements are usually cleaned and deleted. However, this results in random data gaps, which will affect the implementation of maintenance decisions across the entire road network. Therefore, how to adopt scientific and reasonable data imputation methods to perform delayed repair of low-grade rural road condition indicator data after outlier cleansing is one of the urgent problems to be solved in the field of rural road maintenance and data quality repair.

[0004] According to existing literature, in recent years, with the development of practical needs and related technologies, algorithms such as deep learning, machine learning, and graph networks have been widely applied in data quality repair. Patent CN117786529A discloses an intelligent repair method for SCADA data of wind turbine generators. This invention establishes a generator and discriminator using a deep learning generative adversarial network (GTN) and optimizes their respective parameters for intelligent data repair. Patent CN117743938A discloses a method and system for anomaly identification and repair of imbalanced data. It uses algorithms such as clustering and isolated forests to divide data into different clusters, identify abnormal data, and repair mixed data anomalies and missing data based on a data repair model using a heterogeneous variational autoencoder. In the field of transportation, patent CN117633447A discloses a traffic flow data repair method based on the judgment of missing data from traffic flow data acquisition equipment. It constructs a generative adversarial network containing a graph transform network (GTN) to output repaired data.

[0005] However, in terms of road maintenance decision-making and road condition data, especially for automated detection data of rural roads, there is still a lack of relevant repair research and invention methods. There is an urgent need to build a rural road surface condition data repair model based on algorithms such as machine learning and ensemble learning, to scientifically and rationally fill in road condition index data, and to accurately repair it, thereby further improving the implementation of maintenance decision-making. Summary of the Invention

[0006] Purpose of the invention: This invention proposes a method and system for repairing the quality of rural road condition index data based on Stacking ensemble learning. Addressing the problem of random missing road condition index data, this invention repairs the road condition index data of low-grade rural roads, substantially improving data quality and providing solid technical support for the technical condition assessment, maintenance management, and other work related to low-grade rural roads.

[0007] Technical Solution: To achieve the above-mentioned objectives, the present invention provides a method for restoring the quality of rural road condition index data based on Stacking ensemble learning, which specifically includes the following steps:

[0008] (1) Perform anomaly cleaning on rural road condition index data and original collected data, and classify the missing patterns of rural road condition index data, including source missing patterns and sub-index missing patterns.

[0009] (2) For data with the source missing pattern, a fill feature set is constructed based on the inherent correlation of the disease and the pavement decay law; for data with the sub-index missing pattern, a fill feature set is constructed based on the pavement decay law.

[0010] (3) For the constructed imputed feature set, the Lasso regression algorithm is used to select variables, remove redundant variables, and obtain the imputed combination after dimensionality reduction;

[0011] (4) Select the base learners of the Stacking ensemble model, split the original data into training and test sets, and train each base learner.

[0012] (5) Based on the trained Stacking ensemble model, fill in the data of rural road automation detection indicators according to the repair combination after dimensionality reduction.

[0013] Furthermore, the rural road condition index data mentioned in step (1) includes the historical road damage index (PCI), road driving quality index (RQI), and road technical condition index (PQI), of which the total index is PQI and the sub-indices are PCI and RQI; the original data collected in step (1) is the data used to calculate the rural road condition index, including the historical data on the area of ​​defects related to PCI, the defect rate (DR), and the international roughness index (IRI) related to RQI, of which the area of ​​defects includes cracks, longitudinal cracks, transverse cracks, potholes in asphalt pavement, and broken slabs, cracks, potholes, exposed aggregate, etc. in cement pavement.

[0014] Furthermore, the source-level missing pattern refers to the missing index data caused by the lack of original collected data. For the source-level missing pattern, a repair method combining vertical and horizontal repair can be adopted. The specific ideas include: ① filling the defect area based on the correlation between defects (cracking, longitudinal cracks, transverse cracks, potholes in asphalt pavement, broken slabs, cracks, potholes, exposed aggregate defects in cement pavement); ② repairing DR or PCI based on the correlation between the non-missing defect area data and DR or PCI; ③ filling PCI, RQI, or PQI data based on the temporal variation pattern of the index data itself. Among them, PCI repair involves the missing of multiple variables related to defect area, and it is advisable to start filling the variable with the lowest missing proportion one by one. The sub-index missing pattern refers to the missing total index PQI data caused by the missing sub-indexes PCI and RQI. Since reasonable index data cannot be calculated from the original collected data whose accuracy is questionable, the original collected data does not have reference value for horizontal repair. It is advisable to adopt the vertical repair method, that is, to fill PCI, RQI, or PQI data based on the temporal variation pattern of the index data itself.

[0015] Furthermore, in step (3), the Lasso regression algorithm is used to select variables and remove redundant variables to obtain the dimension-reduced repair combination, which includes the following steps:

[0016] (3-1) Standardize the dataset;

[0017] (3-2) Set the Lasso regression objective function as:

[0018]

[0019] Where, β Lasso Let X be the coefficient vector of Lasso regression. j Let Y be the j-th feature variable, p be the number of feature variables, Y be the observed value, and β be the parameter vector to be estimated. j Let be the parameter of the j-th feature variable, and λ be the coefficient of the penalty term. This is a penalty item;

[0020] (3-3) The mean square error (RMSE) was calculated using the K-fold cross-validation method. The order of the influencing factors of each feature value entering the model and the change of the proportion of fitting coefficient values ​​were calculated under different λ.

[0021] (3-4) Remove variables whose fitting coefficients are still 0 when the x-axis is 0, and the remaining variables are the repaired combination after dimensionality reduction.

[0022] Further, the base learner of the Stacking ensemble model described in step (4), including KNN, includes the filling step comprising:

[0023] (4-1-1) Sort the missing data items from low to high according to the missing ratio;

[0024] (4-1-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation;

[0025] (4-1-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The KNN model is trained using data records without missing values ​​to fill in the missing values.

[0026] (4-1-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set.

[0027] (4-1-5) Train the KNN model by iterating through all specified k values, and repeat cross-validation K times for each k value;

[0028] (4-1-6) Calculate the average of the accuracy of K cross-validations to obtain the model accuracy corresponding to the value of k;

[0029] (4-1-7) Compare the model accuracy corresponding to all k values ​​to obtain the optimal k value, the corresponding KNN model, and the corresponding imputation results;

[0030] (4-1-8) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

[0031] Further, the base learner of the Stacking ensemble model described in step (4), including MissForest, has a filling step comprising:

[0032] (4-2-1) Sort the missing data items from low to high according to the missing ratio;

[0033] (4-2-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation;

[0034] (4-2-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The MissForest model is trained using data records without missing values ​​to fill in the missing values.

[0035] (4-2-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set.

[0036] (4-2-5) Calculate the average accuracy of K cross-validations to obtain the optimal MissForest model and the corresponding imputation results;

[0037] (4-2-6) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

[0038] Further, the base learner of the Stacking ensemble model described in step (4) includes XGBoost, whose filling step includes:

[0039] (4-3-1) Sort the missing data items from low to high according to the missing ratio;

[0040] (4-3-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation;

[0041] (4-3-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The XGBoost model is trained using data records without missing values ​​to fill in the missing data.

[0042] (4-3-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set.

[0043] (4-3-5) The XGBoost model is trained and its hyperparameters are tuned using cross-validation with a pre-set learning rate and maximum depth of each tree.

[0044] (4-3-6) Calculate the average of the accuracy of K cross-validations to obtain the model accuracy corresponding to each learning rate and the maximum depth of each tree;

[0045] (4-3-7) Compare the accuracy of all models to obtain the optimal learning rate, the maximum depth of each tree and the corresponding XGBoost model, as well as the corresponding imputation results;

[0046] (4-3-8) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

[0047] Furthermore, step (5), which involves filling in the data of automated detection indicators for low-level rural roads based on the trained Stacking ensemble model and the dimensionality-reduced repair combination, includes the following steps:

[0048] (5-1) The prediction results of KNN, MissForest and XGBoost are merged horizontally and used as a new training set to input into the secondary model for training.

[0049] (5-2) Using the trained Stacking ensemble model, the data filling of the rural road automation detection index is completed based on the dimension-reduced repair combination.

[0050] Based on the same inventive concept, this invention provides a rural road condition index data quality repair system based on Stacking ensemble learning, comprising:

[0051] The preprocessing module is used to clean up anomalies in rural road condition index data and raw collected data, and to classify the missing patterns of rural road condition index data, including source missing patterns and sub-index missing patterns.

[0052] The feature set construction module is used to construct a feature set for data with the source missing pattern based on the inherent correlation of defects and the pavement decay law; and to construct a feature set for data with the sub-index missing pattern based on the pavement decay law.

[0053] The feature set dimensionality reduction module is used to select variables from the constructed imputed feature set using the Lasso regression algorithm, remove redundant variables, and obtain the dimensionality-reduced imputed combination.

[0054] The model training module is used to select the base learners for the Stacking ensemble model, split the original data into training and test sets, and train each base learner.

[0055] And a data incomplete module, which is used to fill in the data of rural road automation detection indicators based on the trained Stacking ensemble model and the dimensionality-reduced repair combination.

[0056] Based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for restoring the quality of rural road condition index data based on Stacking ensemble learning.

[0057] Beneficial Effects: Compared with existing technologies, the Stacking ensemble learning-based method for restoring rural road condition index data quality in this invention, combined with the characteristics of automated rural road detection data, classifies the missing data patterns of rural road condition indicators. Addressing the problem of random missing road condition index data during the collection and cleaning of automated rural road detection data, it constructs a feature set for data filling based on the inherent correlation of road defects and pavement decay patterns, uses the Lasso regression algorithm for dimensionality reduction, and performs data filling based on the Stacking ensemble model. This invention fully utilizes existing automated rural road detection data results, employing a scientific and reasonable filling method to repair the rural road condition index data after outlier cleaning, addressing the problem of random missing road segment index data caused by data cleaning. The filling effect of this invention is excellent, meeting the needs of practical engineering applications, and providing solid technical and data support for substantially improving the quality of automated detection data for low-grade rural roads, assessing the technical condition of rural roads, and making decisions on the maintenance of the entire road network. Attached Figure Description

[0058] Figure 1 This is a flowchart of a method for improving the quality of rural road condition index data based on Stacking ensemble learning, according to an embodiment of the present invention. Detailed Implementation

[0059] To more clearly illustrate the objectives, technical solutions, and advantages of this invention, the specific embodiments of the invention will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The following embodiments are intended to enable those skilled in the art to more fully understand this invention, but do not limit the invention in any way.

[0060] Example 1:

[0061] See Figure 1 This invention discloses a method for restoring the quality of rural road condition index data based on Stacking ensemble learning. The method includes the following steps:

[0062] Step (1) involves anomaly cleaning of rural road condition index data and original collected data, and classifying the missing patterns of rural road condition index data into source missing patterns and sub-index missing patterns. Among them, the source missing pattern refers to the missing index data caused by the missing original collected data; the sub-index missing pattern refers to the missing total index PQI data caused by the missing sub-indicators PCI and RQI.

[0063] To verify the effectiveness of the rural road condition index data quality restoration method based on Stacking ensemble learning, this example uses low-grade rural road condition detection data from X County, T County, and C City in Henan Province from 2022 to 2023. The detected road surface types include asphalt and cement pavements, and the technical grade of the detected routes is level three or below. After data matching and anomaly cleaning, the asphalt pavement dataset contains 995 samples, and the cement pavement dataset contains 8918 samples, both with 13 dimensions. These dimensions include: ① road construction year; ② original and process data from 2023, including damage area, DR, and IRI; ③ index data from 2022 to 2023, including PCI, RQI, and PQI. A simple descriptive statistical summary of the above data is shown in Table 1.

[0064] Table 1. Descriptive statistics of the dataset in specific embodiments.

[0065]

[0066] Step (2): For data with the source missing pattern, construct a fill feature set based on the inherent correlation of the disease and the pavement decay law; for data with the sub-index missing pattern, construct a fill feature set based on the pavement decay law.

[0067] The specific implementation examples of data set imputation feature sets are shown in Table 2. Wherein:

[0068] For PCI and PQI with missing source patterns, the disease area, current year's IRI and RQI, previous years' PCI, RQI, PQI, and year are selected as the filler feature set, as shown in combinations 1-1, 1-2, 2-1, and 2-2 in Table 2.

[0069] For PCI and PQI with missing sub-indicators, the current year's IRI and RQI, previous years' PCI, RQI, PQI, and the year are selected as the filler feature set, as shown in combinations 1-3, 1-4, 2-3, and 2-4 in Table 2.

[0070] Table 2 Specific Examples: Dataset Imputation Feature Set

[0071]

[0072]

[0073] Step (3): For the constructed imputed feature set, the Lasso regression algorithm is used to select variables, remove redundant variables, and obtain the imputed combination after dimensionality reduction.

[0074] Furthermore, step (3) includes the following steps:

[0075] (3-1) Standardize the dataset for the eight patch combinations of the dataset in the specific embodiment.

[0076] (3-2) Set the Lasso regression objective function as:

[0077]

[0078] Where, β Lasso Let X be the coefficient vector of Lasso regression. j Let Y be the j-th feature variable, p be the number of feature variables, Y be the observed value, and β be the parameter vector to be estimated. j Let be the parameter of the j-th feature variable, and λ be the coefficient of the penalty term. This is a penalty item;

[0079] (3-3) The mean squared error (RMSE) was calculated using the 10-fold cross-validation method. The order in which the influencing factors of each feature value entered the model and the changes in the proportion of the fitting coefficients were calculated under different λ values. The RMSE calculation method is as follows:

[0080]

[0081] In the formula, y t and These represent the true value and the predicted value, respectively, and n is the number of imputed samples. A smaller RMSE value indicates a smaller imputation error and higher imputation accuracy.

[0082] (3-4) Remove variables whose fitting coefficients are still 0 when the x-axis is 0, and the remaining variables are the repaired combination after dimensionality reduction.

[0083] For the specific implementation dataset, taking cement pavement combination 2-1 as an example, when λ = 10 -5 At that time, RMSE reached its minimum value of 1.6 × 10. -5 At this point, the fitting coefficients of variables RQI_22, PQI_22, IRI_23, and age were compressed to 0, indicating that these features were redundant. Broken slabs, cracks, potholes, exposed aggregate, RQI_23, and PCI_22 were the main features of the repair sub-index PCI under the source-deficient mode obtained after screening. Similarly, the feature selection results of the 8 repair combinations are shown in Table 3, thus generating 8 new repair combinations for reduced maintenance.

[0084] Table 3 shows the specific examples of dataset dimensionality reduction and feature set completion.

[0085]

[0086] Step (4): Select the base learners for the Stacking ensemble model, split the original dataset with missing data into training and test sets, and train each base learner. This method uses three different learning algorithms, KNN, MissForest, and XGBoost, as base learners, and combines them for effective complementarity, thereby improving overall performance and robustness: KNN predicts the label or value of the test sample by finding the K most similar samples to the test sample in the training set, which can capture non-linear relationships; MissForest uses the observation part of the dataset to train a random forest to directly predict missing values, and has strong adaptability to various missing patterns; XGBoost makes the predicted value and the true value approach each other through continuous iteration until they converge, with strong generalization ability and low computational complexity.

[0087] Further, in step (4), selecting the base learners of the Stacking ensemble model, splitting the training set and the test set, and training each base learner includes the following steps:

[0088] (4-1) Training KNN, the process includes the following steps:

[0089] (4-1-1) Sort the missing data items from low to high according to the missing ratio;

[0090] (4-1-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation;

[0091] (4-1-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The KNN model is trained using data records without missing values ​​to fill in the missing values.

[0092] (4-1-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set.

[0093] (4-1-5) Train the KNN model by iterating through all specified k values, and repeat cross-validation K times for each k value;

[0094] (4-1-6) Calculate the average of the accuracy of K cross-validations to obtain the model accuracy corresponding to the value of k;

[0095] (4-1-7) Compare the model accuracy corresponding to all k values ​​to obtain the optimal k value, the corresponding KNN model, and the corresponding imputation results.

[0096] (4-1-8) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

[0097] (4-2) Training MissForest, the implementation process includes the following steps:

[0098] (4-2-1) Sort the missing data items from low to high according to the missing ratio;

[0099] (4-2-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation;

[0100] (4-2-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The MissForest model is trained using data records without missing values ​​to fill in the missing values.

[0101] (4-2-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set.

[0102] (4-2-5) Calculate the average accuracy of K cross-validations to obtain the optimal MissForest model and the corresponding imputation results.

[0103] (4-2-6) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

[0104] (4-3) Training XGBoost, the implementation process includes the following steps:

[0105] (4-3-1) Sort the missing data items from low to high according to the missing ratio;

[0106] (4-3-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation;

[0107] (4-3-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The XGBoost model is trained using data records without missing values ​​to fill in the missing data.

[0108] (4-3-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set.

[0109] (4-3-5) The XGBoost model is trained and its hyperparameters are tuned using cross-validation with a pre-set learning rate and maximum depth of each tree.

[0110] (4-3-6) Calculate the average of the accuracy of K cross-validations to obtain the model accuracy corresponding to each learning rate and the maximum depth of each tree;

[0111] (4-3-7) Compare the accuracy of all models to obtain the optimal learning rate, the maximum depth of each tree and the corresponding XGBoost model, as well as the corresponding imputation results;

[0112] (4-3-8) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

[0113] Step (5): Based on the trained Stacking ensemble model, fill in the data of automated detection indicators for low-level rural roads according to the repair combination after dimensionality reduction.

[0114] Furthermore, in step (5), based on the Stacking ensemble model, the data filling for automated detection indicators of low-grade rural roads according to the dimensionality-reduced repair combination includes the following steps:

[0115] (5-1) The prediction results of KNN, MissForest and XGBoost are merged horizontally and used as a new training set to train secondary models such as linear regression.

[0116] For the specific implementation dataset, step (5-1) is implemented. The imputation performance based on the Stacking ensemble learning method is the best. The Stacking ensemble model refers to the ensemble imputation model built with KNN, MissForest, and XGBoost as base learners. Among the base learners used, KNN performs better overall.

[0117] Three indicators—Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and Mean Cross-Class Rate (MCLE)—were used to analyze and evaluate the model's completion results for two evaluation objects: data completion for all road segments and data completion for key road segments. The indicators are defined as follows:

[0118] ①Mean Absolute Percentage Error (MAPE):

[0119]

[0120] In the formula, y t and These represent the true value and the predicted value, respectively, and n is the number of imputed samples. A smaller MAPE value indicates a smaller imputation error and higher imputation accuracy.

[0121] ②Root Mean Square Error (RMSE):

[0122]

[0123] A smaller RMSE value indicates a smaller filling error and higher filling accuracy.

[0124] ③ Average Cross-Rank Rate (MCLE):

[0125]

[0126] In the formula, l t and These represent the true value level of the indicator data and the level corresponding to the imputed data, respectively. The smaller the MCLE value, the smaller the imputation error, and the less likely the indicator data is to be imputed across levels.

[0127] Table 4 shows the average model imputation performance of different maintenance reduction imputation combinations at different missing rates on key road sections and all road sections, with the best results shown underlined. Among them, the imputation model based on Stacking ensemble learning has the lowest MCLE, MAPE values ​​on key road sections and the lowest MCLE, MAPE, and RMSE values ​​on all road sections, indicating the lowest estimation deviation between the imputed values ​​and the actual values. Compared to the state-of-the-art KNN method among base learners, the MCLE and MAPE of the imputed values ​​on key road sections are reduced by 23% and 11%, respectively, and the MAPE of the imputed values ​​on all road sections is reduced by 19%.

[0128] Table 4. Repair effects of different repair models in specific embodiments.

[0129]

[0130] (5-2) Using the trained Stacking ensemble model, the data filling of automated detection indicators for low-grade rural roads is completed based on the dimensionality-reduced repair combination. Among them, when PCI repair involves multivariate missing values ​​related to the area of ​​damage, repair is carried out one by one starting from the variable with the lowest missing value.

[0131] To further illustrate the superior repair effect of different variable combinations after dimensionality reduction, model verification was performed on the initial features (see Table 2) and the dimensionality-reduced features (see Table 3). Table 5 shows the average filling effect and change value of the model using feature subsets before and after dimensionality reduction in specific embodiments.

[0132] Table 5. Repair effects of different feature subsets before and after dimensionality reduction in specific embodiments.

[0133]

[0134] The model imputation performance using the dimensionality-reduced feature subset is roughly equivalent to that using all initial feature variables, but the imputation efficiency is improved by approximately 30%. As shown in the table, the RMSE of key road segment index imputation after feature dimensionality reduction decreased by 20.5%, while the MCLE of the cross-level rate slightly increased. However, given the massive volume of automated detection data and the presence of multiple missing variables, to optimize model performance, imputation requires iterative imputation for each missing feature variable. Therefore, reducing the number of model features significantly improves computational efficiency. With essentially equal imputation results, choosing the dimensionality-reduced feature subset with fewer variables is more suitable for practical applications.

[0135] As can be seen from the above description of the specific embodiments in conjunction with the accompanying drawings, this invention addresses the issue of random missing road condition index data during the collection and cleaning of automated rural road inspection data by classifying missing road condition index data patterns. For different patterns, a filling feature set is constructed based on the inherent correlation of defects and pavement decay patterns. Redundant variables are removed from the filling feature set using the Lasso regression algorithm to obtain a dimension-reduced repair combination. Based on the Stacking ensemble model, the automated inspection index data of low-grade rural roads is filled using the dimension-reduced repair combination. This invention achieves the repair of road condition index data for low-grade rural roads, possessing high practical value and good effectiveness, meeting the needs of actual engineering applications, substantially improving data quality, and providing solid data support for the assessment and maintenance management of the technical condition of low-grade rural roads.

[0136] Example 2:

[0137] This invention discloses a rural road condition index data quality repair system based on Stacking ensemble learning, comprising: a preprocessing module for anomaly cleaning of rural road condition index data and original collected data, and classifying the missing patterns of rural road condition index data, including source missing patterns and sub-index missing patterns; a feature set construction module for constructing a fill feature set based on the inherent correlation of defects and pavement decay laws for data with source missing patterns, and for constructing a fill feature set based on pavement decay laws for data with sub-index missing patterns; a feature set dimensionality reduction module for using the Lasso regression algorithm to select variables from the constructed fill feature sets, removing redundant variables to obtain a dimensionality-reduced repair combination; a model training module for selecting base learners of the Stacking ensemble model, splitting the original data into training and testing sets, and training each base learner; and a data imputation module for imputing rural road automated detection index data based on the trained Stacking ensemble model and the dimensionality-reduced repair combination. The specific implementation of each module is described in the above method embodiment and will not be repeated here.

[0138] Example 3:

[0139] This invention discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for restoring the quality of rural road condition index data based on Stacking ensemble learning.

[0140] In this embodiment, the processor can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The memory can include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0141] Those skilled in the art will understand that the content disclosed in the embodiments can be provided as a method, system, or computer program product. Therefore, this solution can take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this solution can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage) containing computer-usable program code.

[0142] This solution is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of this solution. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0145] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0146] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for restoring the quality of rural road condition index data based on Stacking ensemble learning, characterized in that, Includes the following steps: (1) Perform anomaly cleaning on rural road condition index data and original collected data, and classify the missing patterns of rural road condition index data, including source missing patterns and sub-index missing patterns. (2) For data with the source missing pattern, a fill feature set is constructed based on the inherent correlation of the disease and the pavement decay law; for data with the sub-index missing pattern, a fill feature set is constructed based on the pavement decay law. (3) For the constructed imputed feature set, the Lasso regression algorithm is used to select variables, remove redundant variables, and obtain the imputed combination after dimensionality reduction; (4) Select the base learners of the Stacking ensemble model, split the original data into training and test sets, and train each base learner; (5) Based on the trained Stacking ensemble model, fill in the data of rural road automation detection indicators according to the dimension-reduced repair combination; The rural road condition index data mentioned in step (1) includes the historical pavement damage index (PCI), road driving quality index (RQI), and pavement technical condition index (PQI). The total index is PQI, and the sub-indices are PCI and RQI. The original data collected in step (1) is the data used to calculate the rural road condition index, including the damage area data, damage rate (DR), and international roughness index (IRI) related to the pavement damage index (PCI) over the years. For data with missing source patterns, a combined vertical and horizontal repair method is adopted, including: ① filling in the defect area based on the correlation between defects; ② repairing DR or PCI based on the correlation between the non-missing defect area data and the defect rate (DR) or pavement damage index (PCI); ③ filling in PCI, RQI, or PQI data based on the temporal variation patterns of the index data itself. Among these, PCI repair involves multiple variable missing values ​​related to defect area, and is started from the variable with the lowest missing ratio and filled one by one. For data with missing sub-indicator patterns, a vertical repair method is adopted, filling in PCI, RQI, or PQI data based on the temporal variation patterns of the index data itself. The implementation process of step (5) is as follows: (5-1) The prediction results of KNN, MissForest and XGBoost are merged horizontally and used as a new training set to input into the secondary model for training; (5-2) Using the trained Stacking ensemble model, complete the data filling of rural road automation detection indexes based on the dimension-reduced repair combination.

2. The method for restoring the quality of rural road condition index data based on Stacking ensemble learning according to claim 1, characterized in that, The implementation process of step (3) is as follows: (3-1) Standardize the dataset; (3-2) Set the Lasso regression objective function as: ； in, Let be the coefficient vector of Lasso regression. Let j be the j-th feature variable. The number of feature variables, For the observed values, Let be the parameter vector to be estimated. Let j be the parameter of the j-th feature variable. The coefficient for the penalty term. This is a penalty item; (3-3) Calculate the mean squared error using the K-fold cross-validation method, and calculate different... The order in which the influencing factors of each feature value enter the model and the changes in the proportion of the fitting coefficient values; (3-4) Remove the variables whose fitting coefficients are still 0 when the x-axis is 0, and the remaining variables are the repaired combination after dimensionality reduction.

3. The method for restoring the quality of rural road condition index data based on Stacking ensemble learning according to claim 1, characterized in that, The base learner of the Stacking ensemble model described in step (4), including KNN, includes the filling step comprising: (4-1-1) Sort the missing data items from low to high according to the missing percentage; (4-1-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation; (4-1-3) The remaining data item with the lowest missing percentage is used as the prediction target, and the other variables are used as influencing factors. The KNN model is trained using data records without missing values ​​to fill in the missing values. (4-1-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set. (4-1-5) Train the KNN model by iterating through all specified k values, and repeat cross-validation K times for each k value; (4-1-6) Calculate the average of the accuracy of K cross-validations to obtain the model accuracy corresponding to the value of k; (4-1-7) Compare the model accuracy corresponding to all k values ​​to obtain the optimal k value, the corresponding KNN model, and the corresponding imputation results; (4-1-8) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

4. The method for restoring the quality of rural road condition index data based on Stacking ensemble learning according to claim 1, characterized in that, The base learner of the Stacking ensemble model described in step (4), including MissForest, has a filling step comprising: (4-2-1) Sort the missing data items from low to high according to the missing percentage; (4-2-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation; (4-2-3) The remaining data item with the lowest missing percentage is used as the prediction target, and the other variables are used as influencing factors. The MissForest model is trained using data records without missing values ​​to fill in the missing data. (4-2-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set. (4-2-5) Calculate the average accuracy of K cross-validations to obtain the optimal MissForest model and the corresponding imputation results; (4-2-6) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

5. The method for restoring the quality of rural road condition index data based on Stacking ensemble learning according to claim 1, characterized in that, The base learner of the Stacking ensemble model described in step (4), including XGBoost, includes the following filling steps: (4-3-1) Sort the missing data items from low to high according to the missing percentage; (4-3-2) For the data items with the lowest missing rate, the mean imputation method is used for initial imputation; (4-3-3) The remaining data item with the lowest missing proportion is used as the prediction target, and the other variables are used as influencing factors. The XGBoost model is trained using data records without missing values ​​to fill in the missing values. (4-3-4) The original data is randomly divided into K parts. Each time, one part is selected as the training set and the rest are the test set. (4-3-5) The XGBoost model is trained and its hyperparameters are tuned using cross-validation with a pre-set learning rate and maximum depth of each tree. (4-3-6) Calculate the average of the accuracy of K cross-validations to obtain the model accuracy corresponding to each learning rate and the maximum depth of each tree; (4-3-7) Compare the accuracy of all models to obtain the optimal learning rate, the maximum depth of each tree and the corresponding XGBoost model, as well as the corresponding imputation results; (4-3-8) Update the filling matrix with the obtained filling results, and fill the remaining variables in turn until all data is filled.

6. A rural road condition index data quality restoration system based on Stacking ensemble learning, used to implement the rural road condition index data quality restoration method based on Stacking ensemble learning as described in any one of claims 1-5, characterized in that, include: The preprocessing module is used to clean up anomalies in rural road condition index data and raw collected data, and to classify the missing patterns of rural road condition index data, including source missing patterns and sub-index missing patterns. The feature set construction module is used to construct a feature set for data with the source missing pattern based on the inherent correlation of defects and the pavement decay law; and to construct a feature set for data with the sub-index missing pattern based on the pavement decay law. The feature set dimensionality reduction module is used to select variables from the constructed imputed feature set using the Lasso regression algorithm, remove redundant variables, and obtain the dimensionality-reduced imputed combination. The model training module is used to select the base learners for the Stacking ensemble model, split the original data into training and test sets, and train each base learner. And a data incomplete module, which is used to fill in the data of rural road automation detection indicators based on the trained Stacking ensemble model and the dimensionality-reduced repair combination.

7. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method for restoring the quality of rural road condition index data based on Stacking ensemble learning as described in any one of claims 1-5.

Images