Pavement maintenance data abnormal value cleaning method considering time sequence characteristics

Abstract

The invention belongs to the technical field of asphalt pavement maintenance management, and particularly relates to abnormal value identification according to characteristics of maintenance data and a quartile spacing method to obtain a normal data set and an abnormal data set after abnormal maintenance measures are removed; constructing a long short-term memory network model, and finding a group of optimal hyper-parameters by using a quintuple cross validation grid search method; and training by using the LSTM model and the collected normal data set to obtain the optimal weight and offset parameter of the model, and obtaining an asphalt pavement maintenance engineering data prediction model for predicting and filling the vacancy in the data set. The invention provides a pavement maintenance data abnormal value cleaning method considering time sequence characteristics. According to the method, reference is provided for asphalt pavement management data cleaning research, the availability of data in a pavement management system is improved, and a decision maker can be helped to formulate a scientific and reasonable pavement maintenance scheme.

Description

Technical Field

[0001] The present invention belongs to the technical field of asphalt pavement maintenance management, and in particular relates to a method for cleaning abnormal values ​​of pavement maintenance data taking into account time series characteristics. Background Art

[0002] With the continuous expansion of my country's road network, the amount of road maintenance projects has also increased significantly. However, the annual maintenance funds and resources are limited, and it is difficult to meet the growing maintenance needs. Therefore, how to use limited maintenance funds scientifically and rationally and optimize maintenance plans has become a key issue in road management and maintenance. To this end, it is necessary to rely on complete and high-quality pavement maintenance data to support the scientific formulation of maintenance decisions. However, due to manual entry errors, measurement equipment failures, errors in statistical processing, etc., road maintenance data often have a series of outliers. The existence of outliers will affect the accuracy of maintenance decisions. Therefore, establishing an accurate outlier cleaning model for road pavement maintenance data has become a key basis for obtaining high-quality maintenance data.

[0003] At present, most of the existing outlier cleaning models are based on artificial neural networks, but these models mainly focus on the spatial characteristics of the data, while ignoring the time series characteristics of road maintenance data itself. In contrast, the outlier cleaning model based on LSTM (Long Short-Term Memory Network) can better capture the temporal patterns in the data, and thus significantly improve the accuracy of the prediction results. The advantage of this model is that it can fully consider the temporal nature of the data, thereby improving the accuracy of the prediction and providing more reliable data support for maintenance decisions. Summary of the invention

[0004] The purpose of the present invention is to provide a method for cleaning outliers in pavement maintenance data taking into account time series characteristics, so as to solve the problems raised in the background technology.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for cleaning outliers in pavement maintenance data considering time series characteristics comprises the following steps:

[0007] S1: Identify outliers based on the characteristics of maintenance data and the interquartile range method to obtain a normal data set and an abnormal data set after eliminating abnormal maintenance measures;

[0008] S2: Build an LSTM model and use a five-fold cross-validation grid search method to find a set of optimal hyperparameters;

[0009] S3: Use the LSTM model and the collected normal data set to train the model to obtain the optimal weight and bias parameters, and obtain the asphalt pavement maintenance engineering data prediction model.

[0010] The specific method of constructing the LSTM model in step S2 and finding a set of optimal hyperparameters using a five-fold cross-validation grid search method includes the following steps:

[0011] S21: Determine model hyperparameters;

[0012] Specifically, model hyperparameters include: loss function, learning rate, batch size, sliding window size, number of iterations, number of hidden layer units, and the choice of optimizer;

[0013] S22: Build LSTM model;

[0014] The model includes: data reading, feature and label extraction, and then using StandardScaler to standardize the feature data. The formula is:

[0015]

[0016] Where z is the standardized value; x is the input feature value; μ is the mean value; σ is the standard deviation;

[0017] S23: Divide the normal data set into five equal parts. Five of the data are used as validation data in turn during training, and the remaining four are used as training data. The average loss of model validation is taken as a reference. Specifically, KFold is used for five-fold cross validation and model training. Finally, the average loss value of each fold is recorded.

[0018] S24: Compare the average loss of the training results. The combination with the smallest average loss is the best hyperparameter combination of the model.

[0019] S25: Divide the normal data set used for training into a training data set and a test data set according to a ratio of 4:1;

[0020] S26: Use the above hyperparameters to build an LSTM model, set different numbers of iterations for model training, and use the model loss value and prediction accuracy on the test set to evaluate the model performance.

[0021] S27: Stop training when the performance becomes stable, obtain the optimal weight and bias parameters of the model and save them.

[0022] The pavement management data in step S1 include: road age, base thickness, base material, surface thickness, surface material, annual average daily traffic volume of trucks, annual average daily traffic volume, pavement condition index, pavement condition index of the past two years and maintenance data;

[0023] Identify outliers in the data, including three abnormal situations:

[0024] The first type is that there is no maintenance record, but the PCI value becomes larger when the road condition is inspected that year;

[0025] The second type has maintenance records and the PCI value decreases;

[0026] The third type is that there is a maintenance record and the PCI increases, but the increase in the PCI value is not within the normal range;

[0027] The first two abnormal measures can be directly identified and eliminated, while the third abnormal situation uses the interquartile range method to calculate a reasonable value-added range, and then eliminates the data outside the range. The formula of the interquartile range method is as follows:

[0028] IQR=Q3-Q1

[0029] Where: IQR is the interquartile range; Q3 is the 75th percentile; Q1 is the 25th percentile. Data points outside the range of [Q1-1.5IQR, Q3+1.5IQR] are considered outliers. If Q1-1.5IQR is less than 0, the reasonable interval is (0, Q3+1.5IQR].

[0030] Abnormal maintenance measures are identified and eliminated based on abnormal changes in the pavement condition index and the interquartile range method, and the data set is divided into a normal data set and an abnormal data set.

[0031] In step S22, LSTM is mainly controlled by three “gates”;

[0032] First, the forget gate determines the storage and discarding of information. The formula is:

[0033] f t =σ(w f ·[h t-1 ,x t ]+b f )

[0034] Secondly, input gate i t Decide on adding new information to the cell state, including the sigmoid layer and the tanh layer. t-1 Updated to c t The formula is:

[0035] i t =σ(w i ·[h t-1 ,x t ]+b t )

[0036] c′ t =tanh(w c ·[h t-1 ,x t ]+bc )

[0037] c t =f t c t-1 +i t c′ t

[0038] Finally, the output gate is based on the new cell state c t , the neuron state h at the previous moment t-1 , and the current input X t The formula for determining the output value is:

[0039] o t =σ(w0·[h t-1 ,x t ]+b o )

[0040] h t =o t tanhc t

[0041] Where: c′ t is the input unit state at time t; w f 、w i 、w o and b f 、b i 、b o are the weights and bias parameters of the forget gate, input gate, and output gate; t is the output of the neural network at time t; σ and tanh are the corresponding activation functions.

[0042] The present invention is a method for cleaning outliers of pavement maintenance data considering time series characteristics. By identifying outliers based on the characteristics of the maintenance data itself and the interquartile range method, a normal data set and an abnormal data set after eliminating abnormal maintenance measures are obtained; an LSTM model is constructed, and a set of optimal hyperparameters are found using a five-fold cross-validation grid search method; the LSTM model is used to train the collected normal data set to obtain the optimal weights and bias parameters of the model, and a road pavement maintenance engineering data prediction model is obtained to fill in the maintenance measures that are missing in the abnormal data set. The present invention proposes a method for cleaning outliers of pavement maintenance data considering time series characteristics. This method can better capture the temporal characteristics of the data, further improve the prediction accuracy, increase the availability of data in the pavement management system, help managers formulate scientific maintenance plans, solve the problem of insufficient accuracy of the existing asphalt pavement maintenance engineering data cleaning method, and also contribute to the rational allocation of maintenance funds. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention is further illustrated by means of the non-limiting examples given in the accompanying drawings.

[0044] Figure 1 A framework diagram of a method for cleaning outliers in pavement maintenance data taking into account time series characteristics provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the LSTM network model structure mentioned in the present invention;

[0046] Figure 3 This is a graph showing the change in loss value and accuracy during the LSTM model training process of the present invention;

[0047] Figure 4 This is a comparison chart of the loss value and accuracy based on the LSTM model and the artificial neural network model of the present invention;

[0048] Figure 5 A flow chart of a method for cleaning outliers in pavement maintenance data taking into account time series characteristics provided by the present invention;

[0049] Figure 6 Flowchart for finding an optimal set of model hyperparameter steps and training the model to obtain the best weight and bias parameters. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the present invention, the technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0051] like Figure 1-6 As shown, a method for cleaning outliers in pavement maintenance data considering time series characteristics of the present invention comprises the following steps:

[0052] S1: Identify outliers based on the characteristics of maintenance data and the interquartile range method to obtain a normal data set and an abnormal data set after eliminating abnormal maintenance measures;

[0053] Specifically, the pavement management data include: road age, base thickness, base material, surface thickness, surface material, annual average daily truck traffic volume, annual average daily traffic volume, pavement condition index, pavement condition index of the past two years and maintenance data.

[0054] Outlier identification is performed on the data, including three abnormal situations: the first is that there is no maintenance record, but the PCI value increases during the road condition inspection of the year; the second is that there is a maintenance record, but the PCI value decreases; the third is that there is a maintenance record, the PCI increases, but the increase in the PCI value is not in the normal range (too large or too small).

[0055] The first two abnormal measures can be directly identified and eliminated, while the third abnormal situation uses the interquartile range method to calculate a reasonable value-added range, and then eliminates the data outside the range. The formula of the interquartile range method is as follows:

[0056] IQR=Q3-Q1

[0057] Where: IQR is the interquartile range; Q3 is the 75th percentile; Q1 is the 25th percentile. Data points outside the range of [Q1-1.5IQR, Q3+1.5IQR] are considered outliers. If Q1-1.5IQR is less than 0, the reasonable interval is (0, Q3+1.5IQR].

[0058] S2: Build an LSTM model and use a five-fold cross-validation grid search method to find a set of optimal hyperparameters;

[0059] The specific method is:

[0060] S21: Determine model hyperparameters;

[0061] Specifically, the model hyperparameters include: loss function, learning rate, batch size, sliding window size, number of iterations, number of hidden layer units, and optimization selection, etc. Among them, hidden layer 1, hidden layer 2, and training batch take 5 values ​​as candidate items respectively.

[0062] S22: Build LSTM model;

[0063] Specifically, the model includes: data reading, feature and label extraction, and then using StandardScaler to standardize the feature data. The formula is:

[0064]

[0065] Where z is the standardized value; x is the input feature value; μ is the mean value; σ is the standard deviation.

[0066] Customize the dataset (RoadMaintenanceDataset class), encapsulate the data and labels, set the sliding window; define the LSTM model, and apply the dropout layer after the LSTM layer to randomly discard neurons to prevent the model from overfitting; use the optimizer to train the model, and update the model weights through backpropagation. LSTM is mainly controlled by three "gates". First, the forget gate determines the storage and discarding of information. The formula is:

[0067] f t =σ(w f ·[h t-1 ,x t ]+b f )

[0068] Secondly, input gate i t Decide on adding new information to the cell state, including the sigmoid layer and the tanh layer. t-1 Updated to c t The formula is:

[0069] i t =σ(w i ·[h t-1 ,x t ]+b t )

[0070] c′ t =tanh(w c ·[h t-1 ,x t ]+b c )

[0071] c t =f t c t-1 +i t c′ t

[0072] Finally, the output gate is based on the new cell state c t , the neuron state h at the previous moment t-1 , and the current input X t The formula for determining the output value is:

[0073] o t =σ(w0·[h t-1 ,x t ]+b o )

[0074] h t =o t tanhc t

[0075] Where: c′ t is the input unit state at time t; w f 、w i 、w o and b f 、b i 、b o are the weights and bias parameters of the forget gate, input gate, and output gate; t is the output of the neural network at time t; σ and tanh are the corresponding activation functions.

[0076] S23: Divide the normal data set into five equal parts. Five of the data are used as validation data in turn during the training process, and the remaining four are used as training data. The average loss of the model validation is taken as a reference.

[0077] Specifically, KFold is used to perform five-fold cross validation and model training; finally, the average loss value of each fold is recorded.

[0078] S24: Compare the average loss of the training results. The combination with the smallest average loss is the best hyperparameter combination of the model.

[0079] S25: Divide the normal data set used for training into a training data set and a test data set according to a ratio of 4:1;

[0080] S26: Use the above hyperparameters to build an LSTM model and set different numbers of iterations to train the model. Use the model loss value and prediction accuracy on the test set to evaluate the model performance.

[0081] S27: Stop training when the performance becomes stable, obtain the optimal weight and bias parameters of the model and save them.

[0082] S3: Use the LSTM model and the collected normal data set to train the model to obtain the optimal weight and bias parameters, and obtain the asphalt pavement maintenance engineering data prediction model.

[0083] Example

[0084] This embodiment takes the highway in Shanxi Province as an example, and the data comes from the highway pavement management data of some areas in Shanxi Province from 2017 to 2019. The data types mainly include maintenance engineering data, pavement conditions, pavement structure material data, traffic volume data and road age. Specifically including: unmaintained, low-noise anti-skid fine surface (Thin Friction Course, TFC), MS-3 type micro-surfacing, fine surface, super-sticky wear layer, SMC ultra-thin wear layer, in-situ hot regeneration, milling and resurfacing one layer, PCI, annual average daily traffic volume, annual average daily traffic volume of trucks, base layer thickness, surface layer thickness, base layer material, surface layer material and road construction time.

[0085] According to the changes in PCI, outliers in the collected data are identified and eliminated. The first is that the PCI value increases in two consecutive years without maintenance records; the second is that the PCI value decreases in two consecutive years with maintenance records; the third is that the change in PCI value in two consecutive years with maintenance records is unreasonable. In the first two cases, outliers are directly eliminated, and in the third case, the interquartile range method is used to identify and eliminate abnormal data. The reasonable range of PCI value-added corresponding to each maintenance measure is calculated, and data outside the range is regarded as abnormal data. The obtained reasonable PCI value-added range is shown in Table 1.

[0086] Table 1 PCI reasonable value-added range

[0087]

[0088] A total of 2,868 normal data samples were collected, and each road section unit had a road surface management data set each year from 2017 to 2019. These data were grouped into groups of three in chronological order, and finally 956 groups of normal data were formed. The groups were randomly divided into 765 training data sets and 191 test data sets according to a 4:1 ratio. After many attempts, the LSTM model structure is shown in the attached figure. Figure 2 The hyperparameters are: 2 LSTM layers, 0.01 learning rate, Adam optimizer, 0.3 dropout layer size, and 3 sliding window size. Figure 3 When the model iteration number is 100, the performance tends to be stable, so the iteration number is set to 100. The forget gate, input gate and output gate use the Sigmoid function as the activation function, and the formula is:

[0089]

[0090] When updating the unit state, the hyperbolic tangent function tanh is used as the activation function, and the formula is:

[0091]

[0092] The number of neurons in the model LSTM layer 1 and the model LSTM layer 2 is empirically set to 7, 14, 28, 42, and 56 as five candidate items, and the training batch size is set to 60, 120, 180, 240, and 300 as five candidate items. The five-fold cross-validation method is used to divide the data set into five equal subsets, and one of the subsets is taken as the validation set each time, and the other four are taken as the training set. By calculating the average loss value of five evaluations under each set of hyperparameters, the overall performance index of the model under the hyperparameter combination is obtained. The number of nodes and the training batch size with the minimum loss value are selected as the hyperparameter combination of the model. LSTM layer 1, LSTM layer 2, and training batch are 42, 7, and 180, respectively. The loss function is the CrossEntropyLoss function, and the formula is as follows:

[0093]

[0094] Where: I represents the number of model output values, i represents the sequence number of these values; Zi is the i-th output value of the output layer; N is the batch size, where n is the sequence number of the samples in the batch, and yn represents the true label of the n-th sample. The final loss value is represented by Loss, which represents the error of the model.

[0095] For comparison purposes, an artificial neural network (ANN) with two hidden layers is established. Its forward propagation formula is:

[0096] Y=W3 A2(W 2 A1(W 1 X))

[0097] Where: X is the input feature; W1 is the weight matrix between the input layer and hidden layer 1; A1 is the activation function of the output of hidden layer 1; W2 is the weight matrix between hidden layer 1 and hidden layer 2; A2 is the activation function of the output of hidden layer 2; W3 is the weight matrix between hidden layer 2 and the output layer. Y is the model output value.

[0098] The same method is used to select the hyperparameters of the ANN model. The CrossEntropyLoss function is also selected to calculate the model loss value, and the loss value is used to evaluate the model performance.

[0099] After model training, the comparison of loss value and prediction accuracy of LSTM model and ANN model on the test set is shown in the figure below. Figure 4 , the former model performance is obviously better than the latter. This proves the superiority of the LSTM-based asphalt road pavement maintenance data outlier cleaning method proposed in the present invention. Finally, the road maintenance data cleaning method based on the LSTM model was used to clean the data road set with 31% outliers, and the outlier ratio dropped to 13.6%, which illustrates the rationality of the present invention.

[0100] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects: the present invention considers the time correlation of road section management data and establishes a method for filling outliers in asphalt pavement maintenance engineering data. Before formally establishing the prediction method, a five-fold cross-validation method is used to search for a set of optimal hyperparameter combinations for the LSTM model, thereby constructing a maintenance measure prediction model with better performance. This provides a certain reference for the research on repairing outliers in road maintenance measures in pavement management data.

[0101] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for cleaning outliers in pavement maintenance data considering time series characteristics, characterized in that; The following steps are involved: S1: Identify outliers based on the characteristics of maintenance data and the interquartile range method to obtain a normal data set and an abnormal data set after eliminating abnormal maintenance measures; S2: Build an LSTM model and use a five-fold cross-validation grid search method to find a set of optimal hyperparameters; S3: Use the LSTM model and the collected normal data set to train the model to obtain the optimal weight and bias parameters, and obtain the asphalt pavement maintenance engineering data prediction model. The specific method of constructing the LSTM model in step S2 and finding a set of optimal hyperparameters using a five-fold cross-validation grid search method includes the following steps: S21: Determine model hyperparameters; Specifically, model hyperparameters include: loss function, learning rate, batch size, sliding window size, number of iterations, number of hidden layer units, and the choice of optimizer; S22: Build LSTM model; The model includes: data reading, feature and label extraction, and then using StandardScaler to standardize the feature data. The formula is: Where z is the standardized value; x is the input feature value; μ is the mean value; σ is the standard deviation; S23: Divide the normal data set into five equal parts. Five of the data are used as validation data in turn during training, and the remaining four are used as training data. The average loss of model validation is taken as a reference. Specifically, KFold is used for five-fold cross validation and model training. Finally, the average loss value of each fold is recorded. S24: Compare the average loss of the training results. The combination with the smallest average loss is the best hyperparameter combination of the model. S25: Divide the normal data set used for training into a training data set and a test data set according to a ratio of 4:1; S26: Use the above hyperparameters to build an LSTM model, set different numbers of iterations for model training, and use the model loss value and prediction accuracy on the test set to evaluate the model performance; S27: Stop training when the performance becomes stable, obtain the optimal weight and bias parameters of the model and save them.

2. According to claim 1, a method for cleaning outliers in pavement maintenance data taking into account time series characteristics, It is characterized by: The pavement management data in step S1 include: road age, base thickness, base material, surface thickness, surface material, annual average daily traffic volume of trucks, annual average daily traffic volume, pavement condition index, pavement condition index of the past two years and maintenance data; Identify outliers in the data, including three abnormal situations: The first type is that there is no maintenance record, but the PCI value becomes larger when the road condition is inspected that year; The second type has maintenance records and the PCI value decreases; The third type is that there is a maintenance record and the PCI increases, but the increase in the PCI value is not within the normal range; The first two abnormal measures can be directly identified and eliminated, while the third abnormal situation uses the interquartile range method to calculate a reasonable value-added range, and then eliminates the data outside the range. The formula of the interquartile range method is as follows: IQR=Q3-Q1 Where: IQR is the interquartile range; Q3 is the 75th percentile; Q1 is the 25th percentile. Data points outside the range of [Q1-1.5IQR, Q3+1.5IQR] are considered outliers. If Q1-1.5IQR is less than 0, the reasonable interval is (0, Q3+1.5IQR].

3. The method for cleaning outliers in pavement maintenance data considering time series characteristics according to claim 1 is characterized in that ； Abnormal maintenance measures are identified and eliminated based on abnormal changes in the pavement condition index and the interquartile range method, and the data set is divided into a normal data set and an abnormal data set.

4. The method for cleaning outliers in pavement maintenance data considering time series characteristics according to claim 1, characterized in that; In step S22, LSTM is mainly controlled by three "gates"; First, the forget gate determines the storage and discarding of information. The formula is: f t =σ(w f ·[h t-1 ,x t ]+b f ) Secondly, the input gate i t Decide on adding new information to the cell state, including sigmoid layer and tanh layer. t-1 Updated to c t The formula is: i t =σ(w i ·[h t-1 ,x t ]+b t ) c' t =tanh(w c ·[h t-1 ,x t ]+b c ) c t =f t c t-1 +i t c′ t Finally, the output gate is based on the new cell state c t , the neuron state h at the previous moment t-1 , and the current input X t The formula for determining the output value is: oh t =σ(w0·[h t-1 ,x t ]+b o ) h t =oh t thank you t Where: c′ t is the input unit state at time t; w f 、w i 、w o and b f 、b i 、b o are the weights and bias parameters of the forget gate, input gate, and output gate; t is the output of the neural network at time t; σ and tanh are the corresponding activation functions.

Images