Disease detection classification and crack identification method for asphalt pavement

Abstract

The invention relates to an asphalt pavement disease detection classification and crack identification method, and relates to the field of traffic pavement detection. The method comprises the steps that a road surface of a road is photographed through a road detection vehicle, and photos are arranged; labeling the image data by using label software to form a picture sample; converting the marked picture sample into a format available for the model, and dividing the picture sample according to a certain proportion; building a pixel segmentation model to train the data, and selecting an optimal model by adjusting parameters; and the model is used for carrying out segmentation processing on unmarked asphalt pavement crack diseases for later-stage area calculation.

Description

Technical Field

[0001] The invention relates to an asphalt pavement disease detection classification and crack identification method, and relates to the field of traffic pavement detection. Background Art

[0002] my country has a vast territory, and highways run through more than 9.6 million square kilometers of land across the country. They are large in scale and widely distributed. In recent years, with the continuous expansion of the highway network and the concentration of a large number of infrastructures entering the middle and late stages of service, the mileage of highways that need maintenance each year has also continued to increase. The existence of diseases not only affects the appearance of the road surface, the smoothness of the road, and the satisfaction of residents' travel, but also may cause traffic accidents if the diseases are not discovered and repaired in time, causing irreparable losses. Every year, a comprehensive road damage inspection is required for national highways, including expressways and first to fourth-level highways. Pavement cracks are the most common and most influential disease. Due to the large mileage and wide range of highways, it is cumbersome and complicated to conduct a comprehensive investigation and statistics on pavement crack diseases and calculate relevant indicators.

[0003] Deep learning is the most advanced branch of machine learning. Its concept originates from the research of artificial neural networks. Its advantage is that the input end only needs to directly input the image, and the subsequent feature extraction and image classification work completely rely on the self-training of the network, which is more suitable for large-scale image processing.

[0004] Pixel segmentation is performed on the most common and difficult-to-detect pavement cracks, that is, the input is a pavement image, and the output is an image of the same size, and the color can be customized to depict the cracks. Using the local pixel segmentation model, the detected crack disease information can be quantified, such as crack length, width, area, etc., and the pavement damage condition can be further assessed in combination with existing specifications. Summary of the invention

[0005] The present invention utilizes deep learning to optimize image processing, builds a crack recognition model, and provides an asphalt pavement disease detection classification and crack recognition method.

[0006] The present invention adopts the following technical solution:

[0007] 1. A method for detecting, classifying and identifying cracks in asphalt pavement, characterized in that the steps are as follows:

[0008] Step 1: Take photos of the road surface by a road inspection vehicle and organize the photos;

[0009] Step 2: Use labelme software to label the image data to form image samples;

[0010] Step 3: Convert the labeled image samples into a format that can be used by the model and divide them according to a certain ratio;

[0011] Step 4: Build a pixel segmentation model to train the data and select the best model by adjusting the parameters;

[0012] Step 6: Use the model in step 5 to segment the unmarked asphalt pavement cracks for later area calculation.

[0013] In the asphalt pavement disease detection classification and crack identification method described in the present invention, the architecture of the pixel segmentation model in step 4 is built based on the improved U-Net network, and the pixel segmentation loss function is constructed, which is expressed as follows:

[0014]

[0015] Where log is the natural logarithm; y i is the true label of the i-th sample. For the binary classification problem with only background and crack, y i The value is 0 or 1. If sample i is a positive sample, that is, a crack, then y i =1, otherwise 0; p i is the probability that the model predicts that the i-th sample is a positive sample; w i is the weight of the i-th sample, which is used to deal with the imbalance problem of background and crack samples. N represents the number of input samples, that is, the average loss of all input samples is calculated;

[0016] The training is carried out with the goal of minimizing the pixel segmentation loss function, and the gradient back propagation is used as the strategy to continuously update the parameters of the current pixel segmentation model until the training of all iterative cycles is completed;

[0017] The sample weight w through the loss function i , and the number of epochs of network training to select the model with the best relative effect.

[0018] In the asphalt pavement disease detection classification and crack identification method of the present invention, the evaluation method of the best model in step 4 is based on the training model, ACC index and F1 index:

[0019] The calculation method of ACC is as follows:

[0020]

[0021] Where TP is the true positive example, that is, the number of pixels correctly predicted by the model as positive; TN is the true negative example, that is, the number of pixels correctly predicted by the model as negative; FP is the false positive example, that is, the number of pixels incorrectly predicted by the model as positive; FN is the false negative example, that is, the number of pixels incorrectly predicted by the model as negative.

[0022] The calculation method of F1 is as follows:

[0023]

[0024] Where Precision is the proportion of positive categories predicted to be positive.

[0025] The precision calculation method is:

[0026]

[0027] Recall is the proportion of positive categories predicted to be positive in the actual positive categories. The calculation method of Recall is:

[0028]

[0029] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: it combines the advanced achievements of semantic segmentation in the field of computer vision to achieve high-precision automatic identification of pavement cracks. Labelme software is used to label image data, which ensures the accuracy and consistency of sample data and provides a high-quality training data set for subsequent model training; by converting the format and dividing the data of the labeled image samples, and building a pixel segmentation model, the present invention can systematically train the model and adjust the parameters to select the best model, thereby improving the accuracy and robustness of the model; by using the best model to segment the unlabeled asphalt pavement crack diseases, the present invention can accurately identify crack diseases and provide reliable data support for subsequent disease area calculation and maintenance decisions. The disease detection segmentation provided by the present invention enables pavement maintenance work to be carried out more promptly, reducing traffic safety hazards and potential greater maintenance costs caused by the deterioration of diseases. The data provided by the present invention can be used to support the decision-making process of pavement maintenance and planning, making the decision more scientific and data-driven. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flow chart of the asphalt pavement disease detection classification and crack identification method of the present invention;

[0031] Figure 2 This is an example diagram of the environment for creating the asphalt pavement disease detection classification and crack identification method of the present invention;

[0032] Figure 3 This is an example diagram of the results of arranging photos taken by a road inspection vehicle for the asphalt pavement disease detection classification and crack identification method of the present invention;

[0033] Figure 4 This is an example diagram of image annotation of the asphalt pavement disease detection classification and crack identification method of the present invention;

[0034] Figure 5This is an example diagram of sample data processing of the asphalt pavement disease detection classification and crack identification method of the present invention;

[0035] Figure 6 This is an example diagram of the model structure of the asphalt pavement disease detection classification and crack identification method of the present invention;

[0036] Figure 7 This is an example diagram of the model parameter adjustment results of the asphalt pavement disease detection classification and crack identification method of the present invention;

[0037] Figure 8 This is an example diagram of the model recognition results of the asphalt pavement disease detection classification and crack identification method of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose and technical solution of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] like Figure 1 As shown, a method for automatic segmentation and identification of cracks in asphalt pavement based on deep learning, the method comprises the following steps:

[0040] Step S1: Create a new environment for the project, which is the environment for subsequent training models;

[0041] Step S2: sorting the road surface photos taken by the road inspection vehicle and creating a crack disease folder required for training;

[0042] Step S3: using labelme software to label the cracks in the image data and depict the cracks;

[0043] Step S4: convert the labeled samples into a format that can be used by the model and divide them according to a certain ratio;

[0044] Step S5: Building a pixel segmentation model to train the data;

[0045] Step S6: Select the best model by adjusting parameters;

[0046] Step S7: Use the model to segment the unmarked asphalt pavement cracks to facilitate subsequent area calculation.

[0047] like Figure 2As shown, create a new environment for the project and name it, and assign an existing Python version. When the question "Do you want to continue?" appears (Proceed([y] / n)?"), enter y to continue. Waiting for the installation to be completed indicates that the creation is successful.

[0048] like Figure 3 As shown, the road surface photos taken by the road inspection vehicle are preliminarily screened so that the data covers as many as possible various road crack diseases, totaling about 500 photos. The screened road surface photos are placed in a special folder to prepare for subsequent labeling work.

[0049] like Figure 4 As shown in the figure, the labelme software is used to annotate crack-type diseases (including transverse and longitudinal cracks) at the pixel level to depict the complete crack morphology.

[0050] like Figure 5 As shown in the figure, after the label assignment is completed, in order to ensure the normal progress of subsequent model training, the label format needs to be converted. Furthermore, all labels need to be divided into training sets, validation sets, and test sets according to the requirements of model training. All of the above are completed by running the corresponding code.

[0051] Building a training model in the project includes the following steps:

[0052] (1) Select Figure 2 Install the required toolkits in the built environment and complete the environment configuration;

[0053] (2) Build the network model required for training, such as Figure 6 As shown;

[0054] (3) Construct a pixel segmentation loss function, whose specific calculation formula is:

[0055]

[0056] Where log is the natural logarithm; y i is the true label of the i-th sample. For the binary classification problem with only background and crack, y i The value is 0 or 1. If sample i is a positive sample, that is, a crack, then y i =1, otherwise 0; p i is the probability that the model predicts that the i-th sample is a positive sample; w i is the weight of the i-th sample, which is used to deal with the imbalance problem between background and crack samples. N represents the number of input samples, that is, the average loss of all input samples is calculated.

[0057] (4) Select the sample label data set in steps S2-S4 for training. Figure 5The file path prepared in is used as the training path;

[0058] (5) Specify basic training parameters, such as loss function weight, number of training rounds, number of samples selected for one training, etc.

[0059] (6) Run the code to train with the goal of minimizing the pixel segmentation loss function and gradient backpropagation as the strategy to continuously update the parameters of the current pixel segmentation model until all iteration cycles of training are completed.

[0060] like Figure 7 As shown, by adjusting the parameters in step S6, including the loss function weight and the number of training rounds, repeatedly running the training code, comparing the relevant indicators of the training results, and selecting the model corresponding to the optimal result.

[0061] (1) For the evaluation of the training model, the ACC index is calculated and compared. The ACC index reflects the classification accuracy of the model on all pixels, that is, the proportion of pixels correctly predicted by the model to the total pixels. Specifically, the calculation method of ACC is as follows:

[0062]

[0063] Where TP is the true positive example, that is, the number of pixels correctly predicted by the model as positive; TN is the true negative example, that is, the number of pixels correctly predicted by the model as negative; FP is the false positive example, that is, the number of pixels incorrectly predicted by the model as positive; FN is the false negative example, that is, the number of pixels incorrectly predicted by the model as negative.

[0064] (2) For the evaluation of the training model, the F1 index is calculated and compared. The F1 index combines the performance of precision and recall. Compared with the ACC index, F1 can provide a fairer performance evaluation in the case of class imbalance. Specifically, the calculation method of F1 is as follows:

[0065]

[0066] In the formula, Precision is the proportion of positive categories in the predicted positive categories, and the calculation method is Recall is the ratio of the predicted positive category to the actual positive category, and is calculated as

[0067] like Figure 8 As shown, the trained target detection model is used to Figure 5 The images in the test set are identified, the crack damage is segmented by pixels, and the crack morphology is depicted, which is convenient for the subsequent statistical calculation of the road surface damage area.

[0068] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for detecting, classifying and identifying cracks in asphalt pavement, characterized by: Here are the steps: Step 1: Take photos of the road surface by a road inspection vehicle and organize the photos; Step 2: Use labelme software to label the image data to form image samples; Step 3: Convert the labeled image samples into a format that can be used by the model and divide them according to a certain ratio; Step 4: Build a pixel segmentation model to train the data and select the best model by adjusting the parameters; Step 6: Use the model in step 5 to segment the unmarked asphalt pavement cracks for later area calculation.

2. The asphalt pavement disease detection, classification and crack identification method according to claim 1 is characterized by: The architecture of the pixel segmentation model in step 4 is built based on the improved U-Net network, and the pixel segmentation loss function is constructed, which is expressed as follows: Where log is the natural logarithm; y i is the true label of the i-th sample. For the binary classification problem with only background and crack, y i The value is 0 or 1. If sample i is a positive sample, that is, a crack, then y i =1, otherwise 0; p i is the probability that the model predicts that the i-th sample is a positive sample; w i is the weight of the i-th sample, which is used to deal with the imbalance problem of background and crack samples. N represents the number of input samples, that is, the average loss of all input samples is calculated; The training is carried out with the goal of minimizing the pixel segmentation loss function, and the gradient back propagation is used as the strategy to continuously update the parameters of the current pixel segmentation model until the training of all iterative cycles is completed; The sample weight w through the loss function i , and the number of epochs of network training to select the model with the best relative effect.

3. The asphalt pavement disease detection classification and crack identification method according to claim 1 is characterized by: The evaluation method of the best model in step 4 is based on the training model, ACC index, and F1 index: The calculation method of ACC is as follows: Where TP is the true positive example, that is, the number of pixels correctly predicted by the model as positive; TN is the true negative example, that is, the number of pixels correctly predicted by the model as negative; FP is the false positive example, that is, the number of pixels incorrectly predicted by the model as positive; FN is the false negative example, that is, the number of pixels incorrectly predicted by the model as negative. The calculation method of F1 is as follows: In the formula, Precision is the proportion of positive categories in the predicted positive categories. The calculation method of Precision is: Recall is the proportion of positive categories predicted to be positive in the actual positive categories. The calculation method of Recall is:

Images