Pavement crack identification method and model training method based on improved DeeplabV3 + and morphology

Abstract

The invention relates to a pavement crack identification method based on improved DeeplabV < 3 + > and morphology, a model training method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a pavement crack detection training data set; preprocessing the sample images in the pavement crack detection training data set by adopting a Canny algorithm and a morphological method to obtain a training image of each sample image in the pavement crack detection training data set; generating a pavement crack detection marking training data set based on MATLAB; and constructing a pavement crack recognition network structure which takes the sample image containing the pavement cracks as input and takes the pavement crack recognition result as output through DeepLabV3 +, training by adopting the pavement crack detection labeling training data set, and outputting a pavement crack recognition model. By adopting the method, the accuracy of crack detection can be improved while image details are reserved.

Description

Technical Field

[0001] The present application relates to the field of road engineering technology, and in particular to a pavement crack identification method, model training method, device, computer equipment, storage medium and computer program product based on improved DeeplabV3+ and morphology. Background Art

[0002] Pavement cracks, as a common but seriously underestimated disease, can lead to the destruction and weakening of the pavement structure, causing the pavement to undergo greater deformation and displacement when subjected to vehicle loads.

[0003] At present, many concrete pavement crack automatic detection technologies still rely on traditional fully convolutional neural network (FCN) models, which have limitations in recognition efficiency and are not easy to deploy on lightweight platforms due to the complexity of the model. Although a few technologies have adopted improved fully convolutional neural networks, there is still the problem of information loss. This information loss limits the comprehensiveness of the model in extracting image features and affects the ability to exclude false cracks in complex environments. Therefore, it is necessary to further improve the crack segmentation effect and detection accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a pavement crack identification method, model training method, device, computer equipment, storage medium and computer program product based on improved DeeplabV3+ and morphology, which can significantly improve data utilization efficiency and recognition accuracy, so as to more effectively manage the health status of the road surface.

[0005] In a first aspect, the present application provides a training method for a pavement crack recognition model based on improved DeeplabV3+ and morphology, comprising:

[0006] Step S1, obtaining a pavement crack detection training data set, where the pavement crack detection training data set includes at least one sample image containing pavement cracks;

[0007] Step S2, using the Canny algorithm and the morphological method to preprocess the sample images in the pavement crack detection training data set, wherein the preprocessing includes using the Canny algorithm to perform edge detection on the sample images, and using the morphological method to perform corrosion and expansion on the sample images processed by the Canny algorithm, so as to obtain a training image of each sample image in the pavement crack detection training data set;

[0008] Step S3, annotating the pavement cracks in the sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection annotated training data set;

[0009] Step S4, constructing a pavement crack recognition network structure using DeepLabV3+ with sample images containing pavement cracks as input and pavement crack recognition results as output, and training the pavement crack recognition network structure using a pavement crack detection and annotation training data set to output a pavement crack recognition model.

[0010] In one embodiment, step S2 comprises:

[0011] The Canny algorithm is used to perform edge detection on the sample images in the pavement crack detection training data set, remove noise in the sample images and retain the pavement crack edges, and generate a gradient map of the sample images; the gradient amplitude and direction in the gradient map of the sample images are calculated, and non-maximum suppression is performed on the sample images; the low threshold and high threshold of the Canny algorithm are adjusted, and the sample images are processed using the adjusted Canny algorithm to obtain binary images of the sample images in the pavement crack detection training data set;

[0012] The morphological method is used to perform corrosion and expansion processing on the binary image of the sample image processed by the Canny algorithm to obtain the training image of each sample image in the pavement crack detection training data set.

[0013] In one embodiment, before step S4, the method further includes:

[0014] The network architecture mobilenetV2 is used as a feature extractor to extract features from the pavement crack detection and annotation training dataset, and the features are mapped to DeepLabV3+.

[0015] In one embodiment, step S4 further includes:

[0016] Obtain a pavement crack detection test dataset;

[0017] The pavement crack detection test data set is input into the pavement crack recognition model, and the pavement crack recognition model is evaluated using pavement crack recognition indicators, where the pavement crack recognition indicators include IOU, PA, and RA.

[0018] In a second aspect, the present application further provides a pavement crack recognition method based on improved DeeplabV3+ and morphology, the method using the pavement crack recognition model provided in the first aspect, the method comprising:

[0019] Obtain a crack image dataset of the road surface to be identified;

[0020] The pavement crack recognition model is called, and the crack image data set of the road surface to be recognized is input into the pavement crack recognition model to obtain the pavement crack recognition result.

[0021] In a third aspect, the present application also provides a training device for a pavement crack recognition model based on an improved DeeplabV3+ and morphology, comprising:

[0022] A pavement crack detection training data set acquisition module is used to acquire a pavement crack detection training data set, where the pavement crack detection training data set includes at least one sample image containing pavement cracks;

[0023] A pavement crack detection training data set preprocessing module is used to preprocess the sample images in the pavement crack detection training data set using the Canny algorithm and the morphological method, wherein the preprocessing includes using the Canny algorithm to perform edge detection on the sample images, and using the morphological method to perform corrosion and expansion on the sample images processed by the Canny algorithm, so as to obtain a training image for each sample image in the pavement crack detection training data set;

[0024] A pavement crack detection and annotation training data set generation module is used to annotate pavement cracks in sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection and annotation training data set;

[0025] The pavement crack recognition model training module is used to build a pavement crack recognition network structure through DeepLabV3+, which takes sample images containing pavement cracks as input and pavement crack recognition results as output, and uses the pavement crack detection and annotation training data set to train the pavement crack recognition network structure and output the pavement crack recognition model.

[0026] In a fourth aspect, the present application further provides a pavement crack recognition device based on improved DeeplabV3+ and morphology, the device using the pavement crack recognition model provided in the first aspect, the device comprising:

[0027] A crack image data set acquisition module is used to acquire a crack image data set of a road surface to be identified;

[0028] The pavement crack recognition model calling module is used to call the pavement crack recognition model, input the crack image data set of the road surface to be identified into the pavement crack recognition model, and obtain the pavement crack recognition result.

[0029] In the fifth aspect, the present application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements a training method for a pavement crack recognition model based on improved DeeplabV3+ and morphology, and steps of a pavement crack recognition method based on improved DeeplabV3+ and morphology.

[0030] In the sixth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a pavement crack recognition model based on improved DeeplabV3+ and morphology and steps for a pavement crack recognition method based on improved DeeplabV3+ and morphology.

[0031] In the seventh aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a training method for a pavement crack recognition model based on improved DeeplabV3+ and morphology and steps of a pavement crack recognition method based on improved DeeplabV3+ and morphology.

[0032] The pavement crack recognition method, model training method, device, computer equipment, storage medium and computer program product based on improved DeeplabV3+ and morphology obtain a pavement crack detection training data set, which includes at least one sample image containing pavement cracks; preprocess the sample images in the pavement crack detection training data set using the Canny algorithm and the morphological method, wherein the preprocessing includes edge detection of the sample images using the Canny algorithm, and corrosion and expansion of the sample images processed by the Canny algorithm using the morphological method, so as to obtain a training image of each sample image in the pavement crack detection training data set; annotate the pavement cracks in the sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection annotated training data set; construct a pavement crack recognition network structure using DeepLabV3+ with the sample images containing pavement cracks as input and the pavement crack recognition results as output, and train the pavement crack recognition network structure using the pavement crack detection annotated training data set to output a pavement crack recognition model. The algorithm is lightweight and efficient, and can be deployed on lightweight platforms such as drones to achieve real-time detection of cracks with very different characteristics such as size, shape, depth, and growth morphology. It also reduces the impact of various weather and lighting conditions, and the impact of various contrast and clarity two-dimensional image data obtained in rainy, snowy, or low-light conditions on pavement crack detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a process of training a pavement crack recognition model based on improved DeeplabV3+ and morphology in one embodiment;

[0034] Figure 2 A schematic diagram of a dilation operation in a morphological method in one embodiment;

[0035] Figure 3 A schematic diagram of an erosion operation in a morphological method in one embodiment;

[0036] Figure 4 A schematic diagram of an opening operation in a morphological method in an embodiment;

[0037] Figure 5 It is a flowchart of a pavement crack identification method based on improved DeeplabV3+ and morphology in one embodiment;

[0038] FIG6(a), FIG6(b), and FIG6(c) are sample images of pavement crack elements in one embodiment;

[0039] Figure 7 A binary image obtained after a road crack detection training data set is initially processed by Canny in an embodiment;

[0040] Figure 8 An image obtained by processing a road crack detection training data set after optimizing a high-pass threshold using the Canny algorithm in an embodiment;

[0041] Fig. 9 A binary image obtained after a pavement crack detection training data set is optimized by a morphological method in an embodiment;

[0042] Fig.10 A schematic diagram of a road crack detection training data set annotated with pavement cracks by Image Labeler in one embodiment;

[0043] Fig.11 A schematic diagram of a road crack recognition model training result in one embodiment;

[0044] FIG. 12( a ), FIG. 12 ( b ), FIG. 12 ( c ), FIG. 12 ( d ), and FIG. 12 ( e ) are schematic diagrams of test samples and effects of a pavement crack identification model in one embodiment. DETAILED DESCRIPTION

[0045] Pavement cracks are a common but seriously underestimated disease that can lead to the destruction and weakening of the pavement structure, causing the pavement to deform and move more when subjected to vehicle loads. This deformation and displacement will further aggravate the expansion and deepening of the cracks, forming a vicious cycle, and ultimately leading to serious damage to the pavement. In order to ensure the continued operation of the highway network, it is essential to regularly check and repair potential problems. The traditional manual method of pavement damage detection faces many challenges, including but not limited to traffic interference, safety risks, insufficient detection accuracy, and errors caused by human factors. With the integration of the field of highway maintenance and cutting-edge computer technology, advanced technologies such as digital technology, laser scanning and radar detection are integrated with computer systems, promoting the innovation of automatic data collection technology.

[0046] At present, many concrete pavement crack automatic detection technologies still rely on traditional fully convolutional neural network (FCN) models, which have limitations in recognition efficiency and are not easy to deploy on lightweight platforms due to the complexity of the model. Although a few technologies have adopted improved fully convolutional neural networks and achieved better semantic segmentation effects, solved some problems of traditional FCNs, and improved the accuracy and detection speed of crack recognition, these improved networks often ignore the information flow between the encoder and a single decoder. Their encoder-decoder structure is usually simply connected in series, without establishing a connection between adjacent encoder output information, resulting in the loss of detail information during image feature extraction. This information loss limits the comprehensiveness of the model in extracting image features and affects the ability to exclude false cracks in complex environments, thus requiring further improvement of crack segmentation effects and detection accuracy.

[0047] In order to significantly improve data utilization efficiency and recognition accuracy, and thus more effectively manage the health of the road surface, the present application provides a widely applicable, accurate and efficient road crack detection technology. The Canny edge detection algorithm in the field of computer vision, the morphological method in mathematical morphology theory, and the convolutional neural network technology in the field of deep learning are applied to the problem of identifying road crack images. Compared with earlier edge detection methods, the combination of the Canny algorithm and the morphological method can effectively suppress signal noise and suppress the non-maximum of gradient changes in normalized grayscale binary images, greatly improving the edge discontinuity defects of traditional algorithms. This application mainly conducts algorithm research from two aspects: road crack image preprocessing and edge detection, and verifies the performance of the algorithm through experiments.

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] In an exemplary embodiment, Figure 1 As shown, a training method for a pavement crack recognition model based on improved DeeplabV3+ and morphology is provided, including:

[0050] Step S1, obtaining a pavement crack detection training data set, where the pavement crack detection training data set includes at least one sample image containing pavement cracks.

[0051] First, data set collection is performed. Specifically, a public pavement crack detection training data set is obtained from an open source network platform, and its specifications are standardized using MATLAB.

[0052] Step S2, using the Canny algorithm and morphological methods to preprocess the sample images in the pavement crack detection training data set, wherein the preprocessing includes using the Canny algorithm to perform edge detection on the sample images, and using the morphological method to corrode and dilate the sample images processed by the Canny algorithm to obtain a training image for each sample image in the pavement crack detection training data set.

[0053] Specifically, the Canny algorithm preprocessing is to use the Canny algorithm to perform edge detection on the sample images in the pavement crack detection training data set, remove noise and retain the crack edges, and modify the upper and lower thresholds and hysteresis compensation values. Furthermore, the morphological method is optimized, that is, the morphological method is used for corrosion and expansion, and finally a binary image with significant enhancement and weakened interference signals is obtained, that is, the training image of each sample image in the pavement crack detection training data set.

[0054] Step S3, annotating the pavement cracks in the sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection annotated training data set.

[0055] Specifically, the data set is labeled, that is, the pavement crack detection training data set is labeled at the pixel level using the Image Labeler in MATLAB, and then the label information is superimposed on the original image, that is, the label information of the sample image in the pavement crack detection training data set after labeling is superimposed on the sample image in the pavement crack detection training data set.

[0056] Step S4, constructing a pavement crack recognition network structure using DeepLabV3+ with sample images containing pavement cracks as input and pavement crack recognition results as output, and training the pavement crack recognition network structure using a pavement crack detection and annotation training data set to output a pavement crack recognition model.

[0057] Specifically, DeepLabV3+ model training uses the lightweight deep learning architecture of DeepLabV3+ to perform efficient and accurate semantic segmentation training to build a lightweight but highly accurate pavement crack recognition model.

[0058] In this embodiment, the data set preprocessed using the Canny algorithm and morphological method has prominent targets, intact features, and most noise points are filtered out. Using the preprocessed pavement crack detection training data set for semantic segmentation model training can more efficiently learn pavement crack features and obtain more efficient pavement crack recognition efficiency.

[0059] In an exemplary embodiment, step S2 includes:

[0060] The Canny algorithm is used to perform edge detection on the sample images in the pavement crack detection training data set, remove noise in the sample images and retain the pavement crack edges, and generate a gradient map of the sample images; the gradient amplitude and direction in the gradient map of the sample images are calculated, and non-maximum suppression is performed on the sample images; the low threshold and high threshold of the Canny algorithm are adjusted, and the sample images are processed using the adjusted Canny algorithm to obtain binary images of the sample images in the pavement crack detection training data set;

[0061] The morphological method is used to perform corrosion and expansion processing on the binary image of the sample image processed by the Canny algorithm to obtain the training image of each sample image in the pavement crack detection training data set.

[0062] For example, in order to obtain obvious pavement crack features, the Canny algorithm and morphological methods are used for preprocessing. Among them, the Canny algorithm is first used to preliminarily process the sample images in the pavement crack detection training data set. After the preliminary processing of the Canny algorithm, the noise is suppressed to a certain extent and almost fits within a certain threshold. In order to further refine the pavement crack features, the gradient map of the sample image after preliminary processing is screened to suppress non-maximum values ​​within a certain range. In this range, the most significant gradient feature enhancement can further refine the image processing effect. The non-maximum upper and lower thresholds are one of the most important parameters of the Canny algorithm.

[0063] Furthermore, the Sobel operator is used to calculate the pixel gradient. The Sobel operator is two 3*3 matrices, S x and S y The former S x Used to calculate the pixel gradient matrix G in the x direction of the image x , the latter S y Used to calculate the pixel gradient matrix G in the x direction of the image y The specific forms are shown in formula (1) and formula (2):

[0064]

[0065] Where I is the grayscale image matrix, and the * here represents the cross-correlation operation, that is, the convolution operation can be regarded as the cross-correlation operation after the convolution kernel is rotated 180°. It should be noted that the origin of the image matrix coordinate system is in the upper left corner, and the positive direction of x is from left to right, and the positive direction of y is from top to bottom, then the gradient amplitude is shown in formula (3):

[0066]

[0067] The gradient direction is shown in formula (4):

[0068]

[0069] After obtaining the gradient amplitude and direction, non-maximum suppression is performed on the entire image, that is, a local maximum search is performed, the local maximum value is retained, and all values ​​of non-local maximum values ​​are suppressed.

[0070] Furthermore, the low threshold sets a limit, below which pixel gradient intensities are considered non-edges and ignored in the edge detection process. Reducing this threshold will cause more pixels to be considered edges, improving the sensitivity of detection, but it will also introduce more noise, resulting in more false edges in the output image. Reducing the low threshold will result in more sensitive edge detection, detecting more edges, but it may also produce more false positives due to increased noise. The high threshold determines that pixel gradient intensities above this value are determined as strong edge points, while pixels below this value but above the low threshold are considered weak edge points. Increasing the high threshold will increase the number of strong edge points and reduce the number of weak edge points, thereby improving the accuracy and continuity of edges in the output image, but it may also cause some edges to be lost. Increasing the high threshold will increase the number of strong edges in the output image and reduce the number of false edges, thereby improving the accuracy and continuity of edge detection, but it may cause some edges to be lost. Adjusting the filtering interval between these two thresholds is the key to balancing detection sensitivity and precision. The larger scale focuses on strong edges and reduces weak edges, suitable for clear images; the small-scale strategy shows excellent anti-interference ability when dealing with noisy images. In response to the chaotic response in edge detection, the non-maximum suppression technology cleverly refines the edge definition. Its core principle is that the technology will rigorously evaluate the gradient strength of each pixel and examine the gradient strength of its neighboring pixels on the gradient vector path. Only when the gradient strength of the current pixel exceeds the gradient strength of its neighboring pixels and is confirmed as a local peak, it will be identified as a valid edge, and the rest will be suppressed to eliminate interference. In order to improve the accuracy of recognition, linear interpolation is generally used in the gradient direction to obtain a more accurate comparison value for judgment. After multiple debugging, the image processed by the Canny algorithm has been significantly improved under the dual threshold parameters that are more suitable for pavement feature extraction, and the binary image of the sample image in the pavement crack detection training data set is obtained. Exemplarily, a dual threshold with strong and weak edge pixels is set, that is, a low threshold T1 and a high threshold T2. The grayscale change is greater than T2 for strong edge pixels, between T1 and T2 for weak edges, and below T1 is eliminated, and the strong edge pixels in the neighborhood of the weak edge point are retained.

[0071] Furthermore, the binary image of the sample image processed by the Canny algorithm is subjected to the dilation and erosion operations in the morphological method. The main functions of erosion and dilation are to eliminate noise; separate independent image elements, and when processing the binary image of the pavement cracks that has been optimized by the Canny algorithm, the encouraging noise points can be further removed, and some pavement crack edges that are incorrectly suppressed can be repaired, and the pavement crack binary image can be further optimized using the morphological method. The binary image of the sample image in the pavement crack detection training data set output by Canny has 1 foreground and 0 for background in the image. Among them, the morphological method uses a structural element, which can be of any shape. The structural element has an anchor point O, which is generally defined as the center of the structural element. Optionally, the anchor point O can also be freely defined. Let f be the input image and s be the structural element.

[0072] Dilation is to slide the structure element on the image and set the grayscale value of the image pixel at the structure element anchor point to the maximum value of the pixel in the area where the structure element value is 1. The formula is shown in formula (5), where element is the structure element, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixel with the structure element value of 1 relative to the anchor point O, src is the original image, and dst is the result image. The dilation operation is as follows: Figure 2 As shown, visually the foreground in the image appears to be "expanded":

[0073] dst(x,y)=max(x',y') :element(x',y')≠0 scr(x+x',y+y') (5)

[0074] The expansion operation is expressed in formula notation as

[0075] Erosion is to slide the structure element on the image and set the grayscale value of the image pixel at the structure element anchor point to the minimum value of the pixel in the image area corresponding to the area where the structure element value is 1. The formula is shown in formula (6), where element is the structure element, (x, y) is the position of the anchor point O, x' and y' are the position offsets of the pixel with the structure element value of 1 relative to the anchor point O, src is the original image, and dst is the result image. The erosion operation is as follows: Figure 3 As shown, visually the foreground in the image appears to be "corroded":

[0076] dst(x,y)=min(x',y') :element(x',y')≠0 scr(x+x',y+y') (6)

[0077] The erosion operation is represented by the formula symbol

[0078] Furthermore, using the same structural element s to corrode and then dilate the image f is called an opening operation, as shown in formula (7):

[0079]

[0080] Open operation Figure 4 As shown, visually it seems as if the originally connected objects are "separated".

[0081] For example, a square structural element is selected in MATLAB, a larger connected area is extracted as the curve part, and a smaller connected area is filtered to further optimize the binary image of the pavement crack. The binary image of the sample image in the pavement crack detection training data set is processed by opening operation, the pixel value of the connected area is calculated, and the connected area that is too small is filtered out. This can be achieved by the following code:

[0082] 1.% Use morphological operations to remove small pixels

[0083] 2.se = strel('square',1);

[0084] 3.edge_img_opened=imopen(img,se); 4.

[0086] 5.% Display the results after morphological operation

[0087] 6.% subplot(1,2,2);

[0088] 7.%imshow(edge_img_opened);

[0089] 8.%title('Opened Edge Image'); 9.

[0091] 10.% Find contours in the image

[0092] 11.contour_img=bwlabel(edge_img_opened);

[0093] 12.stats=regionprops(contour_img,'Area','PixelList');

[0094] 13.% Extract the larger connected area as the curve part

[0095] 14.min_area_threshold=b; %Set a threshold to filter small connected areas

[0096] 15.curve_img=false(size(edge_img_opened));

[0097] 16.for i=1:length(stats)

[0098] 17.ifstats(i).Area>min_area_threshold

[0099] 18.curve_img(sub2ind(size(edge_img_opened),stats(i).PixelList(:,2),stats(i).Pi xelList(:,1)))=true;

[0100] 19.end

[0101] 20.end twenty one.

[0103] 22. a_img = curve_img;

[0104] In this embodiment, the appropriate Canny algorithm parameters are first debugged to preliminarily process the binary image of the road surface condition image collection, that is, the road surface crack detection training data set, and the binary image with most noise points filtered and road surface cracks preserved intact is obtained. Then, the morphological method is used to open the binary image, calculate the pixel values ​​of the isolated connected areas in the binary image, filter the areas that are too small, and obtain the training image of each sample image in the road surface crack detection training data set with most noise points removed and road surface crack edge details preserved intact. At this point, the preprocessing of the road surface crack detection training data set is completed, and most noise points are filtered under the premise of preserving the complete road surface crack edge.

[0105] In an exemplary embodiment, before step S4, the following steps are also included:

[0106] The network architecture mobilenetV2 is used as a feature extractor to extract features from the pavement crack detection and annotation training dataset, and the features are mapped to DeepLabV3+.

[0107] Specifically, Image Labeler is used in MATLAB to label the road crack detection training data set processed by the Canny algorithm and morphological method. In most samples in the labeled data set, the proportion of pixels occupied by road cracks in the entire picture is between 10% and 20%, which is exactly the part that the model needs to focus on learning. The importance of a small part of the target (i.e., road cracks) is greater than that of most of the background. The ratio of road cracks to background in each picture is calculated, and this ratio is added as a weight in the model output layer to better converge the training results.

[0108] Furthermore, after MATLAB annotation, a pavement crack detection annotation training dataset was generated. The lightweight network architecture mobilenetV2 was used as the feature extractor, and the features were mapped to DeepLabV3+ for pixel-level semantic segmentation. After multiple iterations, a semantic segmentation model of pavement cracks, namely the pavement crack recognition model, was obtained.

[0109] In an exemplary embodiment, step S4 further includes:

[0110] Obtain a pavement crack detection test dataset;

[0111] The pavement crack detection test data set is input into the pavement crack recognition model, and the pavement crack recognition model is evaluated using pavement crack recognition indicators, where the pavement crack recognition indicators include IOU, PA, and RA.

[0112] For example, the performance of the model is evaluated through the test set, using indicators such as IOU, PA, RA, etc. Finally, verification and testing are carried out, and the classic training scheme is compared. The accuracy of the models obtained by different training schemes in various identical scenarios is analyzed and compared from several experimental indicators.

[0113] Specifically, IOU (Intersection over Union) is Intersection over Union, which measures the degree of overlap between the model's prediction results for a specific category and the true annotation. Specifically, it is the ratio between the intersection of pixels predicted as positive and pixels actually positive, and the union of the two. The higher the IOU, the higher the segmentation accuracy of the model; PA (Pixel Accuracy), that is, the ratio of the number of correctly predicted pixels to the total number of pixels, reflects the overall classification accuracy of the model; RA (Recognition Accuracy), that is, the ratio of the number of correctly identified samples to the total number of samples, is used to measure the accuracy of the model in identifying cracks.

[0114] In an exemplary embodiment, a pavement crack recognition method based on improved DeeplabV3+ and morphology is provided. The method uses the pavement crack recognition model provided in the above embodiment. The method includes:

[0115] Obtain a crack image dataset of the road surface to be identified;

[0116] The pavement crack recognition model is called, and the crack image data set of the road surface to be recognized is input into the pavement crack recognition model to obtain the pavement crack recognition result.

[0117] In one embodiment, a pavement crack recognition method based on improved DeeplabV3+ and morphology is as follows: Figure 5 As shown, including:

[0118] The open source pavement crack detection training dataset was obtained from the Internet, and its specifications were unified using MATLAB.

[0119] The Canny algorithm is used to detect the edge of the image, including Gaussian filtering, gradient calculation, non-maximum suppression and double threshold processing.

[0120] The morphological method is applied to optimize the image processed by the Canny algorithm, including erosion and dilation operations, to remove noise points and retain the crack edges.

[0121] Use MATLAB's Image Laheler to perform pixel-level annotation on the preprocessed images.

[0122] Use the lightweight network architecture MobileNetV2 as the feature extractor. Map the extracted features to the DeepLabV3+ architecture for pixel-level semantic segmentation. After multiple rounds of iterative training, the model accuracy is stabilized.

[0123] The model is evaluated using indicators such as IOU (Intersection over Union), PA (Pixel Accuracy) and RA (Recognition Accuracy). The actual effect is demonstrated and evaluated on the test set.

[0124] In this embodiment, the traditional Canny edge detection algorithm and morphological method are used for image preprocessing, and deep learning technology, especially DeepLabV3+ model, is combined for semantic segmentation. This combination can improve the accuracy of crack detection while retaining image details.

[0125] In one embodiment, we first collect open source pavement crack detection training data sets from Github. These data sets should contain a variety of pavement crack samples and possible interference elements, such as zebra crossings, branches, pedestrians, etc. The pavement crack element sample images are shown in Figure 6 (a), Figure 6 (b), and Figure 6 (c). The Canny edge detection algorithm is used to preliminarily process the pavement crack detection training data set images. By optimizing the high and low thresholds and hysteresis threshold parameters, the image noise is effectively filtered out, while ensuring that the edge information of the pavement cracks is retained. The binary image obtained after the pavement crack detection training data set is preliminarily processed by Canny is shown in Figure 6 (a), Figure 6 (b), and Figure 6 (c). Figure 7 As shown in the figure, the image obtained by processing the pavement crack detection training dataset after optimizing the high-pass threshold with the Canny algorithm is as follows: Figure 8 Following the Canny algorithm, morphological operations are implemented, and the pixel values ​​of the detected connected areas are analyzed using the opening operation, and those areas with too small areas are screened out to ensure that only cracks with significant features are retained. The binary image obtained after the pavement crack detection training data set is optimized by the morphological method is shown in Fig. 9 In the MATLAB R2022a environment, the built-in Image Labeler tool is used to accurately label the data set after Canny algorithm and morphological processing, and a pavement crack detection labeling training data set for training is created. The schematic diagram of the pavement crack detection training data set labeled with Image Labeler is shown in Fig.10 As shown. Subsequently, the lightweight MobileNetV2 network was used as the feature extraction network, and combined with the DeepLabV3+ architecture for pixel-level semantic segmentation. After 20 cycles of 7400 iterations of training, a stable model with an accuracy of 95.7% and a loss value of 0.12 was finally obtained, namely the pavement crack recognition model, semantic segmentation model, and pavement crack recognition model training results are shown in Fig.11 As shown. Finally, some typical samples were collected from the Internet and actual scenes for test verification. The model test samples and effects are shown in Figure 12(a), Figure 12(b), Figure 12(c), Figure 12(d), and Figure 12(e). The test results show that the trained semantic segmentation model, namely the pavement crack recognition model, can effectively identify pavement cracks under different conditions and meet the requirements of the recognition task. In addition to the direct visual effect display, the performance of the semantic segmentation model can also be evaluated by relying on the following key indicators, namely IOU (intersection over union), PA (pixel accuracy), and RA (recognition accuracy). The model evaluation results are shown in Table 1.

[0126] Table 1

[0127]

[0128] On a randomly collected test data set, the recognition accuracy of the pavement crack recognition model reached 93.97%. This shows that the model has a high accuracy in identifying pavement cracks and can effectively assist in pavement maintenance and repair work. The IOU (intersection over union) of the model on the test set is 71.93%, and the PA (pixel accuracy) is 69.06%. These indicators further prove that the model has good performance in the pavement crack detection task and can effectively distinguish between cracks and non-crack areas. The pavement crack recognition model can achieve good detection results under different weather conditions and lighting conditions, including in rainy and snowy weather or insufficient light. This shows that the pavement crack recognition model has good adaptability and robustness to environmental changes.

[0129] This application uses a lightweight network architecture MobilenetV2 as a feature extractor, and then maps the features to DeepLabV3+ for pixel-level semantic segmentation. The resulting semantic segmentation model, i.e., the pavement crack recognition model, has a target recognition accuracy of 93.97%, an IOU of 71.93%, and a PA of 69.06% in the test set.

[0130] In summary, the pavement crack recognition model achieves efficient and accurate pavement crack recognition by combining the Canny algorithm and morphological methods for image preprocessing and using a lightweight deep learning architecture for semantic segmentation. These advantages make the model highly valuable in actual pavement inspection and maintenance work.

[0131] At least a portion of the steps in the flowcharts involved in the embodiments described above may include multiple steps or multiple stages. These steps or stages may be executed at different times, or may be executed in turn or alternation with other steps or at least a portion of the steps or stages in other steps.

[0132] Based on the same inventive concept, the embodiments of the present application also provide a training device based on an improved DeeplabV3+ and morphological pavement crack recognition model for implementing the above-mentioned training method of a pavement crack recognition model based on improved DeeplabV3+ and morphology, and a pavement crack recognition device based on an improved DeeplabV3+ and morphology based on a pavement crack recognition method based on improved DeeplabV3+ and morphology. The implementation solutions to the problems provided by the above-mentioned devices are similar to the implementation solutions recorded in the above-mentioned methods. Therefore, the specific limitations in one or more device embodiments provided below can refer to the limitations on the corresponding methods above and will not be repeated here.

[0133] In an exemplary embodiment, a training device for a pavement crack recognition model based on an improved DeeplabV3+ and morphology is provided, comprising:

[0134] A pavement crack detection training data set acquisition module is used to acquire a pavement crack detection training data set, where the pavement crack detection training data set includes at least one sample image containing pavement cracks;

[0135] A pavement crack detection training data set preprocessing module is used to preprocess the sample images in the pavement crack detection training data set using the Canny algorithm and the morphological method, wherein the preprocessing includes using the Canny algorithm to perform edge detection on the sample images, and using the morphological method to perform corrosion and expansion on the sample images processed by the Canny algorithm, so as to obtain a training image for each sample image in the pavement crack detection training data set;

[0136] A pavement crack detection and annotation training data set generation module is used to annotate pavement cracks in sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection and annotation training data set;

[0137] The pavement crack recognition model training module is used to build a pavement crack recognition network structure through DeepLabV3+, which takes sample images containing pavement cracks as input and pavement crack recognition results as output, and uses the pavement crack detection and annotation training data set to train the pavement crack recognition network structure and output the pavement crack recognition model.

[0138] In an exemplary embodiment, a pavement crack identification device based on improved DeeplabV3+ and morphology is provided, comprising:

[0139] A crack image data set acquisition module is used to acquire a crack image data set of a road surface to be identified;

[0140] The pavement crack recognition model calling module is used to call the pavement crack recognition model, input the crack image data set of the road surface to be identified into the pavement crack recognition model, and obtain the pavement crack recognition result.

[0141] Each module in the above device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0142] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0144] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0145] For those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these modifications and improvements are all within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A training method for a pavement crack recognition model based on improved DeeplabV3+ and morphology, characterized in that: The method comprises: Step S1, obtaining a pavement crack detection training data set, where the pavement crack detection training data set includes at least one sample image containing pavement cracks; Step S2, using the Canny algorithm and the morphological method to preprocess the sample images in the pavement crack detection training data set, wherein the preprocessing includes using the Canny algorithm to perform edge detection on the sample images, and using the morphological method to perform corrosion and expansion on the sample images processed by the Canny algorithm, so as to obtain a training image of each sample image in the pavement crack detection training data set; Step S3, annotating the pavement cracks in the sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection annotated training data set; Step S4, constructing a pavement crack recognition network structure using DeepLabV3+ with sample images containing pavement cracks as input and pavement crack recognition results as output, and training the pavement crack recognition network structure using a pavement crack detection and annotation training data set to output a pavement crack recognition model.

2. The method according to claim 1, characterized in that The step S2 comprises: The Canny algorithm is used to perform edge detection on the sample images in the pavement crack detection training data set, remove noise in the sample images and retain the pavement crack edges, and generate a gradient map of the sample images; the gradient amplitude and direction in the gradient map of the sample images are calculated, and non-maximum suppression is performed on the sample images; the low threshold and high threshold of the Canny algorithm are adjusted, and the sample images are processed using the adjusted Canny algorithm to obtain binary images of the sample images in the pavement crack detection training data set; The morphological method is used to perform corrosion and expansion processing on the binary image of the sample image processed by the Canny algorithm to obtain the training image of each sample image in the pavement crack detection training data set.

3. The method according to claim 2, characterized in that Before step S4, the following steps are also included: The network architecture mobilenetV2 is used as a feature extractor to extract features from the pavement crack detection and annotation training dataset, and the features are mapped to DeepLabV3+.

4. The method according to claim 3, characterized in that After step S4, the following steps are also included: Obtain a pavement crack detection test dataset; The pavement crack detection test data set is input into the pavement crack recognition model, and the pavement crack recognition model is evaluated using pavement crack recognition indicators, where the pavement crack recognition indicators include IOU, PA, and RA.

5. A pavement crack recognition method based on improved DeeplabV3+ and morphology, characterized in that: The method uses the pavement crack identification model according to any one of claims 1 to 4, and the method comprises: Obtain a crack image dataset of the road surface to be identified; The pavement crack recognition model is called, and the crack image data set of the road surface to be recognized is input into the pavement crack recognition model to obtain the pavement crack recognition result.

6. A training device for a pavement crack recognition model based on improved DeeplabV3+ and morphology, characterized in that: The device comprises: A pavement crack detection training data set acquisition module is used to acquire a pavement crack detection training data set, where the pavement crack detection training data set includes at least one sample image containing pavement cracks; A pavement crack detection training data set preprocessing module is used to preprocess the sample images in the pavement crack detection training data set using the Canny algorithm and the morphological method, wherein the preprocessing includes using the Canny algorithm to perform edge detection on the sample images, and using the morphological method to perform corrosion and expansion on the sample images processed by the Canny algorithm, so as to obtain a training image for each sample image in the pavement crack detection training data set; A pavement crack detection and annotation training data set generation module is used to annotate pavement cracks in sample images in the preprocessed pavement crack detection training data set based on MATLAB to generate a pavement crack detection and annotation training data set; The pavement crack recognition model training module is used to build a pavement crack recognition network structure through DeepLabV3+, which takes sample images containing pavement cracks as input and pavement crack recognition results as output, and uses the pavement crack detection and annotation training data set to train the pavement crack recognition network structure and output the pavement crack recognition model.

7. A pavement crack identification device based on improved DeeplabV3+ and morphology, characterized in that: The device uses the pavement crack identification model as claimed in any one of claims 1 to 4, and comprises: A crack image data set acquisition module is used to acquire a crack image data set of a road surface to be identified; The pavement crack recognition model calling module is used to call the pavement crack recognition model, input the crack image data set of the road surface to be identified into the pavement crack recognition model, and obtain the pavement crack recognition result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When a processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images