Asphalt pavement crack detection method and system

Abstract

The invention provides an asphalt pavement crack detection method and system. The method comprises the following steps: acquiring an asphalt pavement image; preprocessing the asphalt pavement image according to a preset image processing mode to obtain a target asphalt pavement image; analyzing the target asphalt pavement image by using an asphalt pavement crack detection model, and outputting an asphalt pavement crack detection result, the asphalt pavement crack detection model comprising a convolutional neural network-based feature extraction network and a classifier, the feature extraction network comprising a plurality of convolutional layers, and the classifier is used for extracting the crack characteristics in the target asphalt pavement image, and is used for classifying and identifying the extracted crack characteristics, so that the asphalt pavement cracks can be accurately detected in a complicated asphalt pavement scene.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence-based crack detection technology, and in particular to an asphalt pavement crack detection method and system. Background Art

[0002] Asphalt pavement crack detection is a critical task in current road maintenance and management. With the continuous increase in traffic volume and the extension of road service life, timely detection and repair of asphalt pavement cracks are of great significance to ensuring driving safety and extending the service life of roads.

[0003] However, traditional asphalt pavement crack detection methods often lack the accuracy needed for complex asphalt pavement scenarios. For example, they are susceptible to noise interference (such as the random texture of asphalt particles and stains) and uneven lighting, which can lead to inaccurate crack feature extraction and thus affect detection accuracy.

[0004] For example, the technical solution with patent application number 202210117122.3 discloses a pavement crack monitoring method based on image recognition. This method collects an image of the road surface to be tested and determines the target crack profile in the image. The target crack profile is used to indicate the deformation information of the target crack. Based on this deformation information, information about the environment in which the road surface to be tested is located, and a preset crack assessment model, crack information corresponding to the road surface to be tested is generated. The crack information includes the crack degree, which is used to automatically identify the crack degree of the road surface. However, this technical solution is susceptible to problems such as noise interference and uneven lighting on the road surface, resulting in inaccurate crack information extraction, which in turn affects the accuracy of detection. Summary of the Invention

[0005] The present invention provides an asphalt pavement crack detection method and system, which can accurately detect asphalt pavement cracks when facing complex asphalt pavement scenes.

[0006] In order to solve the above problems, the present invention adopts the following technical solutions:

[0007] In one aspect, the present invention provides a method for detecting cracks in an asphalt pavement, comprising:

[0008] Acquire asphalt road surface images;

[0009] Preprocessing the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image;

[0010] The target asphalt pavement image is analyzed using an asphalt pavement crack detection model to output asphalt pavement crack detection results. The asphalt pavement crack detection model includes a feature extraction network and a classifier based on a convolutional neural network. The feature extraction network includes multiple convolutional layers for extracting crack features in the target asphalt pavement image, and the classifier is used to classify and identify the extracted crack features.

[0011] Preferably, the pre-processing of the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image includes:

[0012] Evenly dividing the asphalt road surface image into a plurality of image blocks;

[0013] Counting the number of pixels of each brightness value in each of the image blocks, and calculating the ratio of the number of pixels of each brightness value in each image block to the total number of pixels in the corresponding image block, to form a brightness distribution of each image block;

[0014] Calculating the brightness adjustment amount that should be adjusted for each pixel in each image block according to the preset target brightness distribution and the brightness distribution of each image block;

[0015] By using an interpolation algorithm, the brightness value of each pixel in each image block is adjusted accordingly according to the brightness adjustment amount that each pixel should be adjusted to obtain the target asphalt road surface image.

[0016] Preferably, the pre-processing of the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image includes:

[0017] selecting a plurality of sliding windows of corresponding sizes according to a minimum crack width and a texture granularity in the asphalt pavement image, and sliding the plurality of sliding windows on the asphalt pavement image;

[0018] Calculating the average similarity between each pixel in each sliding window and surrounding pixels in terms of color, brightness, and gradient, and comparing the average similarity between each pixel in each sliding window and surrounding pixels with a similarity threshold;

[0019] Marking the first pixel whose average similarity with surrounding pixels is lower than the similarity threshold as a candidate crack edge pixel;

[0020] A second pixel whose average similarity with surrounding pixels is not less than the similarity threshold is marked as a normal road surface texture pixel, and the pixel value of the normal road surface texture pixel is adjusted according to a polynomial fitting method and the surrounding pixel values of the normal road surface texture pixel to obtain a target asphalt road surface image.

[0021] Preferably, the analyzing the target asphalt pavement image using the asphalt pavement crack detection model and outputting the asphalt pavement crack detection result includes:

[0022] Using an asphalt pavement crack detection model, the target asphalt pavement image is convolved with a preset multi-directional differential operator template to obtain differential values of each candidate crack edge pixel in the target asphalt pavement image in different directions, wherein the differential values are used to reflect the edge strength of the candidate crack edge pixel in the corresponding direction;

[0023] Comparing the differential value of each candidate crack edge pixel in the target asphalt pavement image in different directions with a preset differential threshold to screen out target crack edge pixels;

[0024] Predicting the directional continuity of the target crack edge pixels and constructing the crack edge contour;

[0025] The feature extraction network is called to extract crack features of the crack edge contour, and the classifier is used to classify and identify the crack features to obtain asphalt pavement crack detection results.

[0026] Preferably, constructing the crack edge profile includes:

[0027] Extracting a first crack edge contour segment based on the target crack edge pixel and directional continuity, and using the first crack edge contour segment as a reference, expanding a first preset range of pixel areas at both ends of the first crack edge contour segment along the crack direction to form candidate crack regions;

[0028] Analyzing the degree of dispersion of the candidate crack regions in color and brightness;

[0029] If the degree of dispersion is greater than the average level of a normal area of an asphalt pavement, a crack edge contour is constructed based on the candidate crack area.

[0030] Preferably, constructing a crack edge profile based on the candidate crack region includes:

[0031] At a local level, taking the candidate crack area as the center, selecting an asphalt pavement area within a second preset range around the candidate crack area, and calculating the standard deviation of the local contrast between the candidate crack area and the asphalt pavement area;

[0032] At the global level, the standard deviation of the global contrast of all candidate crack regions of the same type in the target asphalt pavement image is calculated;

[0033] Set the lower standard deviation threshold for local contrast and the upper standard deviation threshold for global contrast;

[0034] When the standard deviation of the local contrast is greater than the lower limit threshold of the standard deviation and the standard deviation of the global contrast is less than the upper limit threshold of the standard deviation, the candidate crack area is taken as the target crack area, the crack edge of the target crack area is identified and predicted, and the outline of the crack edge is marked to obtain the crack edge outline.

[0035] Preferably, constructing a crack edge profile based on the candidate crack region includes:

[0036] Dividing the candidate crack region into a plurality of overlapping analysis blocks, and calculating the angular standard deviation of the strike of the first edge contour segment in each of the analysis blocks;

[0037] When the angle standard deviation is greater than the angle threshold, it is determined that there is a first edge contour fragment with a disordered direction in the candidate crack area. The first edge contour fragment with a disordered direction is eliminated to form a target crack area, the crack edge of the target crack area is identified, and the contour of the crack edge is marked to obtain the crack edge contour.

[0038] Furthermore, before analyzing the target asphalt pavement image using the asphalt pavement crack detection model and outputting the asphalt pavement crack detection result, the method further includes:

[0039] Acquire training samples, the training samples including standard asphalt pavement crack images of different crack types and corresponding labels, the labels being used to mark the locations and types of cracks in the standard asphalt pavement crack images;

[0040] A feature extraction network is constructed based on a convolutional neural network. Based on the feature extraction network, classifiers are laterally connected through a top-down path to form an initial asphalt pavement crack detection model.

[0041] Inputting the training samples into an initial asphalt pavement crack detection model for training, and during the model training, fusing feature maps of different levels generated from the standard asphalt pavement crack image to generate a multi-scale feature map;

[0042] Building a region proposal network on the multi-scale feature map and generating multiple anchor boxes at predetermined positions on each feature map, wherein the anchor boxes have different scales and ratios for preliminary screening of regions that may contain cracks;

[0043] In the feature extraction network, each feature map is processed using a convolutional layer, and a target anchor box is output from a classification branch, and an anchor box offset of the target anchor box is output from a regression branch. The classification branch is used to predict and output the probability that each anchor box belongs to a crack, and the regression branch is used to calculate and output the anchor box offset of each anchor box. The anchor box offset is used to adjust the position and size of the anchor box.

[0044] The target anchor frames of different scales and proportions are normalized using a bilinear interpolation algorithm, and the prediction results are output through a classifier;

[0045] The loss value of the initial asphalt pavement crack detection model is calculated based on a preset loss function and the prediction result. When the loss value is lower than the preset loss value, the trained initial asphalt pavement crack detection model is used as the asphalt pavement crack detection model.

[0046] Furthermore, after calculating the loss value of the initial asphalt pavement crack detection model based on the preset loss function and the prediction result, the method further includes:

[0047] When the loss value is not lower than the preset loss value, returning the gradient information of the preset loss function;

[0048] According to the gradient information, the optimizer is used to update the parameters of the initial asphalt pavement crack detection model, and the initial asphalt pavement crack detection model with updated parameters is retrained until the loss value is lower than the preset loss value, thereby obtaining the asphalt pavement crack detection model.

[0049] In another aspect, the present invention further provides an asphalt pavement crack detection system, comprising:

[0050] An acquisition module, used for acquiring an asphalt road surface image;

[0051] a preprocessing module, configured to preprocess the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image;

[0052] An analysis module is used to analyze the target asphalt pavement image using an asphalt pavement crack detection model and output asphalt pavement crack detection results. The asphalt pavement crack detection model includes a feature extraction network and a classifier based on a convolutional neural network. The feature extraction network includes multiple convolution layers for extracting crack features in the target asphalt pavement image, and the classifier is used to classify and identify the extracted crack features.

[0053] Compared with the prior art, the technical solution of the present invention has at least the following advantages:

[0054] The asphalt pavement crack detection method and system provided by the present invention can effectively remove noise interference, enhance crack features, and improve image quality by acquiring asphalt pavement images and preprocessing the images according to a preset image processing method. In addition, the preprocessed images are analyzed using an asphalt pavement crack detection model based on a convolutional neural network, which includes a specially designed feature extraction network and classifier. The feature extraction network automatically learns the characteristic representation of cracks through multiple convolutional layers. When faced with complex asphalt pavement scenes, it is not easily affected by noise interference and uneven lighting problems of the asphalt pavement, and can accurately capture the various forms and detailed information of cracks. The classifier accurately classifies and identifies the extracted features, distinguishing cracks from non-crack areas, thereby fully utilizing the powerful feature learning ability and classification performance of the convolutional neural network, effectively improving the accuracy and efficiency of asphalt pavement crack detection, and achieving rapid and accurate detection of pavement cracks. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of an embodiment of the asphalt pavement crack detection method of the present invention;

[0056] Figure 2 This is a flowchart of another embodiment of the asphalt pavement crack detection method of the present invention;

[0057] Figure 3 This is a flowchart of another embodiment of the asphalt pavement crack detection method of the present invention;

[0058] Figure 4 This is a structural block diagram of an embodiment of the asphalt pavement crack detection system of the present invention. DETAILED DESCRIPTION

[0059] To help those skilled in the art better understand the present invention, the following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by those skilled in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention.

[0060] Please refer to Figure 1 As shown, the present invention provides a method for detecting cracks in an asphalt pavement, comprising the following steps:

[0061] S11, acquiring an asphalt road surface image;

[0062] S12, pre-processing the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image;

[0063] S13. Analyze the target asphalt pavement image using an asphalt pavement crack detection model and output asphalt pavement crack detection results. The asphalt pavement crack detection model includes a feature extraction network and a classifier based on a convolutional neural network. The feature extraction network includes multiple convolutional layers for extracting crack features in the target asphalt pavement image. The classifier is used to classify and identify the extracted crack features.

[0064] In this embodiment, image data of the asphalt pavement can be acquired using an image acquisition device. This device may include a high-resolution digital camera, a webcam, or an imaging system mounted on a drone. To ensure that the acquired asphalt pavement images accurately reflect the actual conditions of the asphalt pavement, attention should be paid to the influence of environmental conditions during acquisition. Acquisition should be conducted in weather and time periods with sufficient and uniform lighting to avoid image quality degradation due to insufficient or excessive lighting.

[0065] The purpose of preprocessing is to optimize the original collected asphalt pavement images, remove noise interference, and enhance the useful information of the images, so as to be more conducive to subsequent crack detection and analysis. Among them, the preset image processing methods may include:

[0066] Grayscale processing: If the asphalt pavement image is a color image, it will be converted into a grayscale image. Since the color information in a color image is usually not a key factor for crack detection, grayscale processing can reduce the amount of data and improve the efficiency of subsequent processing.

[0067] Noise removal: Actual asphalt pavement images are often subject to interference from various factors, introducing noise, such as salt and pepper noise and Gaussian noise. Noise removal can be achieved by using methods such as median filtering, mean filtering, and Gaussian filtering.

[0068] Image enhancement processing, in order to highlight the crack characteristics of asphalt pavement images and improve the contrast between cracks and the surrounding road surface background, histogram equalization and contrast-limited adaptive histogram equalization methods can be used for image enhancement.

[0069] Then, the trained asphalt pavement crack detection model is used to analyze the preprocessed target asphalt pavement image to achieve automatic detection and identification of cracks.

[0070] The convolutional neural network-based feature extraction network is a key component of the model and consists of multiple convolutional layers. The convolutional layers extract features from asphalt pavement images through convolution operations. Convolution uses a series of convolution kernels to slide across the image, performing a weighted summation operation on local regions to extract local features. As the convolutional layers deepen, the kernels become less susceptible to noise and uneven lighting in complex asphalt pavement scenes, accurately capturing increasingly abstract and advanced features in asphalt pavement images. In asphalt pavement crack detection, shallow convolutional layers may extract basic features such as crack edges and texture, while deeper convolutional layers can extract more complex features such as the crack's overall shape, direction, and contextual relationship with the surrounding road surface. For example, the first convolutional layer may learn simple edge features such as horizontal, vertical, and diagonal edges; subsequent convolutional layers combine these edge features into more complex crack shape features, such as linear cracks and branching cracks.

[0071] The classifier classifies and identifies the extracted crack features, determining whether each pixel or image block in the target asphalt pavement image belongs to the crack category. For asphalt pavement crack detection, there are two possible categories: crack and non-crack. The classifier outputs a probability value for each pixel or image block, based on a set probability threshold (e.g., 0.5). If the probability value exceeds the threshold, the pixel or image block is considered a crack; otherwise, it is considered a non-crack.

[0072] Finally, through the analysis of the asphalt pavement crack detection model, the asphalt pavement crack detection results are output. The detection results can be an image marked with the crack location and category, crack coordinate information, crack length, width and other parameters, or crack statistical information (such as the number of cracks, total length, etc.), providing detailed and accurate crack information to the road maintenance department so that corresponding maintenance measures can be taken in a timely manner.

[0073] The asphalt pavement crack detection method provided by the present invention can effectively remove noise interference, enhance crack features, and improve image quality by acquiring an asphalt pavement image and preprocessing the image according to a preset image processing method. In addition, the preprocessed image is analyzed using an asphalt pavement crack detection model based on a convolutional neural network, which includes a specially designed feature extraction network and classifier. The feature extraction network automatically learns the characteristic representation of cracks through multiple convolutional layers. When faced with complex asphalt pavement scenes, it is not easily affected by noise interference and uneven lighting of the asphalt pavement, and can accurately capture the various forms and detailed information of cracks; the classifier accurately classifies and identifies the extracted features, distinguishing cracks from non-crack areas, thereby fully utilizing the powerful feature learning ability and classification performance of the convolutional neural network, effectively improving the accuracy and efficiency of asphalt pavement crack detection, and achieving rapid and accurate detection of pavement cracks.

[0074] Please refer to Figure 2 As shown, in one embodiment, pre-processing the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image includes:

[0075] S121, dividing the asphalt road surface image into multiple image blocks on an even basis;

[0076] S122, counting the number of pixels of each brightness value in each image block, calculating the ratio of the number of pixels of each brightness value in each image block to the total number of pixels in the corresponding image block, to form a brightness distribution of each image block;

[0077] S123, calculating the brightness adjustment amount that should be adjusted for each pixel in each image block according to the preset target brightness distribution and the brightness distribution of each image block;

[0078] S124 , using an interpolation algorithm, adjusting the brightness value of each pixel in each image block according to the brightness adjustment amount that each pixel should be adjusted to obtain a target asphalt road surface image.

[0079] This embodiment divides the collected asphalt pavement image into a plurality of image blocks of equal size, so that subsequent processing can perform detailed brightness analysis and adjustment on different local areas of the asphalt pavement image.

[0080] Count the brightness values of the pixels in each image block. For example, assume the image block is an 8-bit grayscale image with a brightness range of 0-255. Count the number of pixels with each brightness value in the corresponding image block. Then, calculate the ratio of the number of pixels with each brightness value in each image block to the total number of pixels in the image block. This forms the brightness distribution of each image block, allowing you to understand the brightness composition within each image block.

[0081] The preset target brightness distribution is a desired brightness distribution, such as a uniform distribution or a distribution slightly leaning toward intermediate brightness. For each image block, the brightness distribution of that image block is compared with the preset target brightness distribution, and the brightness adjustment amount for each pixel in the image block is calculated. For example, if the preset target brightness distribution has a high proportion of brightness values of 128, while the actual image block has a low proportion of this brightness value, the brightness of some pixels in the image block needs to be increased.

[0082] An interpolation algorithm is used to adjust the brightness value of each pixel according to the calculated brightness adjustment amount to obtain the target asphalt pavement image, so that the brightness distribution of each image block is closer to the preset target brightness distribution, thereby improving the overall visual effect and quality of the image and enhancing the visibility of features such as cracks.

[0083] Asphalt pavement images exhibit significant variations in brightness under different lighting conditions. For example, under strong direct sunlight, the center of the road surface may appear overexposed, while areas shaded by trees appear darker, obscuring crack details. Therefore, this embodiment can adjust brightness to accommodate asphalt pavement images under varying lighting conditions. This effectively improves the robustness of crack detection and enhances the contrast of asphalt pavement images, making crack features more distinct and facilitating subsequent crack detection. Local brightness adjustment also makes cracks easier to identify, improving crack detection accuracy.

[0084] Please refer to Figure 3 As shown, in another embodiment, preprocessing the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image may specifically include:

[0085] S131, selecting a plurality of sliding windows of corresponding sizes according to the minimum crack width and texture granularity in the asphalt pavement image, and sliding the plurality of sliding windows on the asphalt pavement image;

[0086] S132, calculating the average similarity between each pixel in each sliding window and the surrounding pixels in terms of color, brightness, and gradient, and comparing the average similarity between each pixel in each sliding window and the surrounding pixels with a similarity threshold;

[0087] S133, marking the first pixel whose average similarity with surrounding pixels is lower than the similarity threshold as a candidate crack edge pixel;

[0088] S134. Mark a second pixel whose average similarity with surrounding pixels is not less than the similarity threshold as a normal road surface texture pixel, and adjust the pixel value of the normal road surface texture pixel according to a polynomial fitting method and the values of the surrounding pixels of the normal road surface texture pixel to obtain a target asphalt road surface image.

[0089] To determine the appropriate sliding window size for effectively capturing and analyzing crack features in asphalt pavement images, different sliding window sizes can be selected, such as 3×3 pixels, 5×5 pixels, or 7×7 pixels, based on the minimum crack width and texture granularity of asphalt pavement in real-world scenarios. This sliding window slides pixel by pixel across the image, covering the entire image and ensuring that every area is analyzed.

[0090] For each pixel in each sliding window, the average similarity between it and the surrounding pixels in terms of color, brightness, and gradient direction is calculated. Color similarity can be obtained by calculating the difference in RGB values between pixels; brightness similarity can be obtained by calculating the difference in brightness values; and gradient direction similarity can be obtained by calculating the difference in gradient directions. These three similarity values are then averaged to obtain the average similarity value for the pixel. This average similarity value is compared with a preset similarity threshold to determine whether the pixel is likely to be a crack edge pixel.

[0091] Asphalt pavement has a granular texture that interweaves with cracks, interfering with crack identification. Cracks typically have a regular, linear or mesh-like discontinuous texture, while normal pavement texture is relatively uniform and continuous. If the mean similarity value for a pixel falls below a preset similarity threshold, it indicates significant differences in color, brightness, and gradient from surrounding pixels and may represent a crack edge pixel. This pixel is marked as a candidate crack edge pixel for further processing and confirmation.

[0092] Pixels with a mean similarity value no less than the similarity threshold are considered normal road texture pixels. To smooth the road texture and reduce noise and interference, a polynomial fitting method is used to fit a polynomial surface based on the surrounding pixel values of the normal road texture pixels. The values of this polynomial surface are used to adjust the pixel values of the normal road texture pixels, resulting in a smoother target asphalt road image.

[0093] This embodiment effectively distinguishes crack edge pixels from normal pavement texture pixels through sliding window technology and polynomial fitting. This enables the model to more accurately identify crack locations, reduces misjudgments, improves detection accuracy, and reduces the model's initial workload. Furthermore, smoothing the normal pavement texture pixels reduces road noise and texture interference, resulting in clearer images and facilitating subsequent model analysis and processing. After processing, crack features become more distinct and their contrast with other pavement features is enhanced, helping to more clearly identify cracks and improve detection results.

[0094] In one embodiment, the analyzing the target asphalt pavement image using the asphalt pavement crack detection model and outputting the asphalt pavement crack detection result includes:

[0095] Using an asphalt pavement crack detection model, the target asphalt pavement image is convolved with a preset multi-directional differential operator template to obtain differential values of each candidate crack edge pixel in the target asphalt pavement image in different directions, wherein the differential values are used to reflect the edge strength of the candidate crack edge pixel in the corresponding direction;

[0096] Comparing the differential value of each candidate crack edge pixel in the target asphalt pavement image in different directions with a preset differential threshold to screen out target crack edge pixels;

[0097] Predicting the directional continuity of the target crack edge pixels and constructing the crack edge contour;

[0098] The feature extraction network is called to extract crack features of the crack edge contour, and the classifier is used to classify and identify the crack features to obtain asphalt pavement crack detection results.

[0099] For each candidate crack edge pixel in the target asphalt pavement image, a convolution operation is performed using differential operator templates in multiple directions (such as 0°, 45°, 90°, and 135°). A differential operator template is a small matrix used to detect edge features in an image, such as the Sobel operator or the Prewitt operator. The convolution operation slides the differential operator template over the target asphalt pavement image, multiplies it element-by-element with the pixel values in the target asphalt pavement image, and then sums the results to calculate the differential values of each candidate crack edge pixel in different directions. The magnitude of the differential value reflects the edge strength of the pixel in the corresponding direction, indicating whether the pixel is likely to be a crack edge.

[0100] A differential threshold is preset to further distinguish true crack edge pixels from non-crack edge pixels. The differential value of each candidate crack edge pixel in each direction is compared with the differential threshold. If the differential value of a pixel in one or more directions is greater than or equal to the differential threshold, the pixel is considered to be a target crack edge pixel and has a high probability of being a true crack edge. This effectively reduces false positives and retains and marks pixels that are more likely to be crack edges.

[0101] The directional continuity of the crack edge is predicted based on the distribution of target crack edge pixels and their differential values in different directions. This process involves analyzing the spatial arrangement of target crack edge pixels to determine the crack's orientation. Crack orientation generally exhibits a certain degree of continuity and directional variation. For example, longitudinal crack edges generally follow a straight line or arc, transverse cracks are perpendicular to them, and network-like cracks are multi-directional.

[0102] Based on the prediction of directional continuity, adjacent target crack edge pixels with consistent directions can be connected to construct the crack edge contour to form a complete crack shape, providing clearer structural information for subsequent crack feature extraction and identification.

[0103] The constructed crack edge contours are used as input to the feature extraction network, which extracts crack features. This feature extraction network, based on a convolutional neural network and consisting of multiple convolutional layers, can learn features such as crack shape and texture. The extracted crack features are then fed into a classifier. Based on the learned feature patterns, the classifier classifies and identifies cracks and outputs the final crack detection results, including information such as crack location, length, and type.

[0104] By using multi-directional differential operator templates, differential threshold screening, and directional continuity prediction, we can more accurately identify true crack edge pixels and improve crack detection accuracy. After constructing the crack edge outline, combined with feature extraction networks and classifier analysis, we can more comprehensively identify crack features and reduce missed and false detections. Furthermore, when constructing the crack edge outline, predicting directional continuity helps eliminate interference from random noise and non-crack features, making crack features clearer and more complete, thereby improving the reliability of detection results.

[0105] In one embodiment, constructing the crack edge profile may specifically include:

[0106] Extracting a first crack edge contour segment based on the target crack edge pixel and directional continuity, and using the first crack edge contour segment as a reference, expanding a first preset range of pixel areas at both ends of the first crack edge contour segment along the crack direction to form candidate crack regions;

[0107] Analyzing the degree of dispersion of the candidate crack regions in color and brightness;

[0108] If the degree of dispersion is greater than the average level of a normal area of an asphalt pavement, a crack edge contour is constructed based on the candidate crack area.

[0109] After determining the target crack edge pixels, the directional continuity of these pixels is analyzed to extract the first crack edge contour segment. For example, the first crack edge contour segment can be extracted by tracking the spatial arrangement of the target crack edge pixels. Using this first crack edge contour segment as a reference, an area is expanded along the direction of the crack at both ends to a certain range (for example, a distance of 5-10 pixels) to form candidate crack regions. The expansion range can be pre-set based on the width and morphological characteristics of the crack.

[0110] Perform color and brightness analysis on the pixels within the candidate crack region. Calculate the standard deviation or variance of the color and brightness values within the region to measure dispersion. A higher standard deviation or variance indicates greater variation in color and brightness among the pixels within the region, while a lower standard deviation or variance indicates relatively uniform pixels.

[0111] Compare the color and brightness dispersion of the candidate crack region to the average level of normal asphalt pavement areas. Normal pavement areas have relatively uniform color and brightness, resulting in lower dispersion. If the dispersion of the candidate crack region is significantly higher than the normal level, it indicates that the area is likely a crack, as cracks are often accompanied by discontinuities in the pavement material and variations in color and brightness. At this point, construct a crack edge contour based on the candidate crack region. This can be achieved by connecting edge pixels within the region or using a contour detection algorithm.

[0112] Different types of cracks have different widths. Initially, fine cracks may be only a few tenths of a millimeter wide. As they develop, they will become wider, and the pixel features at both ends of the crack will show symmetrical or gradual differences within the width range. Therefore, this embodiment takes the first crack edge contour segment as a benchmark, and expands the pixel area of the first preset range at both ends of the first crack edge contour segment along the direction of the crack to form a candidate crack area. By analyzing the color and brightness discreteness of the candidate crack area, the true crack edge can be identified more accurately, thereby improving the accuracy of crack detection. At the same time, discreteness analysis helps to distinguish cracks from normal road surface textures and other interfering features (such as stains, oil stains, etc.), thereby improving the reliability of the detection results. In addition, constructing a complete crack edge contour provides clearer structural information for subsequent crack feature extraction and classification identification, and helps to more comprehensively analyze the characteristics of the crack, such as length, width, shape, etc.

[0113] In one embodiment, constructing a crack edge contour based on the candidate crack region includes:

[0114] At a local level, taking the candidate crack area as the center, selecting an asphalt pavement area within a second preset range around the candidate crack area, and calculating the standard deviation of the local contrast between the candidate crack area and the asphalt pavement area;

[0115] At the global level, the standard deviation of the global contrast of all candidate crack regions of the same type in the target asphalt pavement image is calculated;

[0116] Set the lower standard deviation threshold for local contrast and the upper standard deviation threshold for global contrast;

[0117] When the standard deviation of the local contrast is greater than the lower limit threshold of the standard deviation and the standard deviation of the global contrast is less than the upper limit threshold of the standard deviation, the candidate crack area is taken as the target crack area, the crack edge of the target crack area is identified and predicted, and the outline of the crack edge is marked to obtain the crack edge outline.

[0118] The contrast between a crack and the surrounding pavement in a local area may vary due to factors such as lighting and stains, but from a global perspective, there are certain contrast patterns. For example, for a through-crack, while the contrast fluctuates along its entire length, the average value is relatively stable and exceeds the fluctuation amplitude of the random pavement texture. Therefore, at the local level, this embodiment selects a certain area of asphalt pavement surrounding a candidate crack region. The color or brightness contrast between the candidate crack region and the surrounding pavement region is calculated, and the standard deviation of this contrast is then calculated to quantify the local contrast variation.

[0119] At the global level, statistics are collected for all candidate crack regions of the same type in the entire target asphalt pavement image. The contrast changes in these regions are analyzed, and the standard deviation of the global contrast is calculated to understand the contrast distribution of the crack regions in the entire image.

[0120] Based on experience or experimental data, set the lower limit threshold of the local contrast standard deviation and the upper limit threshold of the global contrast standard deviation. For example, the lower limit threshold of the local contrast standard deviation can be set to 0.2, and the upper limit threshold of the global contrast standard deviation can be set to 0.3.

[0121] For each candidate crack region, we check whether its local contrast standard deviation is greater than the set lower standard deviation threshold, and whether its global contrast standard deviation is less than the set upper standard deviation threshold. If both conditions are met, the candidate crack region is considered a reliable target crack region. We then use an edge detection algorithm to identify and predict the crack edge of the target crack region, marking the crack edge outline and ultimately obtaining the crack edge contour.

[0122] By analyzing and calculating local and global contrast standard deviations, the true target crack area can be more accurately identified and the potential crack edges in the target crack area can be accurately predicted, thereby improving the accuracy of crack detection. In addition, contrast standard deviation analysis also helps to distinguish cracks from normal road surface texture and other interfering features, improving the reliability of detection results.

[0123] Preferably, constructing a crack edge profile based on the candidate crack region includes:

[0124] Dividing the candidate crack region into a plurality of overlapping analysis blocks, and calculating the angular standard deviation of the strike of the first edge contour segment in each of the analysis blocks;

[0125] When the angle standard deviation is greater than the angle threshold, it is determined that there is a first edge contour fragment with a disordered direction in the candidate crack area. The first edge contour fragment with a disordered direction is eliminated to form a target crack area, the crack edge of the target crack area is identified, and the contour of the crack edge is marked to obtain the crack edge contour.

[0126] This embodiment divides the candidate fracture area into multiple overlapping analysis blocks. For example, each block is 50×50 pixels in size with an overlap of 10 pixels. This helps capture the trend characteristics of the fracture in the local area while avoiding the problem of diluting the local characteristics due to over-division or inaccurate direction judgment due to over-division.

[0127] The direction of the first edge contour segment within each analysis block is analyzed. The angular variation between adjacent edge pixels is calculated to obtain the angular standard deviation of the edge direction within each analysis block. The angular standard deviation reflects the degree of variability of the edge direction within the analysis block. A larger standard deviation indicates a more irregular edge direction.

[0128] Set an angle threshold, such as 30°. If the angle standard deviation within a certain analysis block is greater than the angle threshold, the first edge contour segment within that block is considered to have a chaotic trend and does not conform to the relatively continuous and regular trend characteristics typically found in fractures. The analysis block with the chaotic first edge contour segment is removed from the candidate fracture region, and the remaining area is the target fracture region. An edge detection algorithm is used to identify the fracture edge of the target fracture region and mark the outline of the fracture edge, ultimately obtaining a clear and continuous fracture edge outline.

[0129] This embodiment can effectively eliminate fragments with chaotic directions by analyzing the standard deviation of the strike angles of edge contour fragments, retain the parts that meet the crack characteristics, and improve the accuracy of crack detection. It can also effectively distinguish between cracks and random noise or non-crack features in the road surface texture (such as the raised edges of stones, etc.), thereby improving the accuracy of the detection results.

[0130] In one embodiment, before analyzing the target asphalt pavement image using the asphalt pavement crack detection model and outputting the asphalt pavement crack detection result, the method further includes:

[0131] Acquire training samples, the training samples including standard asphalt pavement crack images of different crack types and corresponding labels, the labels being used to mark the locations and types of cracks in the standard asphalt pavement crack images;

[0132] A feature extraction network is constructed based on a convolutional neural network. Based on the feature extraction network, classifiers are laterally connected through a top-down path to form an initial asphalt pavement crack detection model.

[0133] Inputting the training samples into an initial asphalt pavement crack detection model for training, and during the model training, fusing feature maps of different levels generated from the standard asphalt pavement crack image to generate a multi-scale feature map;

[0134] Building a region proposal network on the multi-scale feature map and generating multiple anchor boxes at predetermined positions on each feature map, wherein the anchor boxes have different scales and ratios for preliminary screening of regions that may contain cracks;

[0135] In the feature extraction network, each feature map is processed using a convolutional layer, and a target anchor box is output from a classification branch, and an anchor box offset of the target anchor box is output from a regression branch. The classification branch is used to predict and output the probability that each anchor box belongs to a crack, and the regression branch is used to calculate and output the anchor box offset of each anchor box. The anchor box offset is used to adjust the position and size of the anchor box.

[0136] The target anchor frames of different scales and proportions are normalized using a bilinear interpolation algorithm, and the prediction results are output through a classifier;

[0137] The loss value of the initial asphalt pavement crack detection model is calculated based on a preset loss function and the prediction result. When the loss value is lower than the preset loss value, the trained initial asphalt pavement crack detection model is used as the asphalt pavement crack detection model.

[0138] This embodiment can collect a large number of standard asphalt pavement crack images of different crack types, including longitudinal cracks, transverse cracks, and reticular cracks, and add corresponding labels to each image. The labels detail the location (for example, using bounding boxes or pixel-level masks) and type of cracks in the image.

[0139] A feature extraction network based on a convolutional neural network is constructed. This network consists of multiple convolutional layers and is used to extract image features. Based on the feature extraction network, classifiers are laterally connected through a top-down path to form an initial asphalt pavement crack detection model. This structure helps integrate features at different levels, enhancing the model's ability to capture crack characteristics. Training samples are input into the initial asphalt pavement crack detection model for training. During the training process, feature maps at different levels generated from standard asphalt pavement crack images are fused to generate a multi-scale feature map. This multi-scale feature map captures crack characteristics of varying sizes and shapes, improving the model's detection capabilities.

[0140] A region proposal network is constructed on multi-scale feature maps. It generates multiple anchor boxes at predetermined locations on each feature map, each with varying scales and proportions, for preliminary screening of regions likely to contain cracks. The diversity and multi-scale nature of the anchor boxes enables the model to detect cracks of varying sizes and shapes. In the feature extraction network, convolutional layers are used to process each feature map. The classification branch outputs target anchor boxes, predicting and outputting the probability that each anchor box belongs to a crack. The regression branch outputs the anchor box offset for the target anchor box, which is used to adjust the anchor box's position and size to more accurately enclose the crack area.

[0141] The bilinear interpolation algorithm is used to normalize the target anchor frames of different scales and proportions to make them have a uniform size and format. The normalized anchor frames are input into the classifier, which outputs the final prediction results, including the location and type of the crack.

[0142] Finally, the loss value of the initial asphalt pavement crack detection model is calculated based on the preset loss function and the predicted results. This loss function measures the difference between the predicted results and the true labels. When the loss value is lower than the preset loss value, the model training is considered complete, and the initial model at this point is used as the final asphalt pavement crack detection model.

[0143] By leveraging multi-scale feature maps and a region proposal network (RPN), the model can capture crack features of varying sizes and shapes, improving crack detection accuracy. Using a large number of training samples with diverse crack types, the model can learn a variety of crack characteristics, enhancing its adaptability to diverse scenarios. Furthermore, the RPN generates anchor boxes of varying scales and proportions to preliminarily screen regions that may contain cracks, improving the model's detection efficiency. Furthermore, by combining classification and regression branches, the model can accurately locate cracks and classify their types, providing detailed crack information.

[0144] In one embodiment, after calculating the loss value of the initial asphalt pavement crack detection model based on the preset loss function and the prediction result, the method further includes:

[0145] When the loss value is not lower than the preset loss value, returning the gradient information of the preset loss function;

[0146] According to the gradient information, the optimizer is used to update the parameters of the initial asphalt pavement crack detection model, and the initial asphalt pavement crack detection model with updated parameters is retrained until the loss value is lower than the preset loss value, thereby obtaining the asphalt pavement crack detection model.

[0147] When training the initial asphalt pavement crack detection model, a preset loss function (such as the cross-entropy loss function, mean squared error loss function, etc.) is used to measure the difference between the model's prediction results and the true labels, thereby calculating the loss value of the initial model. The size of the loss value reflects the gap between the model's current prediction performance and the expected target. When the calculated loss value is not lower than the preset loss value, it means that the current performance of the model has not yet reached a satisfactory standard and requires further optimization. At this time, the gradient information of the preset loss function is returned through the backpropagation algorithm. The gradient information indicates the direction of model parameter adjustment, that is, which parameters need to be adjusted and how to adjust them to reduce the loss value, so that the model's prediction results are closer to the actual situation.

[0148] Based on the returned gradient information, an optimizer (such as stochastic gradient descent or Adam) is used to update the parameters of the initial asphalt pavement crack detection model. Based on the preset learning rate and gradient information, the optimizer automatically calculates new parameter values, moving the model in the direction of decreasing loss in parameter space.

[0149] The training samples are re-input into the initial asphalt pavement crack detection model with updated parameters for retraining. During retraining, the model relearns crack characteristics based on the new parameter values, generates new predictions, and recalculates the loss value. This process is repeated until the loss value falls below the preset loss value, resulting in a fully trained asphalt pavement crack detection model.

[0150] This embodiment enables the initial model to continuously optimize its performance by returning gradient information and updating model parameters, gradually reducing loss and improving crack detection accuracy. Through repeated training, the model learns more robust crack characteristics, enhancing its adaptability and generalization capabilities for crack detection in diverse scenarios. The resulting asphalt pavement crack detection model can output more accurate crack detection results in practical applications, including detailed information such as crack location and type. This helps to promptly detect and repair pavement damage, ensuring road safety and longevity.

[0151] Please refer to Figure 4 As shown, an embodiment of the present invention further provides an asphalt pavement crack detection system, comprising:

[0152] An acquisition module 11 is used to acquire an asphalt road surface image;

[0153] A preprocessing module 12 is configured to preprocess the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image;

[0154] The analysis module 13 is used to analyze the target asphalt pavement image using an asphalt pavement crack detection model and output an asphalt pavement crack detection result. The asphalt pavement crack detection model includes a feature extraction network and a classifier based on a convolutional neural network. The feature extraction network includes multiple convolutional layers for extracting crack features in the target asphalt pavement image, and the classifier is used to classify and identify the extracted crack features.

[0155] The asphalt pavement crack detection system provided by the present invention can effectively remove noise interference, enhance crack features, and improve image quality by acquiring asphalt pavement images and preprocessing the images according to a preset image processing method. In addition, the preprocessed images are analyzed using an asphalt pavement crack detection model based on a convolutional neural network, which includes a specially designed feature extraction network and classifier. The feature extraction network automatically learns the characteristic representation of cracks through multiple convolutional layers. When faced with complex asphalt pavement scenes, it is not easily affected by noise interference and uneven lighting problems of the asphalt pavement, and can accurately capture the various forms and detailed information of cracks; the classifier accurately classifies and identifies the extracted features, distinguishing cracks from non-crack areas, thereby fully utilizing the powerful feature learning ability and classification performance of the convolutional neural network, effectively improving the accuracy and efficiency of asphalt pavement crack detection, and achieving rapid and accurate detection of pavement cracks.

[0156] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0157] In one embodiment, the present invention further provides a storage medium storing computer-readable instructions. When executed by one or more processors, the computer-readable instructions cause the one or more processors to perform the above-described asphalt pavement crack detection method. The storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0158] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for detecting cracks in an asphalt pavement, characterized in that: include: Acquire asphalt road surface images; Preprocessing the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image; The target asphalt pavement image is analyzed using an asphalt pavement crack detection model to output asphalt pavement crack detection results. The asphalt pavement crack detection model includes a feature extraction network and a classifier based on a convolutional neural network. The feature extraction network includes multiple convolutional layers for extracting crack features in the target asphalt pavement image, and the classifier is used to classify and identify the extracted crack features.

2. The asphalt pavement crack detection method according to claim 1, characterized in that: The pre-processing of the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image includes: Evenly dividing the asphalt road surface image into a plurality of image blocks; Counting the number of pixels of each brightness value in each of the image blocks, and calculating the ratio of the number of pixels of each brightness value in each image block to the total number of pixels in the corresponding image block, to form a brightness distribution of each image block; Calculating the brightness adjustment amount that should be adjusted for each pixel in each image block according to the preset target brightness distribution and the brightness distribution of each image block; By using an interpolation algorithm, the brightness value of each pixel in each image block is adjusted accordingly according to the brightness adjustment amount that each pixel should be adjusted to obtain the target asphalt road surface image.

3. The asphalt pavement crack detection method according to claim 1, characterized in that: The pre-processing of the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image includes: selecting a plurality of sliding windows of corresponding sizes according to a minimum crack width and a texture granularity in the asphalt pavement image, and sliding the plurality of sliding windows on the asphalt pavement image; Calculating the average similarity between each pixel in each sliding window and surrounding pixels in terms of color, brightness, and gradient, and comparing the average similarity between each pixel in each sliding window and surrounding pixels with a similarity threshold; Marking the first pixel whose average similarity with surrounding pixels is lower than the similarity threshold as a candidate crack edge pixel; A second pixel whose average similarity with surrounding pixels is not less than the similarity threshold is marked as a normal road surface texture pixel, and the pixel value of the normal road surface texture pixel is adjusted according to a polynomial fitting method and the surrounding pixel values of the normal road surface texture pixel to obtain a target asphalt road surface image.

4. The asphalt pavement crack detection method according to claim 3, characterized in that: The asphalt pavement crack detection model is used to analyze the target asphalt pavement image and output the asphalt pavement crack detection result, including: Using an asphalt pavement crack detection model, the target asphalt pavement image is convolved with a preset multi-directional differential operator template to obtain differential values of each candidate crack edge pixel in the target asphalt pavement image in different directions, wherein the differential values are used to reflect the edge strength of the candidate crack edge pixel in the corresponding direction; Comparing the differential value of each candidate crack edge pixel in the target asphalt pavement image in different directions with a preset differential threshold to screen out target crack edge pixels; Predicting the directional continuity of the target crack edge pixels and constructing the crack edge contour; The feature extraction network is called to extract crack features of the crack edge contour, and the classifier is used to classify and identify the crack features to obtain asphalt pavement crack detection results.

5. The asphalt pavement crack detection method according to claim 4, characterized in that: The method of constructing the crack edge profile comprises: Extracting a first crack edge contour segment based on the target crack edge pixel and directional continuity, and using the first crack edge contour segment as a reference, expanding a first preset range of pixel areas at both ends of the first crack edge contour segment along the crack direction to form candidate crack regions; Analyzing the degree of dispersion of the candidate crack regions in color and brightness; If the degree of dispersion is greater than the average level of a normal area of an asphalt pavement, a crack edge contour is constructed based on the candidate crack area.

6. The asphalt pavement crack detection method according to claim 5, characterized in that: The constructing of a crack edge contour based on the candidate crack region includes: At a local level, taking the candidate crack area as the center, selecting an asphalt pavement area within a second preset range around the candidate crack area, and calculating the standard deviation of the local contrast between the candidate crack area and the asphalt pavement area; At the global level, the standard deviation of the global contrast of all candidate crack regions of the same type in the target asphalt pavement image is calculated; Set the lower standard deviation threshold for local contrast and the upper standard deviation threshold for global contrast; When the standard deviation of the local contrast is greater than the lower limit threshold of the standard deviation and the standard deviation of the global contrast is less than the upper limit threshold of the standard deviation, the candidate crack area is taken as the target crack area, the crack edge of the target crack area is identified and predicted, and the outline of the crack edge is marked to obtain the crack edge outline.

7. The asphalt pavement crack detection method according to claim 5, characterized in that: The constructing of a crack edge contour based on the candidate crack region includes: Dividing the candidate crack region into a plurality of overlapping analysis blocks, and calculating the angular standard deviation of the strike of the first edge contour segment in each of the analysis blocks; When the angle standard deviation is greater than the angle threshold, it is determined that there is a first edge contour fragment with a disordered direction in the candidate crack area. The first edge contour fragment with a disordered direction is eliminated to form a target crack area, the crack edge of the target crack area is identified, and the contour of the crack edge is marked to obtain the crack edge contour.

8. The asphalt pavement crack detection method according to claim 1, characterized in that: Before analyzing the target asphalt pavement image using the asphalt pavement crack detection model and outputting the asphalt pavement crack detection result, the method further includes: Acquire training samples, the training samples including standard asphalt pavement crack images of different crack types and corresponding labels, the labels being used to mark the locations and types of cracks in the standard asphalt pavement crack images; A feature extraction network is constructed based on a convolutional neural network. Based on the feature extraction network, classifiers are laterally connected through a top-down path to form an initial asphalt pavement crack detection model. Inputting the training samples into an initial asphalt pavement crack detection model for training, and during the model training, fusing feature maps of different levels generated from the standard asphalt pavement crack image to generate a multi-scale feature map; Building a region proposal network on the multi-scale feature map and generating multiple anchor boxes at predetermined positions on each feature map, wherein the anchor boxes have different scales and ratios for preliminary screening of regions that may contain cracks; In the feature extraction network, each feature map is processed using a convolutional layer, and a target anchor box is output from a classification branch, and an anchor box offset of the target anchor box is output from a regression branch. The classification branch is used to predict and output the probability that each anchor box belongs to a crack, and the regression branch is used to calculate and output the anchor box offset of each anchor box. The anchor box offset is used to adjust the position and size of the anchor box. The target anchor frames of different scales and proportions are normalized using a bilinear interpolation algorithm, and the prediction results are output through a classifier; The loss value of the initial asphalt pavement crack detection model is calculated based on a preset loss function and the prediction result. When the loss value is lower than the preset loss value, the trained initial asphalt pavement crack detection model is used as the asphalt pavement crack detection model.

9. The asphalt pavement crack detection method according to claim 8, characterized in that: After calculating the loss value of the initial asphalt pavement crack detection model based on the preset loss function and the prediction result, the method further includes: When the loss value is not lower than the preset loss value, returning the gradient information of the preset loss function; According to the gradient information, the optimizer is used to update the parameters of the initial asphalt pavement crack detection model, and the initial asphalt pavement crack detection model with updated parameters is retrained until the loss value is lower than the preset loss value, thereby obtaining the asphalt pavement crack detection model.

10. An asphalt pavement crack detection system, characterized in that: include: An acquisition module, used for acquiring an asphalt road surface image; a preprocessing module, configured to preprocess the asphalt pavement image according to a preset image processing method to obtain a target asphalt pavement image; An analysis module is used to analyze the target asphalt pavement image using an asphalt pavement crack detection model and output asphalt pavement crack detection results. The asphalt pavement crack detection model includes a feature extraction network and a classifier based on a convolutional neural network. The feature extraction network includes multiple convolutional layers for extracting crack features in the target asphalt pavement image, and the classifier is used to classify and identify the extracted crack features.

Images