Asphalt pavement crack detection method and system based on image analysis, electronic equipment and storage medium

Abstract

The invention belongs to the field of road maintenance detection, and discloses an asphalt pavement crack detection method and system based on image analysis, electronic equipment and a storage medium, and the method comprises the following steps: collecting an image of an asphalt pavement, and preprocessing the collected image to obtain a first image; segmenting the preprocessed image by using a full convolutional neural network, and performing morphological processing on the segmented image to obtain a second image; calculating a gray level co-occurrence matrix of the second image and extracting texture features, detecting edge information of the second image by using an edge detection algorithm, and combining the texture features with the edge information to obtain a crack candidate area; constructing a convolutional neural network model, and training the convolutional neural network model through the historical crack image and the historical non-crack image to obtain a crack recognition model; and inputting the crack candidate region into a crack identification model for crack identification to obtain a crack identification result.

Description

Technical Field

[0001] The present invention relates to the technical field of highway maintenance detection, and specifically relates to a method, system, electronic device and storage medium for detecting asphalt pavement cracks based on image analysis. Background Technique

[0002] In modern society, the booming development of the transportation industry has continuously expanded the application scope of asphalt pavements. However, under the influence of multiple factors such as repeated vehicle rolling and climate change, asphalt pavements are extremely prone to cracks. These cracks may initially be small and inconspicuous, but if not discovered and treated in time, they will gradually expand, ultimately causing serious damage to the pavement structure, such as potholes and looseness, which not only reduces the service life of the road but also seriously threatens driving safety.

[0003] Traditional detection methods mainly rely on manual visual inspection. Detection personnel observe the pavement conditions with the naked eye and record crack information. This method has many defects. On the one hand, the efficiency of manual detection is extremely low. Facing large areas of asphalt pavement, it consumes a lot of manpower and material resources and is difficult to complete a comprehensive inspection in a short time. On the other hand, manual visual inspection is highly subjective. Due to differences in experience and eyesight among different detection personnel, the criteria for crack judgment are inconsistent, resulting in poor accuracy and large errors in the detection results, and unable to provide accurate data for road maintenance. In addition, in complex road conditions or harsh environments, such as high traffic volume and fast driving speed on highways, or low visibility situations such as at night, in rainy or foggy weather, the difficulty of manual detection increases sharply, and even cannot be carried out, posing a great safety hazard.

[0004] With the development of technology, although there are automated detection methods based on simple optical devices or traditional image processing technologies, there are still limitations. Some optical devices have strict requirements for light conditions, and strong light or shadows are likely to interfere with the detection results; traditional image processing technologies are difficult to accurately segment the asphalt pavement area in the face of complex pavement images, such as interference factors like debris, markings, and water accumulation, affecting the crack recognition and extraction effects, and the detection accuracy is difficult to meet the actual engineering requirements. Therefore, there is an urgent need for a more efficient and accurate method for detecting asphalt pavement cracks. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a method for detecting asphalt pavement cracks based on image analysis, and the method includes the following steps:

[0006] Collect an image of the asphalt pavement and preprocess the collected image to obtain a first image;

[0007] Use a fully convolutional neural network to segment the preprocessed image and perform morphological processing on the segmented image to obtain a second image;

[0008] Calculate the gray-level co-occurrence matrix of the second image and extract texture features. Use an edge detection algorithm to detect the edge information of the second image, and combine the texture features and the edge information to obtain a crack candidate region;

[0009] Construct a convolutional neural network model, and train the convolutional neural network model with historical crack images and historical non-crack images to obtain a crack recognition model;

[0010] Input the crack candidate region into the crack recognition model for crack recognition to obtain a crack recognition result.

[0011] Preferably, the method of preprocessing includes:

[0012] Perform denoising processing on the collected image through mean filtering and median filtering to obtain a denoised image;

[0013] Use the weighted average method to grayscale the denoised image to obtain a grayscaled image;

[0014] Perform image enhancement on the grayscaled image through grayscale histogram equalization to obtain the first image.

[0015] Preferably, the method of obtaining the second image includes:

[0016] Input the first image into the fully convolutional neural network, use the encoder part of the network to extract the high-level feature map of the image, and then perform upsampling through the decoder part to restore the high-level feature map to the resolution of the original image to complete image segmentation;

[0017] Use a structuring element to erode the segmented image to remove regions smaller than the structuring element in the image, and then use the same structuring element to dilate the eroded image to obtain the second image.

[0018] Preferably, the method of obtaining the crack candidate region includes:

[0019] Perform quantization processing on the second image at the gray level, calculate the gray-level co-occurrence matrix using the quantized image, and then extract the texture features through the gray-level co-occurrence matrix. The texture features include: contrast, energy, correlation, and homogeneity;

[0020] Perform filtering processing on the second image through Gaussian filtering and calculate the gradient of the image. Perform non-maximum suppression on the gradient magnitude image, and use double thresholds to perform edge detection on the suppressed image to obtain the edge information;

[0021] Fuse the texture feature and the edge information with weights to obtain a comprehensive feature, and segment the comprehensive feature using threshold segmentation to obtain the crack candidate region.

[0022] The present invention also provides an asphalt pavement crack detection system based on image analysis. The system is used to implement the method described above in the claims, and includes: an image acquisition module, an image segmentation module, a candidate region extraction module, a model construction module, and a crack recognition module;

[0023] The image acquisition module is used to acquire an image of the asphalt pavement and preprocess the acquired image to obtain a first image;

[0024] The image segmentation module uses a fully convolutional neural network to segment the preprocessed image and perform morphological processing on the segmented image to obtain a second image;

[0025] The candidate region module is used to calculate the gray-level co-occurrence matrix of the second image and extract texture features, detect the edge information of the second image using an edge detection algorithm, and combine the texture features and the edge information to obtain a crack candidate region;

[0026] The model construction module is used to construct a convolutional neural network model and train the convolutional neural network model with historical crack images and historical non-crack images to obtain a crack recognition model;

[0027] The crack recognition module is used to input the crack candidate region into the crack recognition model for crack recognition to obtain a crack recognition result.

[0028] Preferably, the image acquisition module includes: an image acquisition unit, a denoising unit, a grayscale conversion unit, and an image enhancement unit;

[0029] The image acquisition unit is used to acquire an image of the asphalt pavement;

[0030] The denoising unit performs denoising processing on the acquired image through mean filtering and median filtering to obtain a denoised image;

[0031] The grayscale conversion unit grayscales the denoised image using the weighted average method to obtain a grayscaled image;

[0032] The image enhancement unit performs image enhancement on the grayscaled image through grayscale histogram equalization to obtain the first image.

[0033] Preferably, the image segmentation module includes: an image segmentation unit and a morphological processing unit

[0034] The image segmentation unit is used to input the first image into the fully convolutional neural network, extract the high-level feature map of the image by using the encoder part of the network, and then perform upsampling through the decoder part to restore the high-level feature map to the resolution of the original image, completing image segmentation;

[0035] The morphological processing unit uses a structuring element to erode the segmented image, removing regions in the image that are smaller than the structuring element, and then uses the same structuring element to dilate the eroded image to obtain the second image.

[0036] Preferably, the candidate region extraction module includes: a texture extraction unit, an edge extraction unit, and a region extraction unit;

[0037] The texture extraction unit is used to perform quantization processing on the second image in terms of gray level, calculate the gray-level co-occurrence matrix using the quantized image, and then extract the texture features through the gray-level co-occurrence matrix. The texture features include: contrast, energy, correlation, and homogeneity;

[0038] The edge extraction unit filters the second image through Gaussian filtering and calculates the gradient of the image, performs non-maximum suppression on the gradient magnitude image, and uses double thresholds to perform edge detection on the suppressed image to obtain the edge information;

[0039] The region extraction unit is used to perform weighted fusion on the texture features and the edge information to obtain a comprehensive feature, and perform segmentation on the comprehensive feature using threshold segmentation to obtain the crack candidate region.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned asphalt pavement crack detection method based on image analysis is implemented.

[0041] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the above-mentioned asphalt pavement crack detection method based on image analysis is implemented.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] The present invention uses a high-definition camera to collect images and adopts advanced image processing and deep learning algorithms for analysis and processing, which can complete the crack detection of large-area asphalt pavements in a short time, greatly improving the detection efficiency; through multi-step processing such as image segmentation, feature extraction, and crack recognition, it can accurately identify the cracks in the asphalt pavement, and has a good detection effect on cracks of different shapes and widths, reducing the false detection rate and the missed detection rate. The entire detection process of the present invention is automated without manual intervention, reducing the influence of human factors on the detection results and improving the objectivity and reliability of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 It is a schematic flowchart of the method according to an embodiment of the present invention;

[0046] Figure 2 It is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0048] 1010, processor; 1020, memory; 1030, input / output interface; 1040, communication interface; 1050, bus. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the ordinary meanings understood by those of ordinary skill in the field to which the present disclosure belongs. The "first", "second" and similar terms used in the embodiments of the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Words such as "including" or "comprising" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. Words such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0051] Embodiment 1

[0052] In this embodiment, as Figure 1 shown, a method for detecting cracks in asphalt pavement based on image analysis, the method includes the following steps:

[0053] S1. Collect an image of the asphalt pavement and preprocess the collected image to obtain a first image.

[0054] The preprocessing method includes: denoising the collected image through mean filtering and median filtering to obtain a denoised image; graying the denoised image using the weighted average method to obtain a grayed image; and enhancing the grayed image through gray histogram equalization to obtain the first image.

[0055] In this embodiment, mean filtering achieves the denoising effect by calculating the average value of the pixel values in the neighborhood around each pixel point in the image. Taking a window of size k×k as an example, let the value of the pixel at the center of the window be I(x,y), and the values of the pixels in its neighborhood be I(x+i,y+j), where the values of i and j satisfy the window boundary conditions. The calculation formula for the pixel value Iavg(x,y) after mean filtering is:

[0056]

[0057] Median filtering, on the other hand, selects the pixel values in the neighborhood of each pixel point in the image, sorts them by size, and takes the median as the filtering result. For a window of size k×k, the k 2 pixel values in the neighborhood are sorted, and the th number is the median. The calculation formula for median filtering is:

[0058]

[0059] Among them, k represents the window size, and the denoised image is obtained through these two filtering methods. Image grayscale conversion is the process of converting a color image into a grayscale image, and its formula is:

[0060] I gray (x, y) = w R ·R(x, y) + w G ·G(x, y) + w B ·B(x, y)

[0061] Among them, w R represents the weight of the red channel, R(x, y) represents the red pixel value of the pixel, w G represents the weight of the green channel, G(x, y) represents the red pixel value of the pixel, w B represents the weight of the blue channel, B(x, y) represents the red pixel value of the pixel, and w R + w G + w B = 1. The grayscale image is obtained by grayscale converting the denoised image. Let the grayscale level of the grayscale image be L levels, and its grayscale histogram be h(r t ), which represents the number of times the pixel with grayscale level r t appears. The probability density is calculated using the histogram:

[0062]

[0063] Among them, n is the total number of pixels in the image; the cumulative distribution function CDF is calculated through the probability density:

[0064]

[0065] Then, the equalized grayscale value s t is calculated through the cumulative distribution function:

[0066] s t = (L - 1)·c(r t )

[0067] Round s t to the nearest integer to obtain the pixel value with the equalized grayscale level s t , thus completing image enhancement and obtaining the first image.

[0068] S2. Use a fully convolutional neural network to segment the preprocessed image and perform morphological processing on the segmented image to obtain the second image.

[0069] The method for obtaining the second image includes: inputting the first image into a fully convolutional neural network, using the encoder part of the network to extract the high-level feature map of the image, and then performing upsampling through the decoder part to restore the high-level feature map to the resolution of the original image to complete image segmentation; using a structuring element to erode the segmented image to remove regions in the image smaller than the structuring element, and then using the same structuring element to dilate the eroded image to obtain the second image.

[0070] In this embodiment, first, the preprocessed image is applied to a fully convolutional neural network (FCN) for segmentation. The fully convolutional neural network (FCN) is a deep learning model specifically for semantic segmentation, which can classify each pixel in the image, thereby achieving precise segmentation of the crack area in the asphalt pavement image. In this process, the FCN extracts features and performs upsampling on the image through multiple convolutional layers and deconvolutional layers to generate a segmentation result with the same size as the input image. Specifically, the first image is input into the fully convolutional neural network, and the high-level features of the image are extracted through the encoder part of the model (usually a pre-trained convolutional neural network, such as the VGG network), and then the feature map is restored to the resolution of the original image using the upsampling operation of the decoder part. Finally, a segmentation map is output, in which each pixel is labeled as belonging to the crack area or the background area.

[0071] After that, morphological processing is performed on the segmented image, mainly using erosion and dilation operations to optimize the segmentation result. The erosion operation can remove some small noise points in the segmentation map, while the dilation operation can fill the holes in the segmented area to make the edges of the crack area smoother. Specifically, a structuring element (usually a small square or circle) is used to erode the segmentation map. The erosion process will remove regions in the image smaller than the structuring element, thereby eliminating some fine noises and discontinuous points; after erosion, the same structuring element is used to dilate the image. The dilation operation will expand the edges of the segmented area outward to fill the crack area that may be lost due to the erosion operation. At the same time, dilation can also connect some adjacent but not fully connected crack areas to make the overall crack area more continuous. By combining erosion and dilation (usually called opening or closing operations), the second image is obtained.

[0072] S3. Calculate the gray-level co-occurrence matrix of the second image and extract texture features, use an edge detection algorithm to detect the edge information of the second image, and combine the texture features and edge information to obtain the crack candidate area.

[0073] The method for obtaining the crack candidate region includes: performing gray-level quantization on the second image, calculating the gray-level co-occurrence matrix using the quantized image, and then extracting texture features from the gray-level co-occurrence matrix. The texture features include: contrast, energy, correlation, and homogeneity; performing filtering on the second image through Gaussian filtering and calculating the gradient of the image, performing non-maximum suppression on the gradient magnitude image, and using double thresholds to perform edge detection on the suppressed image to obtain edge information; performing weighted fusion on the texture features and the edge information to obtain comprehensive features, and using threshold segmentation to segment the comprehensive features to obtain the crack candidate region.

[0074] In this embodiment, first, the second image is subjected to gray-level quantization, quantizing the original gray level into 64 levels or 16 levels, and then calculating the gray-level co-occurrence matrix. The calculation of the gray-level co-occurrence matrix requires determining the direction and distance: (1) Four directions of 0°, 45°, 90°, and 135° are selected for the direction, representing pixel pairs in the horizontal, right diagonal, vertical, and left diagonal directions respectively; (2) The distance represents the distance between pixel pairs. After determining the direction and distance, count the number of occurrences of each pair of pixels (i,j) in the specified direction and distance, and normalize it to obtain the gray-level co-occurrence matrix. Then, extract the following common texture features from the gray-level co-occurrence matrix:

[0075] (1) Contrast: Measures the degree of difference in pixel gray values in the image, and the calculation formula is:

[0076]

[0077] where P(i,j) is the value of the (i,j)th element in the gray-level co-occurrence matrix; (2) Energy: Reflects the uniformity of the image texture, and the calculation formula is:

[0078]

[0079] (3) Correlation: Describes the degree of correlation of pixel gray values in the image, and the calculation formula is:

[0080]

[0081] where μ i and μ j are the means of the gray values i and j respectively, and σ i and σ j are the standard deviations of the gray values i and j respectively; (4) Homogeneity: Measures the degree of proximity of pixel gray values in the image, and the calculation formula is:

[0082]

[0083] Through the above steps, the texture features are obtained.

[0084] After that, the classical Canny edge detection algorithm is adopted, and its specific steps are as follows: perform Gaussian filtering on the second image to remove noise and smooth the image. The kernel function of Gaussian filtering is:

[0085]

[0086] where σ represents the standard deviation, which can be adjusted according to the noise level of the image; use the Sobel operator to calculate the gradients of the image in the horizontal and vertical directions to obtain the gradient magnitude and direction:

[0087]

[0088] where G x and G y respectively represent the gradient components in the horizontal and vertical directions, and θ(x, y) represents the direction of the gradient. For each pixel point, judge whether its gradient magnitude is a local maximum: if so, retain the gradient magnitude of this point; otherwise, set its gradient magnitude to 0. Use two thresholds, a high threshold and a low threshold, to perform edge detection on the image after non-maximum suppression. The low threshold is used to detect weak edges, and the high threshold is used to detect strong edges. The strong edges are used as the initial seed points of the edges. Whether the weak edges belong to the edges is judged by their connectivity with the strong edges. By tracking the connectivity of the edge pixels, the isolated edge points are connected into continuous edges. Through the above steps, the edge information of the second image can be obtained for subsequent extraction of crack candidate regions.

[0089] Finally, the texture features and the edge information are weighted and fused to obtain the comprehensive feature F:

[0090] F = α·T + β·E

[0091] where T represents the texture features, α represents the weight of the texture features, E represents the edge information, β represents the weight of the edge information, and α + β = 1; perform threshold segmentation on the comprehensive feature F to obtain the crack candidate region, and the threshold θ can be determined by experiments or an adaptive method. The segmentation formula is:

[0092]

[0093] where R(x, y) represents the crack candidate region.

[0094] S4. Construct a convolutional neural network model, and train the convolutional neural network model with historical crack images and historical non-crack images to obtain a crack recognition model. In this embodiment, the convolutional neural network model is stacked by an input layer, multiple convolutional layers, pooling layers, fully connected layers, and an output layer, and can automatically extract the features of the image and perform classification.

[0095] S5. Input the crack candidate region into the crack recognition model for crack recognition to obtain the crack recognition result.

[0096] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In this case of a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiments of the present disclosure, and these multiple devices will interact with each other to complete the described method.

[0097] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, it should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention. The actions or steps recorded in the claims can be executed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] Embodiment 2

[0099] In this embodiment, an asphalt pavement crack detection system based on image analysis includes: an image acquisition module, an image segmentation module, a candidate region extraction module, a model construction module, and a crack recognition module.

[0100] The image acquisition module is used to acquire an image of the asphalt pavement and preprocess the acquired image to obtain a first image.

[0101] The image acquisition module includes: an image acquisition unit, a denoising unit, a grayscale unit, and an image enhancement unit; the image acquisition unit is used to acquire an image of the asphalt pavement; the denoising unit performs denoising processing on the acquired image through mean filtering and median filtering to obtain a denoised image; the grayscale unit grayscales the denoised image using the weighted average method to obtain a grayscaled image; the image enhancement unit enhances the grayscaled image through grayscale histogram equalization to obtain a first image.

[0102] The image segmentation module uses a fully convolutional neural network to segment the preprocessed image and performs morphological processing on the segmented image to obtain a second image.

[0103] The image segmentation module includes: an image segmentation unit and a morphological processing unit. The image segmentation unit is used to input the first image into a fully convolutional neural network, extract the high-level feature map of the image using the encoder part of the network, and then perform upsampling through the decoder part to restore the high-level feature map to the resolution of the original image, completing image segmentation. The morphological processing unit uses a structuring element to erode the segmented image, removing regions in the image smaller than the structuring element, and then uses the same structuring element to dilate the eroded image to obtain the second image.

[0104] The candidate region module is used to calculate the gray-level co-occurrence matrix of the second image and extract texture features, detect the edge information of the second image using an edge detection algorithm, and combine the texture features and edge information to obtain crack candidate regions.

[0105] The candidate region extraction module includes: a texture extraction unit, an edge extraction unit, and a region extraction unit. The texture extraction unit is used to perform quantization processing on the second image at the gray level, calculate the gray-level co-occurrence matrix using the quantized image, and then extract texture features through the gray-level co-occurrence matrix. The texture features include: contrast, energy, correlation, and homogeneity. The edge extraction unit filters the second image through Gaussian filtering and calculates the gradient of the image, performs non-maximum suppression on the gradient magnitude image, and uses a double threshold to detect edges in the suppressed image to obtain edge information. The region extraction unit is used to perform weighted fusion on the texture features and edge information to obtain a comprehensive feature, and perform segmentation on the comprehensive feature using threshold segmentation to obtain crack candidate regions.

[0106] The model construction module is used to construct a convolutional neural network model and train the convolutional neural network model using historical crack images and historical non-crack images to obtain a crack recognition model.

[0107] The crack recognition module is used to input the crack candidate regions into the crack recognition model for crack recognition to obtain crack recognition results.

[0108] The system of the above embodiments is used to implement the corresponding asphalt pavement crack detection method based on image analysis in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0109] It should be noted that the above asphalt pavement crack detection system based on image analysis is embodied in the form of functional units. The term "module" here can be implemented in software and / or hardware forms, and no specific limitation is made thereto.

[0110] For example, a "module" may be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group of processors, etc.) for executing one or more software or firmware programs, and a memory, a combined logic circuit, and / or other suitable components that support the described functions.

[0111] Embodiment 3

[0112] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the asphalt pavement crack detection method based on image analysis described in any one of the above embodiments.

[0113] Figure 2 FIG. shows a more specific schematic diagram of the hardware structure of the electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. Among them, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other inside the device through the bus 1050.

[0114] The processor 1010 may be implemented in the form of a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0115] The memory 1020 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0116] The input / output interface 1030 is used to connect to the input / output module to achieve information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Among them, the input devices can include keyboards, mice, touchscreens, microphones, various sensors, etc., and the output devices can include displays, speakers, vibrators, indicator lights, etc.

[0117] The communication interface 1040 is used to connect to the communication module (not shown in the figure) to achieve communication interaction between this device and other devices. Among them, the communication module can achieve communication through wired means (such as USB (Universal Serial Bus), network cable, etc.) or through wireless means (such as mobile network, WIFI (Wireless Fidelity), Bluetooth, etc.).

[0118] The bus 1050 includes a path for transmitting information between various components of the device (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0119] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary to implement the solution of the embodiments of this specification, and do not have to include all the components shown in the figure.

[0120] The system of the above embodiment is used to implement the corresponding asphalt pavement crack detection method based on image analysis in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0121] Embodiment 4

[0122] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present invention also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the asphalt pavement crack detection method based on image analysis as described in any of the foregoing embodiments.

[0123] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0124] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the asphalt pavement crack detection method based on image analysis described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0125] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples; under the concept of the present disclosure, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.

[0126] In addition, for the sake of simplicity of description and discussion, and in order not to make this embodiment difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. In addition, the device may be shown in block diagram form to avoid making this embodiment difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which this embodiment is to be implemented (i.e., these details should be fully within the understanding of those skilled in the art). In the case where specific details (such as circuits) are set forth to describe the exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the present disclosure embodiments can be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0127] Although the present disclosure has been described in connection with specific embodiments, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0128] Thus, the units of the examples described in the embodiments of the present application can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0129] Embodiments of the present disclosure are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for detecting cracks in asphalt pavement based on image analysis, characterized in that, The method includes the following steps: Collect images of the asphalt pavement and preprocess the collected images to obtain the first image; Use a fully convolutional neural network to segment the preprocessed image and perform morphological processing on the segmented image to obtain the second image; Calculate the gray-level co-occurrence matrix of the second image and extract texture features, use an edge detection algorithm to detect the edge information of the second image, and combine the texture features and the edge information to obtain a crack candidate region; Construct a convolutional neural network model and train the convolutional neural network model with historical crack images and historical non-crack images to obtain a crack recognition model; Input the crack candidate region into the crack recognition model for crack recognition to obtain a crack recognition result.

2. The method for detecting asphalt pavement cracks based on image analysis according to claim 1, wherein The method of the preprocessing includes: Perform denoising processing on the collected image through mean filtering and median filtering to obtain a denoised image; Use the weighted average method to grayscale the denoised image to obtain a grayscaled image; Perform image enhancement on the grayscaled image through grayscale histogram equalization to obtain the first image.

3. The method for detecting asphalt pavement cracks based on image analysis according to claim 1, wherein The method of obtaining the second image includes: Input the first image into the fully convolutional neural network, use the encoder part of the network to extract the high-level feature map of the image, and then perform upsampling through the decoder part to restore the high-level feature map to the resolution of the original image to complete image segmentation; Use a structuring element to erode the segmented image to remove regions smaller than the structuring element in the image, and then use the same structuring element to dilate the eroded image to obtain the second image.

4. The asphalt pavement crack detection method based on image analysis according to claim 1, characterized in that The method of obtaining the crack candidate region includes: Perform quantization processing on the gray level of the second image, calculate the gray-level co-occurrence matrix using the quantized image, and then extract the texture features through the gray-level co-occurrence matrix. The texture features include: contrast, energy, correlation, and homogeneity; Perform filtering processing on the second image through Gaussian filtering and calculate the gradient of the image, perform non-maximum suppression on the gradient magnitude image, and use double thresholds to perform edge detection on the suppressed image to obtain the edge information; Perform weighted fusion on the texture features and the edge information to obtain a comprehensive feature, and use threshold segmentation to segment the comprehensive feature to obtain the crack candidate region.

5. An asphalt pavement crack detection system based on image analysis, the system is used to implement the method described in any one of claims 1-4, characterized in that, Includes: An image acquisition module, an image segmentation module, a candidate region extraction module, a model construction module, and a crack recognition module; The image acquisition module is used to collect images of the asphalt pavement and preprocess the collected images to obtain the first image; The image segmentation module uses a fully convolutional neural network to segment the preprocessed image and perform morphological processing on the segmented image to obtain the second image; The candidate region module is used to calculate the gray-level co-occurrence matrix of the second image and extract texture features, use an edge detection algorithm to detect the edge information of the second image, and combine the texture features and the edge information to obtain a crack candidate region; The model construction module is used to construct a convolutional neural network model, and train the convolutional neural network model with historical crack images and historical non-crack images to obtain a crack recognition model; The crack recognition module is used to input the crack candidate region into the crack recognition model for crack recognition to obtain a crack recognition result.

6. The asphalt pavement crack detection system based on image analysis according to claim 5, wherein The image acquisition module includes: an image acquisition unit, a denoising unit, a grayscale unit, and an image enhancement unit; The image acquisition unit is used to acquire images of the asphalt pavement; The denoising unit performs denoising processing on the acquired image through mean filtering and median filtering to obtain a denoised image; The grayscale unit grayscales the denoised image using the weighted average method to obtain a grayscaled image; The image enhancement unit performs image enhancement on the grayscaled image through grayscale histogram equalization to obtain the first image.

7. The asphalt pavement crack detection system based on image analysis according to claim 5, wherein The image segmentation module includes: an image segmentation unit and a morphological processing unit The image segmentation unit is used to input the first image into the fully convolutional neural network, extract the high-level feature map of the image using the encoder part of the network, and then perform upsampling through the decoder part to restore the high-level feature map to the resolution of the original image to complete image segmentation; The morphological processing unit uses a structuring element to erode the segmented image to remove regions in the image smaller than the structuring element, and then uses the same structuring element to dilate the eroded image to obtain the second image.

8. The asphalt pavement crack detection system based on image analysis according to claim 5, characterized in that, The candidate region extraction module includes: a texture extraction unit, an edge extraction unit, and a region extraction unit; The texture extraction unit is used to perform quantization processing on the second image at the grayscale level, calculate the gray-level co-occurrence matrix using the quantized image, and then extract the texture features through the gray-level co-occurrence matrix. The texture features include: contrast, energy, correlation, and homogeneity; The edge extraction unit filters the second image through Gaussian filtering and calculates the gradient of the image, performs non-maximum suppression on the gradient magnitude image, and uses double thresholds to perform edge detection on the suppressed image to obtain the edge information; The region extraction unit is used to perform weighted fusion on the texture features and the edge information to obtain a comprehensive feature, and perform segmentation on the comprehensive feature using threshold segmentation to obtain the crack candidate region.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the asphalt pavement crack detection method based on image analysis according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the asphalt pavement crack detection method based on image analysis according to any one of claims 1 to 4.

Images