A method, device, equipment and storage medium for detecting building cracks

Abstract

The embodiment of the present invention discloses a method, device, equipment and storage medium for detecting building cracks. The method comprises: obtaining an image to be detected of a building surface to be detected; using a fully convolutional neural network classification model to identify a unit crack region in the image to be detected as an image of a region of interest; using an image segmentation model based on deep learning to extract cracks in the image of the region of interest to obtain a crack feature region segmentation image; binarizing the crack feature region segmentation image to obtain a unit crack detection map; and confirming a crack detection result map of the image to be detected based on the unit crack detection map. The method solves the problems of long operation time and low robustness of crack extraction in the current sliding window method, as well as the problem of poor crack detection effect caused by the serious imbalance of data between the crack region and the background region during model training in the image segmentation method, thereby achieving the effect of improving the accuracy of crack detection.

Description

Technical Field

[0001] The embodiments of the present invention relate to building crack detection technology, and more particularly to a building crack detection method, device, equipment and storage medium. Background Art

[0002] Concrete structures used in various construction projects, such as bridges, dams, and industrial buildings, will gradually deteriorate in health during long-term service. Cracks are one of the most common and major diseases of concrete, so it is very important to regularly detect crack defects in concrete structures. Traditional manual inspection methods have problems such as low efficiency and inconvenience in high-altitude facade inspection.

[0003] In recent years, the building structure health detection method based on computer vision has attracted widespread attention from academia and industry. This method has the advantages of non-contact sensing, low cost, simple operation, and no environmental impact. Among them, image detection technology for concrete surface cracks is one of the most active research directions. In order to realize the automatic analysis of crack length, width, type and other parameters, the automatic recognition and extraction technology of cracks is particularly important. The method based on deep learning is superior to the traditional image processing method in the performance of concrete surface crack detection. The method based on deep learning can be roughly divided into two categories: sliding window method and image segmentation method.

[0004] The sliding window-based method can only input images according to the sliding window size for calculation, which is very time-consuming. The threshold segmentation method used in crack extraction causes the crack extraction to be wider than the actual crack. At the same time, it is difficult to remove the interference noise in the background area, and the robustness is poor. In the image segmentation method, there is a serious imbalance in the data between the crack area and the background area during model training, and the crack area features are not significant compared to the background area during crack detection, which will affect the detection effect. Summary of the invention

[0005] The present invention provides a building crack detection method, device, equipment and storage medium to achieve the effect of improving the crack detection accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting cracks in a building, comprising:

[0007] Acquire an image to be inspected of a building surface to be inspected;

[0008] Using a fully convolutional neural network classification model to identify the unit crack region in the image to be detected as an image of a region of interest;

[0009] Using a deep learning-based image segmentation model to extract cracks in the image of the region of interest, and obtaining a crack feature region segmentation image;

[0010] After the image is segmented and binarized according to the crack feature region, a unit crack detection map is obtained;

[0011] The crack detection result map of the image to be detected is confirmed according to the unit crack detection map.

[0012] Optionally, the adopting of a fully convolutional neural network classification model to identify a unit crack region in the image to be detected as a region of interest image includes:

[0013] The preset detection frame performs scanning discrimination on the image to be detected with a preset step length to obtain a probability feature map composed of probability values ​​of all detection frames in the image to be detected containing cracks;

[0014] An image corresponding to a detection box with a probability value greater than the first probability value in the probability feature map is selected as a region of interest image.

[0015] Optionally, the preset detection frame is a square area of ​​a preset size, and the preset step size is 1 / 4 of the side length of the preset detection frame.

[0016] Optionally, the ratio of positive and negative sample images in the training sample images of the fully convolutional neural network classification model is 1:3; wherein, the positive sample image is a regional image with a crack area ratio greater than or equal to a first threshold, the negative sample image is a regional image with a crack area ratio less than or equal to a second threshold, and the regional image with a crack area ratio less than the first threshold and greater than the second threshold is an ignored sample.

[0017] Optionally, regional images in which the crack area ratio is greater than or equal to a first threshold are used as training samples for the image segmentation model.

[0018] Optionally, confirming the crack detection result map of the image to be detected according to the unit crack detection map includes:

[0019] Paste at least one of the unit crack detection images onto a pure black image of the same size as the image to be detected according to the corresponding position information to obtain a crack detection result image of the image to be detected; the position information is the position of the unit crack detection image in the corresponding region of interest in the image.

[0020] In a second aspect, an embodiment of the present invention further provides a building crack detection device, comprising:

[0021] An image acquisition module, used for acquiring an image to be detected of a building surface to be detected;

[0022] A first image region extraction module is used to identify a unit crack region in the image to be detected as a region of interest image using a fully convolutional neural network classification model;

[0023] A second image region extraction module is used to extract cracks in the image of the region of interest using a deep learning image segmentation model to obtain a crack feature region segmentation image;

[0024] A crack detection module, used to obtain a unit crack detection map after segmenting the image according to the crack feature area and performing binarization;

[0025] The crack result confirmation module is used to confirm the crack detection result map of the image to be detected according to the unit crack detection map.

[0026] In a third aspect, an embodiment of the present invention further provides a building crack detection device, the device comprising:

[0027] one or more processors;

[0028] a storage device for storing one or more programs,

[0029] When the one or more programs are executed by the one or more processors, the one or more processors implement the building crack detection method as described in any one of the first aspects.

[0030] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a building crack detection method as described in any one of the first aspects.

[0031] The present invention first uses a fully convolutional neural network classification model to identify the unit crack area in the image to be detected as the image of the region of interest, then uses an image segmentation model based on deep learning to extract the cracks in the image of the region of interest, obtains a crack feature region segmentation image, then binarizes the crack feature region segmentation image to obtain a unit crack detection map, and finally confirms the crack detection result map of the image to be detected according to the unit crack detection map; the present invention solves the problems of long operation time and low crack extraction robustness of the current sliding window method, as well as the problem of poor crack detection effect caused by the serious imbalance of data between the crack area and the background area during model training in the image segmentation method, thereby achieving the effect of improving the accuracy of crack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1A A schematic diagram of a flow chart of a building crack detection method provided in Embodiment 1 of the present invention;

[0033] Figure 1B A schematic diagram of an image of a region of interest in a building crack detection method provided in Embodiment 1 of the present invention;

[0034] Figure 1C A schematic diagram of a crack detection result diagram in a building crack detection method provided in Embodiment 1 of the present invention;

[0035] Figure 2A A schematic diagram of a sub-process of a building crack detection method provided in Embodiment 1 of the present invention;

[0036] Figure 2B A schematic diagram of a positive sample image in a building crack detection method provided in Embodiment 1 of the present invention;

[0037] Figure 2C A schematic diagram of ignoring sample images in a building crack detection method provided in Embodiment 1 of the present invention;

[0038] Figure 3 A schematic diagram of the structure of a building crack detection device provided in Embodiment 2 of the present invention;

[0039] Figure 4 This is a structural schematic diagram of a building crack detection device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0041] Embodiment 1

[0042] This embodiment is applicable to the case of crack detection on the surface of a building. The method can be performed by a building crack detection device, such as Figure 1A As shown, the specific steps include:

[0043] Step 110: Acquire an image of the building surface to be inspected.

[0044] The image of the building surface that needs to be inspected is collected and the image to be inspected is generated. The image collection methods include: manual collection, drone inspection collection, inspection robot collection and other methods; the corresponding range of the multiple images collected completely includes the corresponding area of ​​the building surface, and the size of each building surface image is 1024×1024 pixels.

[0045] Step 120: Use a fully convolutional neural network classification model to identify the unit crack area in the image to be detected as a region of interest image.

[0046] Before detecting cracks on the surface of a building, it is necessary to train a fully convolutional neural network classification model. According to preset conditions, the collected crack images on the surface of the building are scanned through a sliding window for dense scanning and segmentation to obtain training samples and test samples. The training samples are screened and divided into positive samples, negative samples, and ignored samples according to preset conditions. Among them, positive samples and negative samples are used to train the classification model, and ignored samples are not used. In addition, online data enhancement is used in the process of model training. The data enhancement methods used include but are not limited to the following: random rotation and flipping, histogram equalization, Gaussian noise, Gamma transformation, etc. When the verification result of the test sample meets the end condition, the training ends and the model training parameters are saved.

[0047] Since the fully convolutional neural network can input images of any size, compared with performing convolutional neural network calculations on multiple sliding window images in sequence (the method of combining convolutional layers with fully connected layers), it reduces a lot of repetitive calculations of window overlapping parts, so the fully convolutional neural network can achieve efficient sliding window calculations.

[0048] like Figure 1B As shown, the entire image to be tested 10 is input into the trained model, and a sliding window 12 of a preset size can be used to perform intensive scanning of the image and determine whether each window contains cracks according to preset conditions. At the same time, most of the non-crack areas are excluded. Each window area is used as a judgment unit image area, and the window area judged to contain cracks is a unit crack area, and all unit crack areas are used as areas of interest 11.

[0049] Step 130: Use a deep learning-based image segmentation model to extract cracks in the image of the region of interest to obtain a crack feature region segmentation image.

[0050] After the detection of the region of interest is completed, the cracks inside the region of interest need to be finely extracted. The image segmentation network model can accurately restore the morphology and detail features of the cracks during the feature extraction process. The image segmentation network model is used to train and extract cracks from the region of interest image. Since the image segmentation model only extracts the region of interest containing obvious cracks, the training samples of the image segmentation model are the positive sample images in the training samples of the above-mentioned full convolution classification model. Optional image segmentation algorithms include FCN, U-Net, deepLab, etc. Exemplarily, selecting a lightweight U-Net neural model to extract the crack area can meet the needs of crack segmentation, thereby improving computational efficiency.

[0051] The U-Net network is a fully convolutional network model that uses the idea of ​​encoding and decoding. The encoder and decoder are completely symmetrical. The encoder performs downsampling and the decoder performs upsampling. The final output segmentation result image is the same size as the input image. The Skip Connection used superimposes and fuses the same scale feature maps of the shallow encoder and the deep decoder of the network. This fusion of shallow edge features and deep abstract features makes the image segmentation effect more refined and accurate. The U-Net network structure used in this embodiment only uses 3 downsamplings, and the downsampling uses a convolution operation with a step size of 2 instead of a pooling operation, because too much downsampling, especially the pooling operation for downsampling, will lose too much image information, and the narrow crack features are more sensitive to information loss. Excessive information loss will affect the precision of crack detection. At the same time, the U-Net network structure is compressed in depth and width compared with the original U-Net structure.

[0052] After crack extraction is performed on the window image in the region of interest image through the image segmentation model, it is processed by the Sigmoid activation function and outputs a value in the range of [0,1], which is then multiplied by 255 to convert it into a unit crack feature region segmentation image with the same size as the window image in the region of interest image. The unit crack feature region segmentation image is a grayscale image, and the grayscale value of the crack region pixels in the segmented image is higher, while the grayscale value of the background region pixels is lower. The unit crack feature region segmentation images corresponding to the same region of interest image are combined into the corresponding crack feature region segmentation image.

[0053] Since the images to be detected are generally large-size high-definition images, but the crack width is generally very narrow, with an average width of about 4 to 12 pixels, the crack area accounts for a very small proportion of the entire image, resulting in a serious imbalance between the crack area data and the background area data. If the images to be detected are directly used for image segmentation training, the model will not learn the crack area sufficiently, thus affecting the accuracy of subsequent crack extraction. In order to illustrate the effectiveness of the region of interest, in the image to be detected, the crack area is abstracted as a rectangular area with a width of w and a length of the side length L of the image to be detected. Since the cracks in the region of interest are compared with the cracks in the original image to be detected, the width of the cracks is also w, and the length of the cracks is reduced to L / n, where n = the side length of the image to be detected / the side length of the region of interest. At this time, the proportion of the crack area in the image to be detected is The proportion of the crack area in the image of the region of interest is It can be seen that the proportion of cracks in the region of interest has increased by n times compared to the image to be detected. When the size of the image to be detected is 1024×1024 and the size of the region of interest is 32×32, the proportion of cracks in the region of interest is about 32 times that of the image to be detected. It can be seen that the proportion of crack area has greatly increased. Since the proportion of crack features inside the region of interest identified by the above classification model is relatively high, the crack features can be effectively learned when training the image segmentation model. At the same time, due to the improvement of the significance of crack features in the image of the region of interest identified by the classification model, the image segmentation model is also improved to accurately detect cracks; from the perspective of data, the problem of imbalance between positive and negative samples is solved, and the recognition accuracy of cracks by the image segmentation model is improved.

[0054] Step 140: binarize the image segmentation according to the crack feature region to obtain a unit crack detection map.

[0055] Each grayscale image obtained above is binarized to obtain the final unit crack detection result image, in which the pixel value of the crack area in the segmentation result is 255, and the pixel value of the background area is 0.

[0056] Step 150: confirm the crack detection result map of the image to be detected according to the unit crack detection map.

[0057] Since the image of the region of interest corresponding to the image to be detected includes multiple unit crack region images, there are multiple corresponding unit crack detection images; the crack detection result image of the image to be detected is confirmed according to the multiple unit crack detection images, such as Figure 1C As shown, specifically including:

[0058] Paste at least one of the unit crack detection images onto a pure black image of the same size as the image to be detected according to the corresponding position information to obtain a crack detection result image 20 of the image to be detected. The crack detection result image 20 includes a white target crack area 21 and a pure black background area 22.

[0059] The position information is the position of the unit crack detection image in the corresponding region of interest image.

[0060] The technical solution of the embodiment of the present invention first uses a fully convolutional neural network classification model to identify the unit crack area in the image to be detected as the image of the region of interest, then uses an image segmentation model based on deep learning to extract the cracks in the image of the region of interest to obtain a crack feature area segmentation image, and then obtains a unit crack detection image after binarization according to the crack feature area segmentation image, and finally confirms the crack detection result image of the image to be detected according to the unit crack detection image; solves the problems of long operation time and low robustness of crack extraction in the current sliding window method, and the problem of poor crack detection effect caused by the serious imbalance of data between the crack area and the background area during model training in the image segmentation method, and optimizes the classification model and the image segmentation model, thereby improving the accuracy of crack detection.

[0061] Based on the above technical solution, Figure 2A As shown, step 120 may preferably include the following specific steps:

[0062] Step 121 , a preset detection frame is used to scan and distinguish the image to be detected with a preset step length to obtain a probability feature map composed of probability values ​​of all detection frames in the image to be detected containing cracks.

[0063] Optionally, the preset detection frame is a square area of ​​a preset size, and the preset step size is 1 / 4 of the side length of the preset detection frame.

[0064] According to the structure of the classification model, a single unit image is output as a 1×1 unit feature value after the full convolutional neural network classification model, and the unit feature value indicates the probability value of the input image containing cracks. The specific value of the preset step size during the detection frame scanning is the product of the step sizes of all convolutional layers or pooling layers with a step size greater than 1 in the full convolutional neural network model. Exemplarily, as shown in Table 1, a grayscale image of size 32×32×1 finally outputs a 1×1×1 unit feature after being operated by the classification model, and the entire network undergoes 3 downsamplings with a step size of 2. Table 1 is a full convolutional network model structure table, where Conv2D represents convolution operation, BN represents batch normalization, LeakyReLU is an activation function, Maxpooling2D represents pooling operation, Sigmoid is an output activation function, padding represents the feature map boundary filling size during convolution, stride represents the convolution kernel step size, and the convolution kernel step size is not specified in Table 1 and defaults to 1. When a 1024×1024 image to be detected is input into the classification network model for scanning discrimination, it is equivalent to a 32×32 window scanning the image in an overlapping manner with a step size of 8 (2×2×2) pixels, and outputting a 125×125 probability feature map. The probability feature map indicates the probability of containing cracks in 125×125 32×32 overlapping detection frames.

[0065] Table 1 Fully convolutional network model structure

[0066] operate Convolution Kernel Input size Conv2D, BN, LeakyReLU 32,3×3×1,padding=1 32×32×1 Maxpooling2D 3×3,stride=2,padding=1 32×32×32 Conv2D, BN, LeakyReLU 64,3×3×32,padding=1 16×16×32 Maxpooling2D 3×3,stride=2,padding=1 16×16×64 Conv2D, BN, LeakyReLU 64,3×3×64, stride=2, padding=1 8×8×64 Conv2D, BN, LeakyReLU 64,3×3×64 4×4×64 Conv2D, BN, LeakyReLU 128,2×2×64 2×2×64 Conv2D 1,1×1×128 1×1×128 Sigmoid 1,1×1×1 1×1×1

[0067] The collected building surface crack images are scanned and segmented using a 32×32 window with a step size of 8 pixels to obtain 32×32 training samples and test samples.

[0068] Step 122: Select an image corresponding to a detection box with a probability value greater than the first probability value in the probability feature map as an image of a region of interest.

[0069] Since a dense window scanning method is adopted, it is not easy to miss the real crack area while retaining only the window area containing significant crack features. On this basis, the image corresponding to the detection box with a probability value greater than the first probability value in the probability feature map is selected as the image of the region of interest. The first probability value P in this embodiment can be judged using a relatively high threshold, which is not easy to cause misjudgment of background interference noise, nor is it easy to cause omission of crack areas. The value range of probability P is (0.9~0.999). For example, probability P is 0.99.

[0070] On the basis of the above technical solution, optionally, the ratio of positive and negative sample images in the training sample images of the fully convolutional neural network classification model is 1:3. This setting is because in the actual detection process of the region of interest, the number of non-crack areas is greater than the number of crack areas, and therefore more attention needs to be paid to the study of non-crack areas; wherein, the positive sample image is an image of a region with a crack area ratio greater than or equal to a first threshold, the negative sample image is an image of a region with a crack area ratio less than or equal to a second threshold, and the image of a region with a crack area ratio less than the first threshold and greater than the second threshold is an ignored sample.

[0071] The specific method for selecting training data samples for the fully convolutional neural network classification model is as follows: 1. Annotate the crack region segmentation map corresponding to the original image. The segmentation map size is the same as the original image. The pixel value of the crack region in the segmentation map is 255, and the pixel value of the background region is 0; 2. Use a 32×32 square window with a step size of 8 pixels to synchronously segment the original image and the crack region segmentation map corresponding to the original image. The original image and the unit image and unit segmentation image after segmentation of the segmented image correspond one to one; 3. Count the proportion of the crack region area in all unit segmentation images in the unit segmentation images, such as Figure 2B As shown, when the crack area 31 accounts for more than 1 / 8, the unit image corresponding to the unit segmentation image 30 is a positive sample image trained by the full convolutional neural network classification model; when the crack area accounts for less than 1 / 32, the unit image corresponding to the unit segmentation image is a negative sample image trained by the full convolutional neural network classification model; Figure 2C As shown, when the proportion of the crack area 41 is less than 1 / 8 and greater than 1 / 32, the unit image corresponding to the unit segmentation image 40 is an ignored sample image, that is, it is not used as a training sample for the full convolutional neural network classification model. The reason for the above-mentioned screening samples is that some unit images containing partial crack areas often have a high similarity with some background spots. If this part of the data is used as a positive sample of the training data, it is easy to misdetect the background interference area during detection, so this part of the data is not used as a positive sample.

[0072] By properly screening training samples, the accuracy of subsequent region of interest detection can be improved.

[0073] Further, optionally, regional images in which the crack area ratio is greater than or equal to a first threshold are used as training samples for the image segmentation model.

[0074] Based on the above technical solution, it can be understood by the same token that since the image segmentation model only extracts cracks in the area of ​​interest with obvious crack features, the positive samples trained by the full convolutional neural network classification model are also the training images of the image segmentation model, and the unit segmentation images corresponding to the above positive samples are the label images trained by the image segmentation model.

[0075] By reasonably screening training samples, the imbalance problem of positive and negative samples in the image segmentation model is solved, and the learning effect of the segmentation model on crack features is improved. At the same time, when extracting cracks, the improvement of the significance of crack features also improves the detection accuracy of the segmentation model on cracks.

[0076] Embodiment 2

[0077] Figure 3 A structural diagram of a building crack detection device provided in Embodiment 2 of the present invention is shown in FIG. Figure 3 As shown, a building crack detection device comprises:

[0078] The image acquisition module 310 is used to acquire the image to be detected of the building surface to be detected.

[0079] The image of the building surface that needs to be inspected is collected and the image to be inspected is generated. The image collection methods include: manual collection, drone inspection collection, inspection robot collection and other methods; the corresponding range of the multiple images collected completely includes the corresponding area of ​​the building surface, and the size of each building surface image is 1024×1024 pixels.

[0080] The first image region extraction module 320 is used to identify the unit crack region in the image to be detected as the region of interest image by using a full convolutional neural network classification model.

[0081] Before detecting cracks on the building surface, it is necessary to train the full convolutional neural network classification model. According to the preset conditions, the collected building surface crack images are scanned by sliding windows for dense scanning and segmentation to obtain training samples and test samples. The training samples are screened and divided into positive samples, negative samples and ignored samples according to the preset conditions. Among them, positive samples and negative samples are used to train the classification model, and ignored samples are not used. When the verification results of the test samples meet the end conditions, the training ends and the detection model parameters are saved.

[0082] The entire image to be tested is input into the above-mentioned trained model, which can realize intensive scanning of the image by sliding window and judge whether each window contains cracks according to the preset conditions. At the same time, most of the non-crack areas are excluded. Each window area is used as a judgment unit image area, and the window area judged to contain cracks is the unit crack area, and all unit crack areas are regarded as regions of interest.

[0083] The second image region extraction module 330 is used to extract cracks in the image of the region of interest by using a deep learning image segmentation model to obtain a crack feature region segmentation image.

[0084] After the detection of the region of interest is completed, the cracks inside the region of interest need to be finely extracted. The image segmentation network model can accurately restore the morphology and detail features of the cracks during the feature extraction process. The image segmentation network model is used to train and extract cracks from the region of interest image. The optional image segmentation algorithms include FCN, U-Net, deepLab, etc. For example, selecting a lightweight U-Net neural model to extract the crack area can meet the needs of crack segmentation, thereby improving computational efficiency; wherein, the U-Net neural model training samples are positive samples in the classification model training.

[0085] The window image in the region of interest image is subjected to crack extraction by the image segmentation model, and then processed by the Sigmoid activation function to output a value in the range of [0,1], which is then multiplied by 255 to be converted into a unit crack feature region segmentation image with the same size as the window image in the region of interest image. The unit crack feature region segmentation image is a grayscale image, and the unit crack feature region segmentation images corresponding to the same region of interest image are combined into the corresponding crack feature region segmentation image.

[0086] The crack detection module 340 is used to obtain a unit crack detection map after segmenting the image according to the crack feature region and performing binarization.

[0087] Each grayscale image obtained above is binarized to obtain the final unit crack detection result image, in which the pixel value of the crack area in the segmentation result is 255, and the pixel value of the background area is 0.

[0088] The crack result confirmation module 350 is used to confirm the crack detection result map of the image to be detected according to the unit crack detection map.

[0089] Since the region of interest image corresponding to the image to be detected includes multiple unit crack region images, there are multiple corresponding unit crack detection images; confirming the crack detection result image of the image to be detected according to the multiple unit crack detection images specifically includes:

[0090] At least one of the unit crack detection images is pasted onto a pure black image of the same size as the image to be detected according to corresponding position information to obtain a crack detection result image of the image to be detected.

[0091] The position information is the position of the unit crack detection image in the corresponding region of interest image.

[0092] The technical solution of the embodiment of the present invention first uses a fully convolutional neural network classification model to identify the unit crack area in the image to be detected as the image of the region of interest, then uses an image segmentation model based on deep learning to extract the cracks in the image of the region of interest, obtains a crack feature area segmentation image, and then obtains a unit crack detection image after binarization according to the crack feature area segmentation image, and finally confirms the crack detection result image of the image to be detected according to the unit crack detection image; solves the problems of the current sliding window method taking a long time to calculate and being difficult to remove interference noise in the background area, and the image segmentation method having a serious imbalance in data between the crack area and the background area during model training, resulting in poor crack detection effect, and achieves the optimization of the classification model and the image segmentation model, thereby improving the effect of crack detection accuracy.

[0093] The building crack detection device provided in the embodiment of the present invention can execute the building crack detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0094] Embodiment 3

[0095] Figure 4 A structural diagram of a building crack detection device provided in Embodiment 3 of the present invention is shown in FIG. Figure 4 As shown, the device includes a processor 40, a memory 41, an input device 42 and an output device 43; the number of processors 40 in the device can be one or more. Figure 4 A processor 40 is taken as an example; the processor 40, memory 41, input device 42 and output device 43 in the device can be connected by a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0096] The memory 41, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the building crack detection method in the embodiment of the present invention (for example, the image acquisition module 310, the first image area extraction module 320, the second image area extraction module 330, the crack detection module 340 and the crack result confirmation module 350 in the building crack detection device). The processor 40 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 41, that is, realizing the above-mentioned building crack detection method.

[0097] The memory 41 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 41 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 41 may further include a memory remotely arranged relative to the processor 40, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] The input device 42 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 43 may include a display device such as a display screen.

[0099] Embodiment 4

[0100] Embodiment 4 of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute a building crack detection method when executed by a computer processor, the method comprising:

[0101] Acquire an image to be inspected of a building surface to be inspected;

[0102] Using a fully convolutional neural network classification model to identify the unit crack region in the image to be detected as an image of a region of interest;

[0103] Using a deep learning-based image segmentation model to extract cracks in the image of the region of interest, and obtaining a crack feature region segmentation image;

[0104] After the image is segmented and binarized according to the crack feature region, a unit crack detection map is obtained;

[0105] The crack detection result map of the image to be detected is confirmed according to the unit crack detection map.

[0106] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided by an embodiment of the present invention are not limited to the operations of the method described above, and can also execute related operations in the building crack detection method provided by any embodiment of the present invention.

[0107] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0108] It is worth noting that in the embodiment of the above-mentioned search device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0109] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A building crack detection method, characterized in that: include: Acquire an image to be inspected of a building surface to be inspected; A fully convolutional neural network classification model is used to identify the unit crack area in the image to be detected as the region of interest image, including: presetting the detection frame to scan the image to be detected with a preset step size to obtain a probability feature map composed of probability values ​​of all detection frames in the image to be detected containing cracks; selecting the image corresponding to the detection frame greater than the first probability value in the probability feature map as the region of interest image; the specific value of the preset step size is the product of the step sizes of all convolutional layers or pooling layers with step sizes greater than 1 in the fully convolutional neural network model; Using a deep learning-based image segmentation model to extract cracks in the image of the region of interest, and obtaining a crack feature region segmentation image; After the image is segmented and binarized according to the crack feature region, a unit crack detection map is obtained; The crack detection result map of the image to be detected is confirmed according to the unit crack detection map.

2. The building crack detection method according to claim 1, characterized in that: The preset detection frame is a square area of ​​a preset size, and the preset step size is 1 / 4 of the side length of the preset detection frame.

3. The building crack detection method according to claim 1, characterized in that: The ratio of positive and negative sample images in the training sample images of the fully convolutional neural network classification model is 1:3; wherein, the positive sample image is a regional image with a crack area ratio greater than or equal to a first threshold, the negative sample image is a regional image with a crack area ratio less than or equal to a second threshold, and the regional image with a crack area ratio less than the first threshold and greater than the second threshold is an ignored sample.

4. The building crack detection method according to claim 1, characterized in that: Regional images with crack area ratios greater than or equal to a first threshold are used as training samples for the image segmentation model.

5. The building crack detection method according to claim 1, characterized in that: Confirming the crack detection result map of the image to be detected according to the unit crack detection map includes: Paste at least one of the unit crack detection images onto a pure black image of the same size as the image to be detected according to the corresponding position information to obtain a crack detection result image of the image to be detected; the position information is the position of the unit crack detection image in the corresponding image of the region of interest.

6. A building crack detection device, characterized in that: include: An image acquisition module, used for acquiring an image to be detected of a building surface to be detected; The first image region extraction module is used to use a full convolutional neural network classification model to identify the unit crack region in the image to be detected as the region of interest image, including: presetting the detection frame to scan the image to be detected with a preset step size to obtain a probability feature map composed of probability values ​​of all detection frames in the image to be detected containing cracks; selecting the image corresponding to the detection frame greater than the first probability value in the probability feature map as the region of interest image; the specific value of the preset step size is the product of the step sizes of all convolutional layers or pooling layers with step sizes greater than 1 in the full convolutional neural network model; A second image region extraction module is used to extract cracks in the image of the region of interest using a deep learning image segmentation model to obtain a crack feature region segmentation image; A crack detection module, used to obtain a unit crack detection map after segmenting the image according to the crack feature area and performing binarization; The crack result confirmation module is used to confirm the crack detection result map of the image to be detected according to the unit crack detection map.

7. A building crack detection device, characterized in that: The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the building crack detection method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the building crack detection method as described in any one of claims 1 to 5 is implemented.

Images