A building surface crack recognition method and system based on a convolutional neural network

Abstract

The application discloses a kind of based on convolutional neural network's building surface crack identification method and system, first, the convolutional neural network model capable of identifying building surface crack is constructed;Then, the image of the building surface to be measured is collected;Finally, the convolutional neural network model constructed is used to the crack identification and display of the building surface image to be measured collected.This application introduces image recognition into building surface crack identification, can realize the unmanned identification of crack, make up the deficiency that present manual detection means is time-consuming and laborious, subjective.

Description

Technical Field

[0001] This invention relates to the field of civil engineering surveying equipment, and more particularly to technology for identifying cracks on building surfaces. Background Technology

[0002] As a crucial aspect of construction engineering, the quality of plastering work directly impacts both the user experience and the overall aesthetic appeal of the building. However, due to the numerous bonding surfaces in plastering, defects are prone to occur, with cracking of the plaster surface on walls being a common problem and a challenging aspect of quality control.

[0003] To ensure the quality of wall construction, conducting construction quality inspections to detect cracks and repair them promptly is a common quality control method. However, current manual inspection methods are time-consuming, labor-intensive, and highly subjective. Furthermore, surface cracks on walls are sometimes difficult to detect and are easily overlooked by manual inspections, especially at the corners and edges of the walls.

[0004] Therefore, how to solve the problems of low efficiency and low accuracy of existing methods for manually detecting cracks on building surfaces is an urgent issue that needs to be addressed in this field. Summary of the Invention

[0005] To address the problems of high human involvement, low measurement efficiency, and high subjectivity in existing building surface crack identification schemes, this invention provides a building surface crack identification method based on convolutional neural networks. This method enables unmanned inspection of building surfaces and rapid crack identification, overcoming the shortcomings of existing manual identification methods. Furthermore, this invention also provides a building surface crack identification system that implements this method.

[0006] To achieve the above objectives, the present invention provides a method for identifying surface cracks in buildings based on convolutional neural networks, comprising the following steps:

[0007] Step 1: Construct a convolutional neural network model capable of identifying cracks on building surfaces;

[0008] Step 2: Acquire images of the surface of the building to be tested;

[0009] Step 3: Use the convolutional neural network model constructed in step (1) to identify and display cracks in the collected images of the building surface to be tested.

[0010] In some embodiments of the present invention, step (1) in constructing a convolutional neural network model includes the following sub-steps:

[0011] Step 1-1: Collect n images of surface cracks in the actual building;

[0012] Steps 1-2: Preprocessing of building surface crack images to form several sub-images;

[0013] Steps 1-3: Manually judge the degree of crack in each sub-image, classifying it as no crack, weak crack, or strong crack, and assign corresponding label values ​​to each;

[0014] Steps 1-4: Perform data augmentation on each segmented sub-image;

[0015] Steps 1-5: Based on the atomic image, the images with label values ​​corresponding to weak cracks and strong cracks from the data augmentation images are added to the training set of the neural network, ultimately forming an image training set composed of m crack grayscale images;

[0016] Steps 1-6: Train a convolutional neural network model using the formed image training set. The convolutional neural network model will output three label values ​​a, b, and c representing the degree of cracking.

[0017] In some embodiments of the present invention, when performing data augmentation on the sub-image in steps 1-4, a random number λ is first generated to satisfy λ ~ Be(0.5, 0.5). Then, the pixel values ​​x1, x2 and their corresponding label values ​​y1, y2 of any two images are merged into the pixel values ​​and label values ​​x1, x2 of the new image according to equations (1) and (2). new y new :

[0018] x new =λx1+(1-λ)x2 Equation (1);

[0019] y new =λy1+(1-λ)y2 Equation (2).

[0020] In some embodiments of the present invention, step (3) includes the following sub-steps when performing crack identification and display:

[0021] Step 3-1: Detect large-scale cracks in the surface image of the building under test based on the constructed convolutional neural network model;

[0022] Step 3-2: Based on the constructed convolutional neural network model, perform small-scale crack classification on the surface image of the building under test;

[0023] Step 3-3: Merge the identification images obtained from Step 3-1 and Step 3-2 to form a building surface image with crack level markings.

[0024] In some embodiments of the present invention, step 3-1, when performing large-scale crack detection, includes the following sub-steps:

[0025] Step 3-1-1: Divide the image to be identified into several 128×128 small images at 64-pixel intervals;

[0026] Step 3-1-2: Scale the small image obtained in Step 3-1-1 to a size of 64×64, and then import it row by row into the convolutional neural network model trained in Step 1; the convolutional neural network model processes the input small image to obtain the corresponding label values ​​a, b, c;

[0027] Step 3-1-3: Identify and determine cracks based on label values ​​a, b, and c;

[0028] Step 3-1-4: Match the crack identification results to the original image and annotate the original image according to the identification results.

[0029] In some embodiments of the present invention, step 3-2, when classifying small-scale cracks, includes the following sub-steps:

[0030] Step 3-2-1: Divide the image to be identified into several 64×64 small images at 64-pixel intervals;

[0031] Step 3-2-2: Import the small images obtained in Step 3-2-1 into the convolutional neural network model trained in Step 1, row by row. The convolutional neural network model processes the input small images to obtain the corresponding label values ​​a, b, c.

[0032] Step 3-2-3: Identify and determine the crack grade based on the label values ​​a, b, and c;

[0033] Step 3-2-4: Map the identified crack level results to the original image and annotate the original image according to the identified crack level results.

[0034] To achieve the above objectives, the present invention provides a building surface crack recognition system based on a convolutional neural network, comprising:

[0035] The neural network training module trains a neural network based on images of cracks on building surfaces, constructing a convolutional neural network model capable of recognizing cracks on building surfaces.

[0036] Building surface image acquisition module, the building surface image acquisition module is used to acquire images of the building surface to be measured;

[0037] The crack recognition module uses a convolutional neural network model constructed by the neural network training module to perform crack recognition processing on the building surface images acquired by the building surface image acquisition module.

[0038] In some embodiments of the present invention, the convolutional neural network model trained by the neural network training module can generate multiple label values ​​representing the degree of cracking for the image to be processed.

[0039] In some embodiments of the present invention, the crack recognition module includes an image segmentation module, an input module, a judgment module, and a merging module.

[0040] The image segmentation module is used to segment the image of the building surface to be tested into several sub-images for large-scale crack detection, or into several sub-images for small-scale crack classification.

[0041] The input module interacts with the image segmentation module to import the sub-images obtained by the image segmentation module into the trained convolutional neural network model in rows.

[0042] The determination module performs crack detection and identification or crack classification identification on the surface image of the building under test based on the output result of the convolutional neural network model, and marks the original image of the surface of the building under test based on the identification result.

[0043] The merging module interacts with the determination module to merge the images of the building surface to be tested that have been marked with cracks and crack level markings, forming a building surface image with crack level markings.

[0044] The advantages of the solution provided by this invention compared to the prior art are as follows:

[0045] 1) The present invention introduces image recognition into the identification of cracks on building surfaces, which can realize unmanned crack identification and make up for the shortcomings of existing manual detection methods, which are time-consuming, labor-intensive and highly subjective.

[0046] 2) The innovative dual-scale crack recognition technology (i.e., large-scale crack detection and small-scale crack classification) in the present invention can avoid the special situation where crack information is difficult to extract because the crack position is exactly on the image segmentation line, thus ensuring higher accuracy of crack recognition.

[0047] 3) The building surface crack identification solution provided by this invention can be integrated with equipment such as drones and robots to promote the unmanned and automated detection of engineering quality. Attached Figure Description

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0049] Figure 1 This is an overall flowchart of the building surface crack recognition method based on convolutional neural networks in an example of the present invention;

[0050] Figure 2This is a flowchart illustrating the construction process of the convolutional neural network model in this invention example;

[0051] Figure 3 This is a flowchart illustrating crack identification and display in an example of the present invention.

[0052] Figure 4 This is a schematic diagram of the crack identification and display process in an example of the present invention;

[0053] Figure 5 This is a flowchart illustrating large-scale crack detection in an example of the present invention.

[0054] Figure 6 This is a flowchart illustrating small-scale crack classification in an example of the present invention;

[0055] Figure 7 This is a schematic diagram illustrating the structure of the building surface crack identification system in this invention.

[0056] Figure 8 This is a diagram illustrating the effect of identifying wall cracks at the inside corner in an example of the present invention. Detailed Implementation

[0057] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0058] To address the issues of high human involvement, low measurement efficiency, and significant subjectivity in current building surface crack identification methods, the inventors, through extensive research, have innovatively applied digital image recognition to building structural defect detection, presenting a building surface crack identification technology based on convolutional neural networks. This technology can significantly improve detection accuracy and reduce labor costs.

[0059] Accordingly, this invention provides a method for identifying surface cracks in buildings based on convolutional neural networks, combined with... Figure 1 As shown, the method for identifying surface cracks in this building mainly consists of the following steps:

[0060] Step 1: Construct a convolutional neural network model capable of identifying cracks on building surfaces;

[0061] Step 2: Acquire images of the surface of the building to be tested;

[0062] Step 3: Use the convolutional neural network model constructed in step (1) to identify and display cracks in the collected images of the building surface to be tested.

[0063] In some embodiments of the present invention, see Figure 2 For step (1), when constructing the convolutional neural network model, the specific sub-steps are as follows:

[0064] Step 1-1: Collect n images of surface cracks in the actual building;

[0065] The n images of building surface cracks collected here will serve as samples for subsequent training of the convolutional neural network. No specific requirements are set for the images of building surface cracks; they can be determined based on actual needs.

[0066] Step 1-2: Preprocess the n images of building surface cracks collected in Step 1-1 to form several sub-images.

[0067] Specifically, in this step, the exposure and contrast of the acquired crack image are adjusted, and the original image is randomly cropped into sub-images of size 64×64.

[0068] In this step, it is preferable to crop the original image into 64×64 sub-images, which facilitates subsequent sampling and training of the neural network and improves processing efficiency.

[0069] Steps 1-3: Manually judge the degree of cracks in each sub-image obtained in Steps 1-2 and assign corresponding label values ​​to each.

[0070] As an example, in this step, the degree of crack in each sub-image is judged by human judgment, and is respectively determined as no crack, weak crack, and strong crack, and the corresponding label value is given as 0.0-no crack, 0.5-weak crack, and 1.0-strong crack.

[0071] In this step, three crack severity standards are selected: 0.0 - no crack, 0.5 - weak crack, and 1.0 - strong crack. This can be well coordinated with the subsequent convolutional neural network model processing to improve the recognition accuracy.

[0072] Meanwhile, based on the selection of three crack severity standards: 0.0 - no crack, 0.5 - weak crack, and 1.0 - strong crack, and according to these standards, each sub-image used as a sample is labeled, which can improve the subsequent neural network training effect and thus improve the recognition and processing accuracy of the convolutional neural network model.

[0073] It should be noted that the determination of the degree of crack in the image is not limited to the three levels of no crack, weak crack, and strong crack. Other determination levels can be used as needed.

[0074] Furthermore, the label values ​​assigned to different crack degrees are not limited to the three label values ​​of 0.0 (no crack), 0.5 (weak crack), and 1.0 (strong crack). Other label values ​​can be used as needed.

[0075] Steps 1-4: Use the mixup algorithm to perform data augmentation on the segmented sub-images that have been processed in Steps 1-3.

[0076] In this step, when augmenting the sub-images, new sub-image data is generated by fine-tuning the original sub-image data, thus forming augmented sub-images, which are then added to the training set as new sub-image samples. This further improves and supplements the subsequent training set, enhancing the training effect of the neural network.

[0077] As an example, this step preferably achieves data enhancement for each sub-image through the following process:

[0078] First, generate a random number λ such that λ ~ Be(0.5, 0.5). Then, according to equations (1) and (2), merge the pixel values ​​x1, x2 and their corresponding label values ​​y1, y2 of any two images into the pixel values ​​and label values ​​x1, x2 of the new image. new y new :

[0079] x new =λx1+(1-λ)x2 Equation (1);

[0080] y new =λy1+(1-λ)y2 Equation (2).

[0081] Steps 1-5: Based on the atomic image, combined with the image formed by data augmentation in Steps 1-4, add images with only label values ​​of 0.45-0.55 (weak defect) and 0.9-1.0 (strong defect) to the training set of the neural network, and finally form a training set consisting of m 64×64 crack grayscale images.

[0082] Specifically, in this step, the atomic images generated in steps 1-3 and the images formed by data augmentation in steps 1-4 are combined to simultaneously extract atomic images with label values ​​of 0.45-0.55 (weak defects) and their corresponding augmented sub-images, and atomic images with label values ​​of 0.9-1.0 (strong defects) and their corresponding augmented sub-images. Based on this, all the extracted images are added to the training set of the neural network, and finally a training set consisting of m 64×64 crack grayscale images is formed.

[0083] Steps 1-6: Train a convolutional neural network model using the formed image training set. The network input is a 64×64 grayscale image, which is flattened after several convolutions and finally outputs three label values ​​a, b, and c representing the degree of cracking through a fully connected neural network. The resulting convolutional neural network can then take an image as input and output three label values ​​a, b, and c representing the degree of cracking.

[0084] As an example, the correspondence between a, b, c and the degree of crack in this step is shown in Table 1.

[0085] Table 1. Correspondence between label values ​​a, b, c and crack severity

[0086] Label a b c Strong defects 0 0 1 Weakness 0 1 0 No defects 1 0 0

[0087] In this invention, a convolutional neural network model is trained using a training set of sub-images that have undergone crack degree labeling (assigned label values ​​corresponding to crack degree) in steps 1-3. Based on the relationships in Table 1, each training sample sub-image with crack degree labeling is first input into the neural network. The initial neural network processes and outputs three label values ​​a', b', and c' representing the crack degree. By determining the differences between the label values ​​a', b', and c' and the original label values ​​a, b, and c of the input training sample sub-image, the parameters of the neural network are adjusted to form the next-generation neural network. After different training sample sub-images undergo the above operations, the goal of training the neural network model can be achieved, ensuring that the neural network training of the input image can output accurate label values.

[0088] Thus, a trained convolutional neural network model is obtained. Subsequently, the target image can be processed by the convolutional neural network model, the corresponding label value can be output by the convolutional neural network model, and then the target can be determined as a crack based on the label value.

[0089] In some embodiments of the present invention, see Figure 3 For step (3), when identifying and displaying cracks, the specific sub-steps include the following:

[0090] Step 3-1: Based on the constructed convolutional neural network model, perform large-scale crack detection on the building surface image collected in step (2) to form a building surface image with crack markings;

[0091] Step 3-2: Based on the constructed convolutional neural network model, perform small-scale crack classification on the building surface image collected in step (2) to form a building surface image with crack level markings;

[0092] Step 3-3: Merge the recognition images obtained from Step 3-1 and Step 3-2 (i.e., the building surface image with crack markings and the building surface image with crack level markings) to form a building surface image with crack level markings.

[0093] In some embodiments of the present invention, when performing large-scale crack detection in step 3-1, the specific sub-steps include the following, see below. Figure 5 :

[0094] Step 3-1-1: Divide the image to be identified into several 128×128 small images at 64-pixel intervals.

[0095] In this step, the preferred image is divided into several 128×128 small images at 64-pixel intervals. This facilitates sampling and training of the neural network and improves processing efficiency.

[0096] Meanwhile, the image is divided into several 128×128 smaller images at 64-pixel intervals to match the smaller size in step 3-1-2, so that crack identification can be performed from a larger dimension in subsequent identification steps, thereby improving the accuracy of the identification results.

[0097] Step 3-1-2: Scale the small images obtained in Step 3-1-1 to 64×64 pixels, and then import them into the convolutional neural network model trained in Step 1 row by row. The convolutional neural network model processes the input small images in turn and outputs the corresponding label values ​​a, b, c.

[0098] Step 3-1-3: Based on the label values ​​a, b, c output by the convolutional neural network model, determine whether there are cracks in each small image: If the label values ​​a, b, c meet the following conditions, it is determined that there are surface cracks in the corresponding area of ​​the building surface image to be tested: c>0.95 or a<0.05 or b+c>α; otherwise, it is determined that there are no surface cracks.

[0099] This step uses judgment conditions such as "c>0.95 or a<0.05 or b+c>α", which can achieve a certain degree of tolerance for variation and ensure the accuracy of the final judgment result.

[0100] It should be noted that the threshold α can be adjusted in this step to adapt to the recognition accuracy requirements of different detection tasks.

[0101] Step 3-1-4: Match the crack identification results from Step 3-1-3 to the original image and label them as having cracks (light red) or no cracks (colorless).

[0102] Specifically, in this step, the region corresponding to the small image that has been completed is determined on the surface image of the building to be tested, and then the corresponding region on the surface image of the building to be tested is marked according to the crack identification result of the small image in step 3-1-3.

[0103] As an example, color is used for labeling. If the crack identification result of the current small image is a crack, the corresponding area on the surface image of the building to be tested will be labeled as light red; if the crack identification result of the current small image is no crack, the corresponding area on the surface image of the building to be tested will be labeled as colorless.

[0104] It should be noted that other colors can be used for the markings as needed, and the examples provided are not limited to those shown here.

[0105] In this way, all the small images segmented in step 3-1-1 are processed in sequence, and then all areas on the building surface image to be tested are labeled accordingly, thus forming a building surface image with crack markings.

[0106] In some embodiments of the present invention, the small-scale crack classification in step 3-2 specifically includes the following sub-steps, see below. Figure 6 :

[0107] Step 3-2-1: Divide the image to be identified into several 64×64 small images at 64-pixel intervals.

[0108] In this step, the preferred image is divided into several 64×64 small images at 64-pixel intervals. This facilitates sampling and training of the subsequent neural network, improving processing efficiency.

[0109] Meanwhile, the image is divided into several 64×64 smaller images at 64-pixel intervals to complement the larger image in step 3-1-1. This allows for the identification of smaller cracks after the identification of larger cracks is completed in subsequent identification steps, thereby improving the accuracy of the identification results.

[0110] Step 3-2-2: Import the small images obtained from the segmentation in Step 3-2-1 directly into the trained convolutional neural network model row by row. The convolutional neural network model processes the input small images sequentially and outputs the corresponding label values ​​a, b, and c.

[0111] Step 3-2-3: Based on the label values ​​a, b, and c output by the convolutional neural network model, determine the crack level of each small image:

[0112] When c>0.95 or a<0.05, the crack in the small image is classified as a strong defect.

[0113] If the above requirements are not met, when b+c>α, the crack on the small image is classified as a medium defect.

[0114] If the requirements are still not met, when a>β, the crack in the small image is classified as a weak defect;

[0115] Otherwise, it is assumed that there are no cracks on the small image.

[0116] The judgment conditions used in this step can both achieve a certain degree of tolerance for variation and ensure the accuracy of the final judgment result.

[0117] It should be noted that in this step, the thresholds α and β can be adjusted to adapt to the accuracy requirements of different detection tasks.

[0118] Step 3-2-4: Map the crack level results identified in Step 3-2-3 to the original image, and annotate the corresponding areas in the original image according to the crack level results.

[0119] Specifically, in this step, the area corresponding to the small image that has been identified is determined on the surface image of the building to be tested. Then, based on the crack level identification result of the small image in step 3-2-3, the corresponding area on the surface image of the building to be tested is marked.

[0120] As an example, color is used for marking. For instance, if the crack level of the current small image is determined to be a strong defect, the corresponding area on the surface image of the building to be tested will be marked in dark red.

[0121] If the crack level of the current small image is determined to be medium defect, then the corresponding area on the surface image of the building to be tested will be marked in red;

[0122] If the crack level of the current small image is determined to be a weak defect, the corresponding area on the surface image of the building to be tested will be marked in light red.

[0123] If the crack level assessment result of the current small image is no crack, then the corresponding area on the surface image of the building to be tested will be marked as colorless.

[0124] It should be noted that other colors can be used for the markings as needed, and the examples provided are not limited to those shown here.

[0125] In this way, all the small images segmented in step 3-2-1 are processed sequentially, and then all areas on the building surface image to be tested are labeled accordingly, thus forming a building surface image with crack level markings.

[0126] In some embodiments of the present invention, when merging in step 3-3, based on the merging of the building surface image with crack markings that has completed large-dimensional identification and the building surface image with crack level markings that has completed small-dimensional identification, the resulting building surface image has crack marking regions distributed in large dimensions, and at the same time also has crack level marking regions distributed in small dimensions.

[0127] In this way, for each building surface image, the areas with cracks can be identified by the large-dimensional crack marking areas; at the same time, the small-dimensional crack level marking areas can be used to further verify the crack areas on the building surface image that are not covered by the large-dimensional crack marking areas.

[0128] For example, when processing a building surface image with crack marking regions distributed in a large dimension, if the crack happens to be located on the segmentation line of the large-dimensional segmented image, the final large-dimensional crack marking regions will not cover the crack region. In the same case, when processing the same building surface image with crack level marking regions distributed in a small dimension, due to the different segmentation dimensions, the same crack will not be located on the segmentation line of the small-dimensional segmented image, and the final small-dimensional crack level marking regions will effectively cover the crack region.

[0129] Conversely, if a crack is located on a segmentation line of a small-dimensional segmented image, it can be identified by the crack marking area that is distributed in a large dimension.

[0130] This solution implements a dual-scale crack recognition method, which can effectively avoid the special situation where crack information is difficult to extract because the crack location is exactly on the image segmentation line, thus ensuring higher accuracy in crack recognition.

[0131] The solution provided by this invention will be further illustrated below through specific examples.

[0132] In this example, a corresponding software program is constructed to form a corresponding building surface crack recognition system based on convolutional neural networks, based on the building surface crack recognition method provided by the present invention. When the software program runs, it executes the aforementioned building surface crack recognition method based on convolutional neural networks and stores it in a corresponding storage medium for the processor to retrieve and execute.

[0133] See Figure 7 The building surface crack recognition system 100 based on convolutional neural network is mainly composed of a neural network training module 110, a building surface image acquisition module 120, and a crack recognition module 130.

[0134] Among them, the neural network training module 110 trains the neural network based on images of cracks on the building surface to build a convolutional neural network model that can identify cracks on the building surface.

[0135] The building surface image acquisition module 120 is used to acquire images of the building surface to be measured.

[0136] The crack recognition module 130 interacts with the neural network training module 110 and the building surface image acquisition module 120 respectively, and uses the convolutional neural network model constructed by the neural network training module to perform crack recognition processing on the building surface image to be tested acquired by the building surface image acquisition module.

[0137] Specifically, when implementing the neural network training module 110 in this system, it is preferable to adopt the above-mentioned scheme of step 1 and its sub-steps.

[0138] The convolutional neural network model trained by this neural network training module 110 can generate multiple label values ​​representing the degree of cracking for the image to be processed.

[0139] The specific configuration of the building surface image acquisition module 120 in this system can be determined according to actual needs, and is not limited here.

[0140] The crack recognition module 130 in this system comprises an image segmentation module 131, an input module 132, a judgment module 133, and a merging module 134. The crack recognition module 130 is formed by the cooperation of these four functional modules to achieve the solution in step 3 above.

[0141] The image segmentation module 131 here is used to segment the building surface image to be tested acquired by the building surface image acquisition module 120 into several sub-images for large-scale crack detection, or into several sub-images for small-scale crack classification.

[0142] The input module 132 interacts with the image segmentation module 131 to import the sub-images obtained by the image segmentation module 131 into the convolutional neural network model trained by the neural network training module 110 in rows.

[0143] The determination module 133 performs crack detection and identification or crack classification identification on the image of the building surface to be tested based on the output result of the convolutional neural network model, and marks the original image of the building surface to be tested based on the identification result, forming a building surface image with crack marking and a building surface image with crack level marking.

[0144] The merging module 134 interacts with the judgment module 133 to merge the building surface image with crack markings formed by the judgment module 133 with the building surface image with crack level markings to form a building surface image with crack level markings.

[0145] The following example illustrates the process of identifying surface cracks in buildings using the convolutional neural network-based building surface crack recognition system developed in this instance.

[0146] Before identifying surface cracks in a building, the building surface crack recognition system first acquires n surface crack images of the actual building by the building surface image acquisition module 120. These images are then used by the neural network training module 110 to train the neural network model capable of recognizing surface cracks in the building.

[0147] Once a convolutional neural network model capable of identifying surface cracks in buildings has been trained, a surface crack recognition system can automatically identify surface cracks in buildings.

[0148] Combination Figure 4 As shown, the building surface image acquisition module 120 acquires an image of the building surface to be measured and transmits the image to the image segmentation module 131.

[0149] Image segmentation module 131 simultaneously performs the following segmentation processing on the received image of the building surface to be tested:

[0150] (1) Divide the image of the building surface to be measured into several 128×128 first small images at 64-pixel intervals;

[0151] (2) Divide the image to be identified into several 64×64 second smaller images at 64-pixel intervals.

[0152] The image segmentation module 131 transmits the segmented first small image of 128×128 and the second small image of 64×64 to the input module 132.

[0153] The input module 132 scales the received 128×128 first small image to a size of 64×64, and then imports it into the pre-trained convolutional neural network model in the system row by row.

[0154] The input module 132 directly imports the received 64×64 second small images into the pre-trained convolutional neural network model in the system, row by row.

[0155] The convolutional neural network model in the system processes the first and second small images in sequence and outputs the corresponding label values ​​a, b, and c.

[0156] The judgment module 133 performs large-scale crack detection on the surface image of the building under test based on the label values ​​a, b, and c corresponding to each first small image, combined with... Figure 5 The entire process is as follows:

[0157] For each first small image, after receiving the corresponding label values ​​a, b, c output by the convolutional neural network model, the determination module 133 performs the following determination and annotation processing on the label values ​​a, b, c:

[0158] If c>0.95 or a<0.05 or b+c>α, then it is determined that the area corresponding to the small image on the surface image of the building under test has a surface crack; and the corresponding area on the surface image of the building under test is marked in light red.

[0159] If none of these conditions are met, then the area corresponding to the small image on the surface image of the building under test is determined to have no surface cracks, and the corresponding area on the surface image of the building under test is marked as colorless, i.e., the original image.

[0160] The judgment module 133 processes all the first small images in sequence, and then completes the corresponding annotations for all areas on the building surface image to be tested, thereby forming a building surface image with crack markings.

[0161] Meanwhile, the judgment module 133 performs small-scale crack classification based on the label values ​​a, b, and c corresponding to each second small image of the building surface under test, combined with... Figure 6 The entire process is as follows:

[0162] For each second small image, after receiving the corresponding label values ​​a, b, c output by the convolutional neural network model, the determination module 133 performs the following determination and annotation processing on the label values ​​a, b, c:

[0163] First, determine whether c > 0.95 or a < 0.05. If so, the crack in the second small image is classified as a strong defect; and the corresponding area on the surface image of the building to be tested is marked in dark red.

[0164] If not, then further determine b+c>α; if true, then identify the crack level on the second small image as a medium defect; and mark the corresponding area on the surface image of the building to be tested in red.

[0165] If this condition is not met, then a > β is further determined. If this condition is met, then the crack level on the second small image is determined to be a weak defect, and the corresponding area on the surface image of the building to be tested is marked in light red.

[0166] If this is not the case, then the second small image is considered to have no cracks; the corresponding area on the image of the building surface to be tested is not marked and is considered the original image.

[0167] Thus, the determination module 133 processes all the second small images in sequence, and then completes the corresponding annotations for all areas on the building surface image to be tested, thereby forming a building surface image with crack level markings.

[0168] The determination module 133 simultaneously transmits the obtained building surface image with crack markings and the building surface image with crack level markings to the merging module 134.

[0169] The merging module 134 merges the two images and identifies and displays the final crack based on the final color depth.

[0170] This dual-scale crack recognition method enables automatic and accurate identification of cracks on building surfaces. It effectively avoids the special situation where crack information is difficult to extract because the crack location is exactly on the image segmentation line, thus ensuring higher accuracy in crack recognition.

[0171] To further verify the crack identification effect of the solution provided by the present invention, this example further conducts a practical application test on the solution of the present invention.

[0172] Practical project application results show that the crack identification technology provided by this invention can effectively identify cracks on building surfaces, and also has a good identification effect on cracks that are difficult to detect with the naked eye.

[0173] See Figure 8 The image shown is an illustration of the effect of the present invention in identifying wall cracks at the inside corner.

[0174] in Figure 8 Image 'a' shows the actual effect of the internal corner crack 1; while... Figure 8 b is a diagram showing the identification results of the present invention for the internal corner crack 1;

[0175] Figure 8 c is the actual effect of the internal corner crack 2; while Figure 8 d is a diagram showing the identification results of the internal corner crack 2 according to the present invention.

[0176] Depend on Figure 8 As can be seen from the actual application effects shown, the solution of the present invention is effective even for applications such as... Figure 8 The wall cracks at the inside corner shown can also be accurately identified.

[0177] The method, specific system unit, or part thereof of the present invention described above is a pure software architecture. It can be deployed via program code on physical media, such as hard disks, optical discs, or any electronic device (such as smartphones or computer-readable storage media). When a machine loads and executes the program code (e.g., a smartphone loads and executes it), the machine becomes an apparatus for implementing the present invention. The method and apparatus of the present invention can also be transmitted in program code form via transmission media, such as cables, optical fibers, or any transmission method. When the program code is received, loaded, and executed by a machine (e.g., a smartphone), the machine becomes an apparatus for implementing the present invention.

[0178] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying surface cracks in buildings based on convolutional neural networks, characterized in that, Includes the following steps: Step 1: Construct a convolutional neural network model capable of identifying cracks on building surfaces, including: Step 1-1: Collect n images of surface cracks in actual buildings as samples for subsequent training of the convolutional neural network; Step 1-2: Adjust the exposure and contrast of the n images of building surface cracks collected in Step 1-1, and randomly crop the original images into sub-images of size 64×64; Steps 1-3: The degree of cracks in each sub-image obtained in Step 1-2 is judged manually and classified as no crack, weak crack, and strong crack, respectively, and a corresponding label value is given. Each sub-image used as a sample is labeled based on the three crack degree standards. Steps 1-4: Perform data augmentation on each segmented sub-image that has been processed in Steps 1-3. Generate new sub-image data by fine-tuning the original sub-image data, thereby forming augmented sub-images, which are then added to the training set as new sub-image samples. First, generate random numbers. To satisfy Then, according to equations (1) and (2), the pixel values ​​x1, x2 of any two images and their corresponding label values ​​y1, y2 are merged into the pixel values ​​and label values ​​x of the new image. new y new : Equation (1); Equation (2); Steps 1-5: Based on the atomic images generated in Step 1-3, and combined with the images formed by data augmentation in Step 1-4, simultaneously extract the atomic images of weak defects corresponding to the label values ​​and the corresponding augmentation sub-images, and extract the atomic images of strong defects corresponding to the label values ​​and the corresponding augmentation sub-images. Add all the extracted images to the training set of the neural network, and finally form a training set consisting of m 64×64 crack grayscale images. Steps 1-6: Train the convolutional neural network model using the formed image training set. The input of the network is a 64×64 grayscale image, which is flattened after several convolutions and finally outputs three label values ​​a, b, and c representing the degree of cracking through a fully connected neural network. Step 2: Acquire images of the surface of the building to be tested; Step 3: Use the convolutional neural network model constructed in step (1) to identify and display cracks in the collected images of the building surface to be tested; the crack identification and display process includes the following sub-steps: Step 3-1: Divide the image to be identified into several 128×128 small images at 64-pixel intervals, then scale the obtained small images to 64×64 size, and then import them into the convolutional neural network model trained in Step 1 in row by row. Based on the constructed convolutional neural network model, perform large-scale crack detection on the building surface image to be tested to form a building surface image with crack markings. Step 3-2: Divide the image to be identified into several 64×64 small images at 64-pixel intervals. Import the small images obtained from the division directly into the convolutional neural network model trained in Step 1. Based on the constructed convolutional neural network model, perform small-scale crack classification on the building surface image to be tested to form a building surface image with crack level markings. Step 3-3: Merge the building surface image with crack markings (completed in Step 3-1) and the building surface image with crack level markings (completed in Step 3-2), forming a building surface image with crack marking regions distributed in both large and small dimensions. For each building surface image, the areas with cracks can be identified through the large-dimensional crack marking regions. Simultaneously, the small-dimensional crack level marking regions are used to further verify crack areas not covered by the large-dimensional crack marking regions. The building surface image is then further processed... When processing crack marking regions with a large dimensional distribution, if the crack is located exactly on the segmentation line of the large dimensional segmented image, the final large dimensional distributed crack marking regions will not cover the crack region. In the same case, when processing the same building surface image with crack level marking regions with a small dimensional distribution, due to the different segmentation dimensions, the same crack will not be located on the segmentation line of the small dimensional segmented image, and the final small dimensional distributed crack level marking regions will effectively cover the crack region. Conversely, if the crack is located on the segmentation line of the small dimensional segmented image, the crack will be identified through the large dimensional distributed crack marking regions.

2. The method for identifying surface cracks in buildings according to claim 1, characterized in that, When performing large-scale crack detection in step 3-1, the following sub-steps are included: Step 3-1-1: Divide the image to be identified into several 128×128 small images at 64-pixel intervals; Step 3-1-2: Scale the small image obtained in Step 3-1-1 to a size of 64×64, and then import it row by row into the convolutional neural network model trained in Step 1; the convolutional neural network model processes the input small image to obtain the corresponding label values ​​a, b, c; Step 3-1-3: Identify and determine cracks based on label values ​​a, b, and c; Step 3-1-4: Match the crack identification results to the original image and annotate the original image according to the identification results.

3. The method for identifying surface cracks in buildings according to claim 1, characterized in that, Step 3-2, which involves classifying small-scale cracks, includes the following sub-steps: Step 3-2-1: Divide the image to be identified into several 64×64 small images at 64-pixel intervals; Step 3-2-2: Import the small images obtained in Step 3-2-1 into the convolutional neural network model trained in Step 1, row by row. The convolutional neural network model processes the input small images to obtain the corresponding label values ​​a, b, c. Step 3-2-3: Determine the crack grade based on the label values ​​a, b, and c; Step 3-2-4: Map the identified crack level results to the original image and annotate the original image according to the identified crack level results.

4. A building surface crack recognition system based on convolutional neural networks, characterized in that, include: A neural network training module is configured to train a neural network based on images of cracks on building surfaces, and to construct a convolutional neural network model capable of recognizing cracks on building surfaces according to the following steps: Step 1-1: Collect n images of surface cracks in actual buildings as samples for subsequent training of the convolutional neural network; Step 1-2: Adjust the exposure and contrast of the n images of building surface cracks collected in Step 1-1, and randomly crop the original images into sub-images of size 64×64; Steps 1-3: The degree of cracks in each sub-image obtained in Step 1-2 is judged manually and classified as no crack, weak crack, and strong crack, respectively, and a corresponding label value is given. Each sub-image used as a sample is labeled based on the three crack degree standards. Steps 1-4: Perform data augmentation on each segmented sub-image that has been processed in Steps 1-3. Generate new sub-image data by fine-tuning the original sub-image data, thereby forming augmented sub-images, which are then added to the training set as new sub-image samples. First, generate random numbers. To satisfy Then, according to equations (1) and (2), the pixel values ​​x1, x2 of any two images and their corresponding label values ​​y1, y2 are merged into the pixel values ​​and label values ​​x of the new image. new y new : Equation (1); Equation (2); Steps 1-5: Based on the atomic images generated in Step 1-3, and combined with the images formed by data augmentation in Step 1-4, simultaneously extract the atomic images of weak defects corresponding to the label values ​​and the corresponding augmentation sub-images, and extract the atomic images of strong defects corresponding to the label values ​​and the corresponding augmentation sub-images. Add all the extracted images to the training set of the neural network, and finally form a training set consisting of m 64×64 crack grayscale images. Steps 1-6: Train the convolutional neural network model using the formed image training set. The input of the network is a 64×64 grayscale image, which is flattened after several convolutions and finally outputs three label values ​​a, b, and c representing the degree of cracking through a fully connected neural network. Building surface image acquisition module, the building surface image acquisition module is used to acquire images of the building surface to be measured; A crack recognition module is provided, which uses a convolutional neural network model constructed by a neural network training module to perform crack recognition processing on the building surface images acquired by the building surface image acquisition module. The crack recognition module includes an image segmentation module, an input module, a judgment module, and a merging module. The image segmentation module is used to segment the image of the building surface to be tested into several sub-images for large-scale crack detection, or into several sub-images for small-scale crack classification. The input module interacts with the image segmentation module, importing the sub-images obtained by the image segmentation module into the trained convolutional neural network model row by row; dividing the image to be recognized into several 128×128 small images at 64-pixel intervals, then scaling the obtained small images to 64×64 size, and then importing them into the trained convolutional neural network model row by row; and dividing the image to be recognized into several 64×64 small images at 64-pixel intervals, and directly importing the segmented small images into the trained convolutional neural network model row by row. The determination module performs crack detection and identification or crack classification identification on the surface image of the building under test based on the output result of the convolutional neural network model, and marks the original image of the surface of the building under test based on the identification result. The merging module interacts with the determination module to merge building surface images with crack markings that have completed large-dimensional identification with building surface images with crack level markings that have completed small-dimensional identification. This results in a building surface image with crack marking regions distributed in large dimensions and crack level marking regions distributed in small dimensions. For each building surface image, the presence of cracks can be determined through the crack marking regions distributed in large dimensions; simultaneously, the crack level marking regions distributed in small dimensions are used to further verify crack areas on the building surface image that are not covered by the crack marking regions distributed in large dimensions. For the building surface image... When processing crack marking regions with a large dimensional distribution, if the crack is located on the segmentation line of the large dimensional segmented image, the resulting large dimensional crack marking regions will not cover the crack region. In the same case, when processing the same building surface image with crack level marking regions with a small dimensional distribution, due to the different segmentation dimensions, the same crack will not be located on the segmentation line of the small dimensional segmented image, and the resulting small dimensional crack level marking regions will effectively cover the crack region. Conversely, if the crack is located on the segmentation line of the small dimensional segmented image, the crack will be identified through the large dimensional crack marking regions.

5. The building surface crack identification system according to claim 4, characterized in that, The convolutional neural network model trained by the neural network training module can generate multiple label values ​​representing the degree of cracking for the image to be processed.

Images