A method, system, medium and program product for detecting cracks in building exterior walls
Through the VMamba encoder and decoder combined with detailed guidance, the gated fusion mechanism is solved with the problem of variable shape and size identification in the crack detection of building exterior walls, the detection accuracy and robustness are improved, and efficient crack extraction is achieved.
Patent Information
- Application Number
- CN202411476635.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2044-10-22
AI Technical Summary
Existing methods for detecting cracks in exterior walls of building easily lead to segmentation and fractures when dealing with cracks of varying shapes and sizes. To improve segmentation accuracy requires complex algorithms and multiple computing resources, resulting in a decrease in processing speed, while simple models may reduce accuracy.
Using the VMamba encoder and decoder combined with detailed guidance of the gated fusion mechanism, the network structure is constructed through visual state space blocks and convolutional neural networks, multi-scale features are extracted, and crack geometric features are estimated using digital image processing methods to improve detection accuracy and robustness.
It improves the precision of crack structure extraction, improves the accuracy and robustness of crack detection in building exterior walls, solves the contradiction between crack identification integrity and processing speed, and achieves a more efficient detection effect.
Smart Images
Figure CN119006454B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to image processing technology in the field of building exterior wall safety, and in particular to a building exterior wall crack detection method, system, medium and program product. Background Art
[0002] For buildings that have been built and put into use for a certain number of years, there may be some damage to the exterior walls of the buildings, such as cracks, which poses a certain threat to the safety of the buildings. Therefore, it is particularly important to detect whether there are damages such as cracks on the exterior walls of buildings to ensure the safety of the exterior walls of buildings. Existing crack detection technologies are divided into two types: traditional methods and deep learning-based methods.
[0003] Traditional methods mainly use digital image processing technology, such as threshold segmentation and edge detection algorithms. Although simple, they are used in a single scenario and lack robustness and generalization. In recent years, deep learning technology has demonstrated powerful image feature extraction capabilities and complex structure expression capabilities in the field of vision. Introducing it into crack detection tasks has become a development trend in crack detection, which further improves the crack detection accuracy and robustness and generalization in complex scenarios. For example, CrackSegNet (Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction and Building Materials 234 (2020): 117367) uses convolutional neural networks to build an encoder-decoder network structure, and introduces dilated convolution and spatial pyramid pooling, which increases the receptive field to a certain extent, captures the multi-scale information of the image, enables the network to obtain a wider range of contextual information, and improves the recognition accuracy of crack detection. EMRA-Net (EMRA-Net: A pixel-wise network fusing local and global features for tiny and low-contrast surface defect detection. IEEE Transactions on Instrumentation and Measurement71(2022): 1-14) proposes a pyramid edge module and a multi-scale fusion module based on the pre-trained ResNet34. Through the self-learning weighting of the multi-scale fusion module, the local pyramid edge features, global convolutional features and global dynamic multi-scale features are effectively fused to effectively improve the detection accuracy of tiny defects. CrackFomer-II (Crackformer networkfor pavement crack segmentation. IEEE Transactions on IntelligentTransportation Systems (2023)) embeds a novel Transformer encoder module into an encoder-decoder structure similar to SegNet, and uses the Transformer's self-attention mechanism to obtain long-distance dependencies in the image. In addition, a scale-attention fusion module is proposed to fuse the results of each encoder and decoder block to highlight semantic features and better identify crack features.APFNet (An attention-based progressive fusion networkfor pixelwise pavement crack detection. Measurement 226 (2024): 114159) introduces separable convolution into crack detection to reduce the number of model parameters, achieves multi-scale feature enhancement through cross-layer feature learning fusion, and uses hybrid multi-attention modules to capture the long-term dependencies of cracks to improve the ability to identify cracks. Although many advanced methods using deep learning have been proposed for crack detection, existing crack recognition methods still face huge challenges:.
[0004] (1) Cracks vary in shape and size, from tiny cracks to larger fracture zones, which can easily cause crack breakage during the segmentation process, increasing the difficulty of identifying complete cracks.
[0005] (2) Improving segmentation accuracy usually requires more complex algorithms and more computing resources, and the processing speed decreases; conversely, using a simple model to increase the processing speed may reduce the segmentation accuracy to a certain extent. Summary of the invention
[0006] In view of the problems in the background technology, the purpose of the present invention is to provide a method, system, medium and program product for detecting cracks in the exterior walls of buildings, so as to improve the precision of crack structure extraction and enhance the detection accuracy and robustness of cracks in the exterior walls of buildings.
[0007] In a first aspect, a method for detecting cracks in a building exterior wall is provided, comprising the following steps:
[0008] Acquire the exterior wall image of the building to be detected;
[0009] Input the image of the building exterior wall to be detected into the trained building exterior wall crack detection model, and output the building exterior wall crack detection result;
[0010] Wherein, the building exterior wall crack detection model includes an encoder and a decoder;
[0011] The encoder uses a Vmamba (visual state space model) encoder to extract image features and obtain feature maps of different resolutions in four stages;
[0012] The decoder is constructed using VMamba's visual state space blocks and convolutional neural networks, and a detail-guided gated fusion mechanism is introduced to fuse feature maps of different resolutions at each stage of the encoder and decoder. Finally, a residual module and a classification head are used to obtain the crack detection result.
[0013] Furthermore, the encoder first divides the input image of the building exterior wall to be detected into blocks, retains the two-dimensional structure of the image, and constructs each stage by stacking multiple visual state space blocks. The number of visual state space blocks in the four stages is 2, 2, 4, and 2, respectively, and the feature map of the previous stage is downsampled before inputting the next stage, and finally feature maps of different resolutions in four stages are obtained.
[0014] Furthermore, the decoder first inputs the feature map of the last stage of the encoder into a visual state space block, and then passes it through a convolution module to obtain a corresponding feature map, which is upsampled and then feature fused with the feature map of the corresponding stage of the encoder; the above process is repeated until the feature map of the last stage of the decoder is obtained; the feature map of the last stage of the decoder is passed through a residual module to obtain mapping features, and finally a classification head is used to obtain the crack detection result.
[0015] Furthermore, the gated fusion mechanism introduced with detail guidance fuses feature maps of different resolutions at each stage of the encoder and decoder, specifically including:
[0016] Add the feature map Fd from the decoder and the feature map Fe from the corresponding stage of the encoder;
[0017] The feature map X obtained after addition is input into the detail-guided attention module, and the fusion weight W is obtained after the gating mechanism Sigmoid activation function;
[0018] Multiply the feature map Fe by the fusion weight W, and correspondingly, multiply the feature map Fd by 1-W to obtain the feature maps after detail guidance and enhancement;
[0019] Finally, the enhanced feature map is added to the original Fe and Fd to retain more information, and the number of channels is adjusted through a 1×1 convolution layer to obtain the final fusion feature of the encoding layer feature and the decoding layer feature. , as follows:
[0020]
[0021] In the formula, It is a convolution operation with a convolution kernel of 1×1;
[0022] The detail-guided attention module makes full use of the spatial attention mechanism and the channel attention mechanism. First, the feature map X is subjected to maximum pooling and average pooling operations in the channel dimension, and then the two output feature maps are concatenated in the channel dimension and the number of channels of the feature map is adjusted through a 7×7 convolutional layer to obtain the corresponding attention of different positions in the feature map. , specifically:
[0023]
[0024] In the formula, is a convolution operation with a convolution kernel of 7×7, GAP is an average pooling operation, GMP is a maximum pooling operation, and [] indicates concatenation in the channel dimension;
[0025] The feature map X is subjected to the maximum pooling operation in the spatial dimension. In order to reduce the number of parameters and reduce the model complexity, a 1×1 convolution layer is used to reduce the channel dimension from C to C / r. The second 1×1 convolution layer expands the channel dimension back to C, and r is set to 16 to obtain the channel attention. ; By and Add together to get the preliminary feature attention map , as follows:
[0026]
[0027]
[0028] In the formula, Relu is a linear activation function;
[0029] Then use the input feature map X as a guide to generate the final feature attention map; concatenate the feature map X and the feature attention map Wcos in the channel dimension, shuffle and rearrange the channels, and input them into the 5×5 group convolution. In order to enhance the features and reduce the loss of information, a residual connection is added, and a 1×1 convolution layer is used to align the number of channels of the two; finally, the final fusion weight is obtained through the Sigmoid activation function. , specifically:
[0030]
[0031] in, is the Sigmoid activation function, It is a group convolution with a convolution kernel of 5×5, and CS is channel shuffle.
[0032] Furthermore, it also includes:
[0033] The geometric features of the cracks were estimated using digital image processing methods, the pixel length of the cracks was estimated using a skeleton line extraction method, and the maximum pixel width of the cracks was estimated using the maximum inscribed circle algorithm.
[0034] The pixel length and maximum pixel width of the crack are converted into the actual length and width of the crack according to the corresponding spatial resolution of the image, so as to be able to evaluate the damage of the building exterior wall.
[0035] Furthermore, the method based on skeleton line extraction is used to estimate the crack pixel length, specifically including:
[0036] Extract the skeleton of the crack in the crack detection result image, and obtain the pixel length of the crack by calculating the length of the extracted skeleton line;
[0037] The method of estimating the maximum pixel width of the crack by using the maximum inscribed circle algorithm specifically includes:
[0038] Extracting and marking crack contours on the crack detection result image;
[0039] Construct the minimum circumscribed rectangular frame containing the crack, and use the ray method to determine whether the pixel points in the rectangular frame are within the crack outline. Draw a ray from the pixel to be determined through the crack outline. If the number of intersections with the crack outline is odd, the point is within the crack outline, otherwise it is outside the crack outline, so as to obtain the coordinates of all pixel points within the crack outline.
[0040] The radius of the inscribed circle from each pixel point within the crack contour to the crack contour is calculated, and the pixel radius of the largest inscribed circle is calculated using the dichotomy method. The maximum pixel width of the crack is twice the pixel radius of the largest inscribed circle.
[0041] Furthermore, in the process of training the building exterior wall crack detection model, the loss function used is the cross entropy loss function, and its formula is as follows:
[0042]
[0043] Among them, L represents the cross entropy loss, represents the true label of the sample, Represents the probability value predicted by the building exterior wall crack detection model.
[0044] In a second aspect, a building exterior wall crack detection system is provided, comprising:
[0045] Memory on which computer programs or instructions are stored;
[0046] The processor is used to load and execute the computer program or instructions to implement the building exterior wall crack detection method as described above.
[0047] In a third aspect, a computer-readable storage medium is provided, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the method for detecting cracks in a building exterior wall as described above is implemented.
[0048] In a fourth aspect, a computer program product is provided, wherein a computer program or instruction is stored in the computer program product, and when the computer program or instruction is executed by a processor, the method for detecting cracks in a building exterior wall as described above is implemented.
[0049] The present invention proposes a method, system, medium and program product for detecting cracks in building exterior walls. Compared with the prior art, the present invention mainly has the following advantages: the present invention introduces a VMamba encoder for crack detection, and uses a visual state space module and a convolutional neural network to construct a decoder, fully utilizing the advantages of VMamba, solving the limitations of Transformer in computational complexity to a certain extent, improving the computational rate, while retaining its advantages of global receptive field and dynamic weight, and retaining the network's ability to extract local features through the convolutional neural network, thereby improving the ability to extract complete cracks; a detail-guided adaptive gating fusion strategy is introduced to better fuse encoding features and decoding features from different stages, learn crack structure features with variable shapes and different scales, further improve the refinement of crack structure extraction, enhance the learning of crack features with complex shapes, and improve the detection accuracy and robustness of building exterior wall cracks. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0051] Figure 1 It is a flow chart of a method for detecting cracks in a building exterior wall provided by an embodiment of the present invention;
[0052] Figure 2 It is a framework diagram of a building exterior wall crack detection model provided by an embodiment of the present invention;
[0053] Figure 3 2 is a schematic diagram of the structure of a visual state space block VSS Block provided by an embodiment of the present invention;
[0054] Figure 4 Schematic diagram of the structure of a detail-guided adaptive gating fusion module DGAGF provided in an embodiment of the present invention;
[0055] Figure 5 is a schematic diagram of the structure of a detail-guided attention module DGA provided in an embodiment of the present invention;
[0056] Figure 6 1 is an example diagram of calculating the geometric features of cracks provided by an embodiment of the present invention, wherein (a) is a schematic diagram of estimating the pixel length of a crack using a method based on skeleton line extraction, and (b) is a schematic diagram of obtaining the maximum pixel width of a crack using a maximum inscribed circle algorithm;
[0057] Figure 7is a visualization result of crack segmentation provided by an embodiment of the present invention;
[0058] Figure 8 This is an example of calculating the geometric dimensions of a crack segmentation result provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0059] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0060] Cracks on building exterior walls vary in shape and size, from tiny cracks to larger fracture zones. Existing crack detection methods easily cause cracks to break during the segmentation process, increasing the difficulty of identifying complete cracks. In existing crack detection methods, improving segmentation accuracy usually requires more complex algorithms and more computing resources, and the processing speed decreases; conversely, using a simple model to increase the processing speed may reduce the segmentation accuracy to a certain extent. Therefore, how to improve the processing speed while ensuring the crack segmentation accuracy is a problem that needs to be overcome.
[0061] In view of the above technical difficulties, the present invention proposes a solution for detecting cracks in building exterior walls. The present invention includes constructing a building exterior wall crack data set of drone images, and constructing a codec network based on the VMamba architecture. The encoder is divided into four stages, each of which is composed of multiple visual state space blocks, and the decoder is also divided into four stages accordingly, each of which is constructed by a visual state space block and a convolution module; a detail-guided adaptive gated fusion strategy is introduced to learn the structural features of cracks with variable shapes and scales, further improve the refinement of crack structure extraction, and improve the detection accuracy and robustness of building exterior wall cracks; and the digital image processing method is used to calculate the size information of the cracks, and convert it into actual size information according to the corresponding spatial resolution of the image to evaluate the degree of building damage. The technical solution of the present invention is described in detail below in conjunction with specific embodiments.
[0062] Before detecting cracks in building exterior walls, it is necessary to build a trained model for detecting cracks in building exterior walls. The specific process is as follows:
[0063] Step 1: Obtain building exterior wall images collected by drones, mark the cracks in the building exterior wall images, crop the images, filter out images with cracks, and construct a building exterior wall crack dataset for subsequent training.
[0064] Step 2: Construct a building exterior wall crack detection model, which specifically includes an encoder and a decoder; the encoder uses the VMamba encoder to extract image features and obtain feature maps of different resolutions in four stages; the encoder is constructed using the core modules of VMamba, the visual state space block and the convolutional neural network, and a detail-guided gated fusion mechanism is introduced to fuse the outputs of the encoder and decoder at each stage with different resolutions, and finally a residual module and a classification head are used to obtain the building exterior wall crack detection results.
[0065] Specifically, Figure 2 As shown in the figure, the core of the VMamba encoder is the visual state space block VSS Block, which introduces cross scanning so that Mamba can better adapt to image data and maintain the integrity of the receptive field. VMamba first divides the input image into blocks to retain the two-dimensional structure of the image. Each stage is constructed by stacking multiple visual state space blocks. The number of visual state space blocks in the four stages is 2, 2, 4, and 2 respectively, and the feature map of the previous stage is downsampled before entering the next stage, and finally the feature maps of different resolutions in the four stages are obtained. The visual state space block and convolutional neural network are used to construct the decoder part. The feature map is first input into a visual state space block, and then passed through the convolution module to obtain the corresponding output feature map. , after upsampling, the features with the same resolution as the corresponding stage of the encoder Figure 1 The input detail-guided adaptive gated fusion module DGAGF is used to achieve feature fusion to retain more detail information. Finally, a residual module ResBlock and a classification head, i.e., a 1×1 convolutional layer, are used to accurately identify crack information. The expression of the convolutional module is:
[0066]
[0067] In the formula, F represents the input feature map, is a convolution operation with a convolution kernel of 1×1, BN is batch normalization, and Relu is a linear activation function.
[0068] The expression of the residual module ResBlock is:
[0069]
[0070] Where Fr represents the feature map of the output of the residual module ResBlock, It is a convolution operation with a convolution kernel of 3×3.
[0071] like Figure 3As shown in the figure, the specific structure of VMamba's core module (i.e., visual state space block VSS block) is shown. It contains two residual modules. The residual connection effectively alleviates the problems of gradient vanishing and gradient exploding, making the network easier to optimize. The input is divided into two branches, one of which is input into the two-dimensional selective scanning block SS2D block after layer normalization, and then added to the original features of the other branch. The SS2D block contains the most important two-dimensional selective scanning SS2D. Before entering SS2D, it must first pass through a linear layer, Dwconv (depthwise separable convolution) and SiLU activation function to increase the nonlinearity of the feature map, while ensuring efficiency and improving the performance of the model; the output of SS2D needs to pass through layer normalization and a linear layer to eliminate internal covariate offsets, map the input data to a new space, and form a higher-level feature representation. After passing through the first residual block, the corresponding output features are sent to the second residual block. Similarly, in the second residual block, it is also divided into two branches. One of the branches is input into the FFN layer after layer normalization. The FFN layer, namely the multi-layer perceptron MLP, further extracts and transforms the features of the data, and then adds the features of the other branch to obtain the output features of the final VSS block.
[0072] like Figure 4 The figure shows the structure diagram of the detail-guided adaptive gated fusion module DGAGF. First, the feature map Fd from the decoder needs to be added to the feature map Fe from the corresponding stage of the encoder, and the added feature map X is input into the detail-guided attention module DGA. After the gating mechanism Sigmoid activation function, the fusion weight W is obtained. The feature map Fe is multiplied by the fusion weight W, and correspondingly, the feature map Fd is multiplied by 1-W to obtain the feature maps that have been detail-guided and enhanced. Finally, the enhanced features are added to the original Fe and Fd to retain more information, and the number of channels is adjusted through a 1×1 convolution layer to obtain the final fusion features of the encoding layer features and the decoding layer features. , as follows:
[0073]
[0074] In the formula, It is a convolution operation with a convolution kernel of 1×1.
[0075] like Figure 5 As shown in the figure, the structural diagram of the detail-guided attention module DGA is shown. The detail-guided attention module makes full use of the spatial attention mechanism and the channel attention mechanism. First, the feature map X is subjected to the maximum pooling and average pooling operations in the channel dimension, and then the two output feature maps are spliced in the channel dimension and the number of channels of the feature map is adjusted through a 7×7 convolution layer to obtain the corresponding attention of different positions in the feature map. , specifically:
[0076]
[0077] In the formula, is a convolution operation with a convolution kernel of 7×7, GAP is an average pooling operation, GMP is a maximum pooling operation, and [] indicates concatenation in the channel dimension;
[0078] In addition, the feature map X is subjected to a maximum pooling operation in the spatial dimension. In order to reduce the number of parameters and reduce the model complexity, a 1×1 convolution layer is used to reduce the channel dimension from C to C / r. The second 1×1 convolution layer expands the channel dimension back to C. In this embodiment, r is set to 16, and the channel attention is obtained. ; By and Add together to get the preliminary feature attention map , as follows:
[0079]
[0080]
[0081] Then use the input feature map X as a guide to generate the final feature attention map; concatenate the feature map X and the feature attention map Wcos in the channel dimension, shuffle and rearrange the channels, and input them into the 5×5 group convolution. In order to enhance the features and reduce the loss of information, a residual connection is added, and a 1×1 convolution layer is used to align the number of channels of the two; finally, the final fusion weight is obtained through the Sigmoid activation function. , specifically:
[0082]
[0083] in, is the Sigmoid activation function, It is a group convolution with a convolution kernel of 5×5, and CS is channel shuffle.
[0084] Step 3: Use the building exterior wall crack data set to train the constructed building exterior wall crack detection model to obtain a trained building exterior wall crack detection model. During the training process, the loss function used is the cross entropy damage function, which measures the difference between the probability distribution predicted by the model and the true probability distribution. Its formula is:
[0085]
[0086] Among them, L represents the cross entropy loss, represents the true label of the sample, Represents the probability value predicted by the building exterior wall crack detection model.
[0087] Based on the above-trained building exterior wall crack detection model, an embodiment of the present invention provides a building exterior wall crack detection method, such as Figure 1 As shown, the following steps are included:
[0088] S1: Obtain the exterior wall image of the building to be detected;
[0089] S2: Input the image of the building exterior wall to be detected into the trained building exterior wall crack detection model, and output the building exterior wall crack detection result.
[0090] The method for detecting cracks in the building exterior wall provided by the above embodiment mainly has the following advantages: the image of the building exterior wall is obtained by a drone, the crack area is marked and cropped and screened, and a building exterior wall crack data set is constructed, which makes up for the current situation of insufficient building exterior wall crack data to a certain extent; the VMamba encoder is introduced for crack detection, and the decoder is constructed using the visual state space module and the convolutional neural network, which makes full use of the advantages of VMamba, solves the limitations of the Transformer in computational complexity to a certain extent, improves the computational rate, and retains its advantages of global receptive field and dynamic weight, and retains the network's ability to extract local features through the convolutional neural network, thereby improving the ability to extract complete cracks; the detail-guided adaptive gating fusion strategy is introduced to better fuse the encoding features and decoding features from different stages, enhance the learning of complex-shaped crack features, and achieve higher-precision recognition of cracks.
[0091] In another embodiment of the present invention, a method for detecting cracks in a building exterior wall is provided. Figure 1 As shown, the following steps are included:
[0092] S1: Obtain the exterior wall image of the building to be detected;
[0093] S2: Input the image of the building exterior wall to be detected into the trained building exterior wall crack detection model, and output the building exterior wall crack detection result;
[0094] S3: Use digital image processing methods to estimate the geometric characteristics of the cracks, use a skeleton line extraction method to estimate the pixel length of the cracks, and use the maximum inscribed circle algorithm to estimate the maximum pixel width of the cracks;
[0095] S4: The pixel length and maximum pixel width of the crack are converted into the actual length and width of the crack according to the corresponding spatial resolution of the image, so as to be able to evaluate the damage of the building exterior wall.
[0096] like Figure 6As shown, a schematic diagram of the calculation process of crack geometric characteristics is shown. Figure 6 (a) is a schematic diagram of estimating the pixel length of a crack using a skeleton line extraction method. The pixel length of a crack is defined as the length of the skeleton line at the center of the crack. The skeleton line of each crack is first extracted, and then the length of each crack is calculated for each skeleton line. For the calculation of the skeleton line length, the distance between two adjacent pixels can be calculated in sequence and summed to calculate the length of the entire crack. The distance between the kth pixel and the k+1th pixel on the skeleton line for:
[0097]
[0098] In the formula, , and , are the pixel coordinates of the kth pixel and the k+1th pixel respectively.
[0099] The total length of the crack is:
[0100]
[0101] In the formula, is the total length of the crack.
[0102] For the calculation of crack width, Figure 6 As shown in (b), the maximum inscribed circle algorithm is used to obtain the maximum pixel width and length of the crack. The specific method is:
[0103] (1) Extract and mark the crack contours on the crack detection result image;
[0104] (2) Construct a minimum circumscribed rectangular box containing the crack, and use the ray method to determine whether the pixel points in the rectangular box are within the crack contour. Draw a ray from the pixel to be determined through the crack contour. If the number of intersections with the crack contour is odd, the point is within the crack contour. Otherwise, the point is outside the crack contour. The coordinates of all pixel points within the crack contour are obtained.
[0105] (3) Calculate the radius of the inscribed circle from each pixel point within the crack contour to the crack contour, and use the bisection method to calculate the pixel radius of the largest inscribed circle. The maximum pixel width of the crack is twice the pixel radius of the largest inscribed circle.
[0106] The effects of the scheme of the present invention are further described below with reference to specific examples.
[0107] like Figure 7The figure shows the visualization result of crack segmentation. From left to right are the original image, crack label, CrackSegNet (Image-based concrete crack detection in tunnels using deep fully convolutional networks. Construction and Building Materials 234 (2020): 117367), EMRA-Net (EMRA-Net: A pixel-wise network fusing local and globalfeatures for tiny and low-contrast surface defect detection. IEEETransactions on Instrumentation and Measurement 71 (2022): 1-14), Crackformer-II (Crackformer network for pavement crack segmentation. IEEE Transactions onIntelligent Transportation Systems (2023)), APFNet (An attention-basedprogressive fusion network for pixelwise pavement crack detection. Measurement 226 (2024): 114159), and the crack segmentation result of the method of the present invention. It can be seen that in various scenarios of cracks in building exterior walls, including painted walls, concrete walls and tiled walls, the method of the present invention achieves more accurate segmentation of cracks, reduces the phenomenon of crack identification interruption, and improves the ability to fully identify cracks.
[0108] like Figure 8As shown in the figure, it is an example of the calculation of the geometric dimensions of the crack segmentation results. From left to right, it is the original image, the segmentation result of the method of the present invention, the outline skeleton of the crack result, and the example of the maximum width of the crack. For example, in the first group of crack images, a crack is identified, the length of the crack skeleton line is 578.71 pixels, and the corresponding spatial resolution of the image is 0.14 mm, that is, each pixel corresponds to an actual distance of 0.14 mm. The length of the crack detected by the method of the present invention is calculated to be 81.02 mm; the maximum width of the crack is estimated by the maximum inscribed circle algorithm. The maximum width of the first group of crack images is 4.24 pixels. The crack width at this point is calculated to be 0.59 mm. The width measured by the crack meter is 0.45 mm. It can be seen that it is not much different from the measured result. A crack was identified in the second set of crack images, with a central skeleton line length of 550.5 pixels and an image corresponding spatial resolution of 0.099 mm. The crack length detected by the method of the present invention was calculated to be 54.5 mm. The maximum width of the crack was estimated using the maximum inscribed circle algorithm. The maximum width of the second set of crack images was 3.98 pixels. The crack width at this point was calculated to be 0.39 mm, and the width measured by the crack meter was 0.3 mm, which is basically consistent with the measured results.
[0109] Verify the effectiveness of the method of the present invention. The self-built UAV image building exterior wall crack dataset is used for training and testing, and compared with the methods proposed in recent years for crack detection or defect detection. The UAV image building exterior wall crack dataset is taken by drones and contains a variety of building exterior wall crack scenes, such as tile exterior walls, painted exterior walls and concrete exterior walls. The resolution of the original UAV image taken is about 8000×5000. The manual annotation mode of the semantic segmentation and annotation software Isat-sam is used to annotate the crack area, and the crack area is annotated as white and the background is black. After the annotation is completed, the high-resolution UAV image is cropped to 512×512 pixels, and the pictures with cracks are selected to form the final building exterior wall crack dataset. Finally, 2094 crack pictures are selected and divided into training set and test set according to the ratio of 4:1 for training and testing. Finally, 1671 training set pictures and 423 test set pictures are obtained. This experiment uses the AdamW optimizer, with an initial learning rate of 0.0001 and a batch size of 4. The experimental comparison selected CrackSegNet (Image-basedconcrete crack detection in tunnels using deep fully convolutional networks. Construction and Building Materials 234 (2020): 117367), EMRA-Net (EMRA-Net: Apixel-wise network fusing local and global features for tiny and low-contrastsurface defect detection. IEEE Transactions on Instrumentation and Measurement 71 (2022): 1-14), CrackFomer-II (Crackformer network for pavementcrack segmentation. IEEE Transactions on Intelligent Transportation Systems (2023) ), APFNet (An attention-based progressive fusion network for pixelwisepavement crack detection. Measurement 226 (2024): 114159).
[0110] Table 1 shows the experimental results (%) of the comparison between the method of the present invention and other crack detection methods on a self-built building exterior wall crack dataset.
[0111]
[0112] Crack detection and segmentation can be classified as a binary classification problem. TP is the predicted positive sample, FN is the predicted incorrect positive sample, FP is the predicted incorrect negative sample, and TN is the predicted correct negative sample. Commonly used semantic segmentation evaluation indicators include precision, recall, F1-score (F1), mIoU, etc. Among them, precision refers to the ratio of the predicted correct positive sample to all predicted positive samples. This evaluation indicator reflects the false positive rate; recall refers to the ratio of the predicted correct positive sample to all true positive samples, reflecting the missed detection rate; F1-score is the harmonic mean of precision and recall. It takes into account both the false positive rate and the missed detection rate, and can better reflect the detection ability of the method used; IoU is the intersection-over-union ratio of the predicted image and the actual image, and mIoU is the average of the IoU of all categories. Since F1-score and mIoU can better reflect the ability of segmentation and recognition, this article mainly refers to the F1-score indicator and mIoU to compare the effectiveness of various methods. The expressions of each indicator are as follows:
[0113]
[0114]
[0115]
[0116]
[0117] Table 1 compares the recognition accuracy of the four algorithms and the method used in this example. For F1-score, the scheme of the present invention is 3.22%, 2.78%, 2.78%, and 2.7% higher, respectively; for mIoU, the scheme of the present invention is 2.08%, 1.8%, 1.81%, and 1.75% higher, respectively, which proves the superiority of the scheme of the present invention.
[0118] In addition, an embodiment of the present invention further provides a building exterior wall crack detection system, comprising:
[0119] Memory on which computer programs or instructions are stored;
[0120] The processor is used to load and execute the computer program or instructions to implement the building exterior wall crack detection method described in the above embodiment.
[0121] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the method for detecting cracks in a building exterior wall described in the aforementioned embodiment is implemented.
[0122] In addition, an embodiment of the present invention further provides a computer program product, in which a computer program or instructions are stored. When the computer program or instructions are executed by a processor, the method for detecting cracks in a building exterior wall described in the aforementioned embodiment is implemented.
[0123] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.
[0124] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0125] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0128] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for detecting cracks in a building exterior wall, characterized in that: The steps include: Acquire the exterior wall image of the building to be detected; Input the image of the building exterior wall to be detected into the trained building exterior wall crack detection model, and output the building exterior wall crack detection result; Wherein, the building exterior wall crack detection model includes an encoder and a decoder; The encoder uses a VMamba encoder to extract image features and obtain feature maps of different resolutions in four stages; The decoder is constructed using VMamba's visual state space block and convolutional neural network, and a detail-guided gated fusion mechanism is introduced to fuse feature maps of different resolutions at each stage of the encoder and decoder. Finally, a residual module and a classification head are used to obtain the crack detection result. The gated fusion mechanism introduced with detail guidance fuses feature maps of different resolutions at each stage of the encoder and decoder, specifically including: Add the feature map Fd from the decoder and the feature map Fe from the corresponding stage of the encoder; The feature map X obtained after addition is input into the detail-guided attention module, and the fusion weight W is obtained after the gated mechanism Sigmoid activation function; Multiply the feature map Fe by the fusion weight W, and multiply the feature map Fd by 1-W to obtain the feature maps after detail guidance and enhancement; Finally, the enhanced feature map is added to the original Fe and Fd, and the number of channels is adjusted through a 1×1 convolution layer to obtain the final fusion feature of the encoding layer feature and the decoding layer feature. , as follows: ； In the formula, It is a convolution operation with a convolution kernel of 1×1; The detail-guided attention module first performs maximum pooling and average pooling operations on the feature map X in the channel dimension, then concatenates the two output feature maps in the channel dimension and adjusts the number of channels of the feature map through a 7×7 convolutional layer to obtain the corresponding attention of different positions in the feature map. , specifically: ； In the formula, is a convolution operation with a convolution kernel of 7×7, GAP is an average pooling operation, GMP is a maximum pooling operation, and [] indicates concatenation in the channel dimension; Perform the maximum pooling operation on the feature map X in the spatial dimension, use a 1×1 convolution layer to reduce the channel dimension from C to C / r, and the second 1×1 convolution layer expands the channel dimension back to C to obtain the channel attention ; By and Add together to get the preliminary feature attention map , as follows: ； ； In the formula, Relu is a linear activation function; Then use the input feature map X as a guide to generate the final feature attention map; concatenate the feature map X and the feature attention map Wcos in the channel dimension, shuffle and rearrange the channels, and input them into the 5×5 grouped convolution, add residual connections, and use a 1×1 convolution layer to align the number of channels of the two; finally, the final fusion weight is obtained through the Sigmoid activation function , specifically: ； in, is the Sigmoid activation function, It is a group convolution with a convolution kernel of 5×5, and CS is channel shuffle.
2. The building exterior wall crack detection method according to claim 1, characterized in that: The encoder first divides the input building exterior wall image to be detected into blocks, retaining the two-dimensional structure of the image. Each stage is constructed by stacking multiple visual state space blocks. The number of visual state space blocks in the four stages is 2, 2, 4, and 2, respectively. The feature map of the previous stage is downsampled before inputting the next stage, and finally feature maps of different resolutions in the four stages are obtained.
3. The building exterior wall crack detection method according to claim 1, characterized in that: The decoder first inputs the feature map of the last stage of the encoder into a visual state space block, and then passes it through a convolution module to obtain a corresponding feature map, which is upsampled and then feature-fused with the feature map of the corresponding stage of the encoder; the above process is repeated until the feature map of the last stage of the decoder is obtained; the feature map of the last stage of the decoder is passed through a residual module to obtain mapping features, and finally a classification head is used to obtain a crack detection result.
4. The building exterior wall crack detection method according to claim 1, characterized in that: Also includes: The pixel length of the crack is estimated by using a skeleton line extraction method, and the maximum pixel width of the crack is estimated by using the maximum inscribed circle algorithm. The pixel length and maximum pixel width of the crack are converted into the actual length and width of the crack according to the corresponding spatial resolution of the image.
5. The building exterior wall crack detection method according to claim 4, characterized in that: The method based on skeleton line extraction is used to estimate the crack pixel length, specifically including: Extract the skeleton of the crack in the crack detection result image, and obtain the pixel length of the crack by calculating the length of the extracted skeleton line; The method of estimating the maximum pixel width of the crack by using the maximum inscribed circle algorithm specifically includes: Extracting and marking crack contours on the crack detection result image; Construct the minimum circumscribed rectangular frame containing the crack, and use the ray method to determine whether the pixel points in the rectangular frame are within the crack outline. Draw a ray from the pixel to be determined through the crack outline. If the number of intersections with the crack outline is odd, the point is within the crack outline, otherwise it is outside the crack outline, so as to obtain the coordinates of all pixel points within the crack outline. The radius of the inscribed circle from each pixel point within the crack contour to the crack contour is calculated, and the pixel radius of the largest inscribed circle is calculated using the dichotomy method. The maximum pixel width of the crack is twice the pixel radius of the largest inscribed circle.
6. The building exterior wall crack detection method according to claim 1, characterized in that: In the process of training the building exterior wall crack detection model, the loss function used is the cross entropy loss function, and its formula is as follows: ； Among them, L represents the cross entropy loss, represents the true label of the sample, Represents the probability value predicted by the building exterior wall crack detection model.
7. A building exterior wall crack detection system, characterized in that: include: Memory on which computer programs or instructions are stored; A processor is used to load and execute the computer program or instruction to implement the building exterior wall crack detection method as described in any one of claims 1 to 6.
8. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the method for detecting cracks in a building exterior wall as claimed in any one of claims 1 to 6 is implemented.
9. A computer program product, characterized in that The computer program product stores a computer program or instructions, and when the computer program or instructions are executed by a processor, the method for detecting cracks in a building exterior wall as claimed in any one of claims 1 to 6 is implemented.