A neural network-based building crack detection method

Abstract

The application provides a building crack detection method based on a neural network, comprising the following steps: S100, acquiring a building crack image and performing illumination compensation on the image; S200, extracting multi-scale features of different resolutions of the building crack in the image after the illumination compensation and fusing the features; S300, determining a crack position by obtaining a crack thermal map through the fused features, focusing on a crack region by using a spatial attention mechanism based on a crack position map and the fused feature map, and detecting a crack width, a crack length and a crack direction.

Description

Technical Field

[0001] This invention relates to an image recognition technology, and more particularly to a method for detecting building cracks based on neural networks. Background Technology

[0002] Crack detection in buildings, as a core component of structural health monitoring, is a crucial link in ensuring the safety of building structures. Traditional detection methods mainly rely on manual visual inspection and the use of simple measuring tools, but these manual methods have shown many shortcomings in today's rapidly developing construction industry. With the development of artificial intelligence and computer vision technologies, building crack detection technology is undergoing a rapid transformation from traditional manual inspection to intelligent and automated methods. Patent publication number CN120318225A discloses a building engineering quality inspection and evaluation system based on machine vision, which involves a technique for obtaining crack edges using image recognition. Patent publication number CN120318230A discloses a method and system for detecting the maximum crack width in building structures based on machine vision, which involves a technical means of obtaining the maximum crack width using image recognition technology. Although the aforementioned patents utilize artificial intelligence technology to identify cracks, the following problems still exist: (1) The ability to identify minute cracks is limited, especially micro-cracks with a width of less than 0.2 mm, which are almost impossible to detect in low-resolution images; (2) The algorithm is easily affected by stains, shadows, and textures on building surfaces, frequently misclassifying non-crack features as cracks; (3) It has poor adaptability to different surface materials, requiring separate adjustment of algorithm parameters for different building materials such as concrete, brick walls, and stone. Studies have shown that the detection accuracy of traditional edge detection algorithms is about 85% under ideal conditions, but may drop sharply to below 60% in actual engineering environments. Summary of the Invention

[0003] The purpose of this invention is to provide a method for detecting building cracks based on neural networks, comprising:

[0004] Step S100: Obtain an image of the building cracks and perform illumination compensation on the image;

[0005] Step S200: Extract and fuse multi-scale features of building cracks at different resolutions in the image after illumination compensation.

[0006] Step S300: The crack location is determined by obtaining a crack heat map through feature fusion. The crack location map and the fused feature map are used to focus on the crack region using a spatial attention mechanism to detect the crack width, crack length and orientation.

[0007] Furthermore, in step S100, illumination compensation is performed on the image using an adaptive illumination compensation method, which includes the following steps:

[0008] Step S101: Decompose the input image into illumination component and reflection component;

[0009] Step S102: Apply adaptive Gamma correction to the illumination component;

[0010] Step S103: Convert the corrected illumination component image from BGR color format to HSV color space, and perform CLAHE processing on the V channel in HSV space.

[0011] Step S104: Use nonlocal mean filtering to remove noise.

[0012] Furthermore, the features described in step S200 include high-resolution detail features, medium-resolution structural features, and low-resolution semantic features.

[0013] Furthermore, multi-scale features are obtained through an improved ResNet-50 neural network. The improved method is as follows:

[0014] Three cascaded 3×3 convolutional layers are added after the stem layer and before the residual block, with each convolutional layer followed by a batch normalization layer and a LeakyReLU activation layer;

[0015] The standard bottleneck structure in the residual blocks from conv3_x to conv5_x is replaced with deformable convolutions. Each deformable convolution has a 3×3 kernel and 8 offset channels. Channel attention is introduced in the last residual block. Feature map detail features F3, structural features F4, and semantic features F5 are extracted from conv3_x, conv4_x, and conv5_x, respectively. The semantic feature F5 is upsampled by transposed convolution to obtain feature F5' with the same size as F4.

[0016] Furthermore, in step S200, multi-scale fusion is performed through a cross-layer pyramid structure, and the fusion method includes:

[0017] The detail feature F3 is reduced in dimensionality to obtain feature P3, and feature P3 is refined by depthwise separable convolution to serve as the base layer of the pyramid.

[0018] Structural feature F4 is added element-wise to the upsampled semantic feature F5' to obtain feature P4; feature P4 is introduced and concatenated with feature P3 after a 3×3 convolution, and then concatenated with feature P4 after upsampling, and the outputs of the two paths are fused by convolution.

[0019] The upsampled semantic feature F5' is used to generate feature P5 through convolution, and feature P5' is generated by reweighting feature importance.

[0020] Global average pooling is performed on features P3, P4, and P5' respectively. The three vectors are concatenated and then passed through a fully connected layer to generate attention weights. The weights are normalized by Softmax and then summed for each layer of features.

[0021] Furthermore, the specific process of generating feature P5' using a feature importance reweighting strategy includes:

[0022] Step S2031: Perform global average pooling and global max pooling on feature P5 to obtain two vectors g. avg and g max g avg The average value for each channel, g max The maximum value for each channel;

[0023] Step S2032, g avg and g max Each is fed into the same two-layer bottleneck MLP to obtain two outputs s avg With s max ;

[0024] Step S2033, s avg With s max The mixed score s is obtained by adding elements one by one, and the weight vector w is obtained by s being normalized by Sigmoid.

[0025] Step S2034: Multiply the weight vector w with the feature P5 channel by channel to output P5'.

[0026] Furthermore, step S300 specifically includes:

[0027] Step S301: Extract spatial features from the fused feature map;

[0028] Step S302: Obtain the crack thermal map based on spatial characteristics;

[0029] Step S303: The location of the crack is obtained by suppressing the crack thermal map through non-maximum values;

[0030] Step S304: Using the crack location map as a mask and the fused feature map, a spatial attention mechanism is used to amplify the response of the crack region;

[0031] Step S305: Obtain the crack width, crack length, and direction in the crack area.

[0032] Furthermore, the specific process of step S304 includes:

[0033] Step S3041: Convert the heat map after non-maximum suppression and the peak coordinates of the crack location into a binary crack mask, where the crack pixels are set to 1 and the background pixels are set to 0.

[0034] Step S3042: The fused features are subjected to max pooling and average pooling to obtain the maximum response map and the average response map, respectively.

[0035] Step S3043: Combine the maximum response map, average response map, and crack mask into a three-channel image in the channel dimension;

[0036] Step S3044: Perform convolution processing on the three-channel image;

[0037] Step S3045: The single-channel spatial attention map obtained after convolution and passing through the Sigmoid function;

[0038] Step S3046: Multiply the spatial attention map and the fused feature map pixel by pixel.

[0039] Furthermore, in step S305, the crack width is obtained by calculating the Euclidean distance of the crack edge, the crack length is obtained by extracting the length of the crack centerline, and the direction is obtained by statistical analysis using a direction histogram.

[0040] Compared with the prior art, the present invention has the following advantages: (1) It adopts an improved adaptive illumination compensation algorithm, which solves the problem that traditional visual detection methods (such as edge detection, threshold segmentation, etc.) are insufficient in extracting features of fine cracks in complex backgrounds, and are prone to false detection and missed detection under uneven illumination, surface stains or texture interference, and the detection accuracy is difficult to meet the engineering requirements; (2) It adopts an improved ResNet-50 backbone network, which solves the problem that traditional machine learning schemes rely on manually designed features, have poor generalization ability for crack morphology diversity (such as mesh cracks, diagonal cracks, etc.), and need to be applied to different building materials (concrete, brick walls, etc.). (2) The parameters were readjusted, and the environmental adaptability was insufficient; (3) The cross-layer pyramid structure design scheme (CLFP) was adopted, and a dynamic weight allocation mechanism was designed to handle cracks of different scales. This solved the problem that most existing deep learning methods use general target detection networks, do not consider the linear characteristics and multi-scale features of building cracks, resulting in a large number of model parameters, low computational efficiency, and difficulty in deployment in mobile detection equipment; (4) The detection unit combined with the spatial attention mechanism was used to solve the problem that most methods lack automatic calculation of crack quantification parameters (such as width, length, etc.), and still require manual intervention to complete the damage assessment, resulting in insufficient intelligence.

[0041] The present invention will now be further described with reference to the accompanying drawings. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0043] Combination Figure 1A neural network-based method for detecting building cracks includes:

[0044] Step S100: Obtain an image of the building cracks and perform illumination compensation on the image;

[0045] Step S200: Extract and fuse multi-scale features of building cracks at different resolutions in the image after illumination compensation.

[0046] Step S300: Obtain a crack thermal map by fusing features to determine the crack location, focus on the crack area by crack location, and detect crack width, crack length and direction.

[0047] In step S100, an adaptive illumination compensation algorithm is used to perform illumination compensation on the image, including the following steps:

[0048] Step S101: Decompose the input image into an illumination component and a reflection component; wherein a guided filter is used to smooth the input image I to obtain the illumination component L.

[0049] L = GuidedFilter(I,r,ε),

[0050] Where r is the filter radius and ε is the regularization smoothing parameter; GuidedFilter can effectively preserve the edge details of the image and avoid over-smoothing.

[0051] The Retinex method is used to decompose the reflection component R from the input image I.

[0052] R = I / (L + γ),

[0053] Wherein, γ is the zero-prevention parameter;

[0054] Step S102: Apply adaptive Gamma correction to the illumination component L to obtain the corrected illumination component L'.

[0055] L'=L Γ ,

[0056] Where Γ=1+α*(μ_L-0.5), μ_L is the mean of L, and α is the adjustment factor; through Gamma correction, the dark areas are moderately brightened, while the bright areas remain unchanged;

[0057] Step S103: Convert the corrected illumination component image from BGR color format to HSV color space, and perform CLAHE processing on the V channel in HSV space to enhance the contrast between the crack and the background.

[0058] Step S104: Use nonlocal mean filtering to remove the enhanced noise and maintain the continuity of the crack.

[0059] Step S200 extracts multi-scale features with different resolutions. These scale features, from high to low resolution, include detail features, structural features, and semantic features. Detail features mainly include fine textures, small crack edges, brick joints, and noise information. Structural features mainly include the main body of the crack, its local orientation, and discontinuous connection information. Semantic features mainly include the overall structure, general orientation, and contextual information. In subsequent scale fusion, detail features locate crack edges and detect hairline cracks; structural features identify crack segments and suppress minor interference; and semantic features exclude large areas of non-crack regions.

[0060] This embodiment improves the ResNet-50 neural network to achieve the aforementioned multi-scale feature extraction. The improvements to the ResNet-50 neural network are as follows: A detail-preserving branch (DPB) is added after the stem layer (including the network's initial 7×7 convolutional layer, batch normalization layer, ReLU layer, and 3×3 max pooling layer) and before the residual block. The DPB contains three cascaded 3×3 convolutional layers, each followed by a batch normalization (BN) and LeakyReLU activation layer. The residual block is optimized by replacing the standard bottleneck structure in the residual blocks from conv3_x to conv5_x with deformable convolutions. Each deformable convolution has a 3×3 kernel and 8 offset channels. Channel attention (SE module) is introduced in the last residual block, with a compression ratio of 16. Feature map detail features F3, structural features F4, and semantic features F5 are extracted from conv3_x, conv4_x, and conv5_x, respectively. The feature map sizes for each layer are 1 / 8, 1 / 16, and 1 / 32 of the input image, respectively. Transposed convolution upsampling is performed on F5 to obtain feature map F5' with the same size as F4.

[0061] The improved ResNet-50 neural network exhibits the following advantages in crack detection tasks: (1) The shallow detail-preserving branch improves the network's sensitivity to small cracks by 37%; (2) Deformable convolution improves the detection accuracy of irregular cracks by 22%; (3) The channel attention mechanism reduces the false detection rate of the model in complex backgrounds by 29%; (4) The multi-scale output design lays the foundation for subsequent feature fusion.

[0062] In step S200, multi-scale features are fused using a cross-layer pyramid structure (CLFP). The fusion method includes:

[0063] Step S201, Low-level feature injection: The detail feature F3 is reduced to 256 channels through 1×1 convolution to obtain feature P3. Feature P3 is then refined through 3×3 depthwise separable convolution and used as the base layer of the pyramid.

[0064] Step S202, mid-layer feature fusion: Structural feature F4 is added element-wise with the upsampled semantic feature F5' to obtain feature P4; a bidirectional feature calibration (BFC) mechanism is introduced: in the top-down path, feature P4 is concatenated with feature P3 after a 3×3 convolution; in the bottom-up path, feature P3 is upsampled and concatenated with feature P4; the outputs of the two paths are fused through a 1×1 convolution.

[0065] Step S203, High-level feature refinement: The upsampled semantic feature F5' is processed through two consecutive 3×3 convolutional layers to generate feature P5, and feature importance reweighting (FIR) is used to generate feature P5'. The FIR is as follows:

[0066] P5'=P5⊙σ(Conv1×1([AvgPool(P5),MaxPool(P5)])),

[0067] Where ⊙ represents element-wise multiplication, σ is the Sigmoid function, AvgPool() is the global average pooling operation, and MaxPool() is the max pooling operation.

[0068] Step S204: Introduce cross-layer attention modules (CLA) between each layer of the pyramid. Perform global average pooling on features P3, P4, and P5' respectively to obtain a 1×1×256 vector. Concatenate the three vectors and generate attention weights through a fully connected layer. After the weights are normalized by Softmax, perform a weighted sum on the features of each layer.

[0069] In step S203, the purpose of introducing two pooling methods is twofold: global average pooling captures the "overall response intensity," reflecting the average activation level of the channel across the entire image; global max pooling captures the "most salient response," reflecting the channel's sensitivity to local strong features (such as abrupt changes at crack edges). Average pooling avoids missing weak cracks, while max pooling avoids missing narrow cracks, resulting in more comprehensive information after merging. Specifically...

[0070] Step S2031: Perform global average pooling and global max pooling on feature P5 to obtain two vectors g. avg and g max g avg The average value for each channel, g max The maximum value for each channel;

[0071] Step S2032, put g avg and g max Each is fed into the same two-layer bottleneck MLP to obtain two outputs s avg With s max ;

[0072] Step S2033, s avg With s maxThe mixed score s is obtained by adding each element together. The s is then normalized to 0 to 1 using the Sigmoid function to obtain the final weight vector w.

[0073] In step S2034, the weight vector w is multiplied with the feature P5 channel by channel, and each spatial location is multiplied by the corresponding channel weight to complete the "importance reweighting" output, which is denoted as P5'. The shape is the same as P5, but the crack-related channels are amplified and the irrelevant channels are suppressed.

[0074] Step S300 specifically includes:

[0075] Step S301: Extract spatial features from the fused features using 3×3 convolution + BN + ReLU;

[0076] Step S302: Use 1×1 convolution on the spatial features to output a crack heatmap;

[0077] Step S303: The crack location is obtained by performing non-maximum suppression (NMS) on the heat map;

[0078] Step S304: Spatial attention mechanism (SAM) is used to amplify the response of the crack region on the heat map and fused feature map after NMS.

[0079] The specific process of step S304 includes:

[0080] Step S3041: Convert the heat map after NMS and the peak coordinates of the crack location into a binary crack mask, where the crack pixels are set to 1 and the background pixels are set to 0.

[0081] Step S3042: The fused features are subjected to max pooling and average pooling to obtain the maximum response map and the average response map, respectively.

[0082] Step S3043: Combine the maximum response map, average response map, and crack mask into a three-channel image in the channel dimension. At this point, each pixel location has three types of information: maximum response, average response, and whether it belongs to a crack.

[0083] Step S3044: Feed the three-channel image into a 7×7 convolution, the receptive field of the convolution kernel exactly covers the width of a typical crack, and set the output channel to 1.

[0084] Step S3045: After convolution, the Sigmoid function is used to compress the value of each pixel to between 0 and 1, resulting in a single-channel image. This single-channel image is the spatial attention map. The closer the value is to 1, the more likely it is a crack. The closer it is to 0, the more likely it is the background.

[0085] Step S3046: Multiply the spatial attention map and the fused feature map pixel by pixel. The channel response in the crack region is amplified, while the response in the background region is suppressed.

[0086] The model used in step S300 is obtained through training, and the training model L is...

[0087] L=λ1Lcls+λ2Lwidth+λ3Lori,

[0088] The classification loss Lcls uses Focal Loss (α = 0.25, γ = 2). The width regression loss Lwidth uses smoothed L1 loss. The homing loss Lori uses angle cosine loss. The balance coefficients λ1 = 1.0, λ2 = 0.5, and λ3 = 0.2.

[0089] In step S300, after focusing the crack region, three parameters are output: crack width, length, and orientation. The width is obtained by calculating the Euclidean distance of the crack edge, the length is obtained by extracting the length of the crack centerline, and the orientation is obtained by statistical analysis using a direction histogram.

[0090] Crack width is measured at the sub-pixel level using Canny edge detection and region growing algorithms to improve accuracy. Cracks are categorized by width as follows: hairline cracks (<0.1mm), which are merely fine lines on the building surface and are usually harmless; macroscopic cracks (0.3-2mm), which may affect structural durability and require monitoring; and structural cracks (>2mm), which are usually accompanied by steel reinforcement corrosion and require urgent treatment. Crack orientation is classified as horizontal, vertical, and diagonal. Horizontal and vertical cracks may be caused by uneven loading or shrinkage, while diagonal cracks are commonly seen in shear failure or foundation settlement.

Claims

1. A method for detecting building cracks based on neural networks, characterized in that, include: Step S100: Obtain an image of the building cracks and perform illumination compensation on the image; Step S200: Extract and fuse multi-scale features of building cracks at different resolutions in the image after illumination compensation. Step S300: Obtain a crack heat map by fusing features to determine the crack location; use a spatial attention mechanism to focus on the crack region by combining the crack location map and the fused feature map to detect the crack width, crack length and orientation. In step S200, multi-scale features are obtained through an improved ResNet-50 neural network. The improvement method is as follows: Three cascaded 3×3 convolutional layers are added after the stem layer and before the residual block, with each convolutional layer followed by a batch normalization layer and a LeakyReLU activation layer; The standard bottleneck structure in the residual blocks from conv3_x to conv5_x is replaced with deformable convolutions. Each deformable convolution is set with a 3×3 kernel and 8 offset channels. Channel attention is introduced in the last residual block. Feature map detail features F3, structural features F4, and semantic features F5 are extracted from conv3_x, conv4_x, and conv5_x, respectively. The semantic feature F5 is upsampled by transposed convolution to obtain feature F5' with the same size as F4. In step S200, multi-scale fusion is performed using a cross-layer pyramid structure. The fusion method includes: The detail feature F3 is reduced in dimensionality to obtain feature P3, and feature P3 is refined by depthwise separable convolution to serve as the base layer of the pyramid. Structural feature F4 is added element-wise to the upsampled semantic feature F5' to obtain feature P4; feature P4 is introduced and concatenated with feature P3 after a 3×3 convolution, and then concatenated with feature P4 after upsampling, and the outputs of the two paths are fused by convolution. The upsampled semantic feature F5' is used to generate feature P5 through convolution, and feature P5' is generated by reweighting feature importance. Global average pooling is performed on features P3, P4, and P5' respectively. The three vectors are concatenated and then passed through a fully connected layer to generate attention weights. The weights are normalized by Softmax and then weighted and summed on the features of each layer. The specific process of generating feature P5' using the feature importance reweighting strategy includes: Step S2031: Perform global average pooling and global max pooling on feature P5 to obtain two vectors. g avg and g max , g avg The average value for each channel, g max The maximum value for each channel; Step S2032, will g avg and g max Each input is fed into the same two-layer bottleneck MLP, resulting in two outputs. s avg and s max ; Step S2033, will s avg and s max The mixed fraction is obtained by adding elements one by one. s ,right s Perform sigmoid normalization to obtain the weight vector w; Step S2034: Multiply the weight vector w with the feature P5 channel by channel to output P5'; Step S300 specifically includes: Step S301: Extract spatial features from the fused feature map; Step S302: Obtain the crack thermal map based on spatial characteristics; Step S303: The location of the crack is obtained by suppressing the crack thermal map through non-maximum values; Step S304: Using the crack location map as a mask and the fused feature map, a spatial attention mechanism is used to amplify the response of the crack region; Step S305: Obtain the crack width, crack length, and orientation in the crack area; The specific process of step S304 includes: Step S3041: Convert the heat map after non-maximum suppression and the peak coordinates of the crack location into a binary crack mask, where the crack pixels are set to 1 and the background pixels are set to 0. Step S3042: The fused features are subjected to max pooling and average pooling to obtain the maximum response map and the average response map, respectively. Step S3043: Combine the maximum response map, average response map, and crack mask into a three-channel image in the channel dimension; Step S3044: Perform convolution processing on the three-channel image; Step S3045: The single-channel spatial attention map obtained after convolution and passing through the Sigmoid function; Step S3046: Multiply the spatial attention map and the fused feature map pixel by pixel.

2. The method according to claim 1, characterized in that, In step S100, the image is illuminated using an adaptive illumination compensation method, which includes the following steps: Step S101: Decompose the input image into illumination component and reflection component; Step S102: Apply adaptive Gamma correction to the illumination component; Step S103: Convert the corrected illumination component image from BGR color format to HSV color space, and perform CLAHE processing on the V channel in HSV space. Step S104: Use nonlocal mean filtering to remove noise.

3. The method according to claim 1, characterized in that, The features described in step S200 include high-resolution detail features, medium-resolution structural features, and low-resolution semantic features.

4. The method according to claim 1, characterized in that, In step S305, the crack width is obtained by calculating the Euclidean distance at the crack edge, the crack length is obtained by extracting the length of the crack centerline, and the direction is obtained by statistical analysis using a direction histogram.

Images