A deep learning-based concrete structure crack detection method

Abstract

This invention discloses a deep learning-based method for detecting cracks in concrete structures, comprising: acquiring images of the concrete structure surface; inputting the image to be detected into a preset lightweight image classification model ResNet_GL to determine whether cracks exist in the image; inputting the image containing cracks into a preset lightweight object detection model YOLOv8n_EB to obtain the bounding box location information of the crack region; extracting the crack region image based on the bounding box location information and inputting it into a preset semantic segmentation model U-Net_EC to obtain the pixel-level contour and morphological information of the crack; the deep learning-based concrete structure crack detection method constructed in this invention achieves high-precision, fully automated crack detection with low computational cost through the cascading of three lightweight models, solving the problems of low efficiency and poor robustness of traditional methods and the difficulty of balancing accuracy and speed in existing deep learning methods, and has extremely high engineering application value.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and specifically to a method for detecting cracks in concrete structures based on deep learning. Background Technology

[0002] Concrete structures are widely used in infrastructure such as buildings, bridges, and highways. During long-term service, they are prone to cracking due to factors such as temperature, stress, and chemical corrosion. Cracks not only affect aesthetics but can also lead to water seepage, steel reinforcement corrosion, and protective layer peeling. Existing crack detection methods mainly include manual inspection, traditional digital image processing inspection, and deep learning inspection. Manual inspection relies on the experience of inspectors, resulting in low efficiency, strong subjectivity, and high risks associated with working at heights. Traditional digital image processing methods, such as threshold segmentation and edge detection (Canny operator, etc.), while reducing the workload of manual inspection, are extremely sensitive to factors such as changes in lighting, shadows, and background texture interference, exhibiting insufficient robustness and generalization ability, making it difficult to adapt to complex and changing engineering site environments. In recent years, deep learning inspection methods have emerged, automatically extracting crack features through convolutional neural networks, significantly improving both accuracy and robustness. Current research mainly focuses on improving models for single tasks, such as crack classification, object detection, or semantic segmentation. However, in practical engineering applications, a complete inspection process often requires the coordinated operation of multiple steps, including "macroscopic screening → region localization → fine measurement." A single model cannot simultaneously meet the requirements of rapid screening, accurate localization, and pixel-level segmentation. In addition, existing models generally suffer from large numbers of parameters and high computational complexity, making it difficult to achieve real-time deployment on edge devices with limited computing resources, such as drones and mobile terminals. Summary of the Invention

[0003] Purpose of the invention: The purpose of this invention is to provide a deep learning-based method for detecting cracks in concrete structures. The crack detection task is decomposed into three collaborative stages: image classification, object detection, and semantic segmentation. For the characteristics and deployment requirements of each stage, a dedicated lightweight improved model is designed and trained to form an end-to-end cascaded detection process.

[0004] Technical solution: The present invention provides a method for detecting cracks in concrete structures based on deep learning, comprising the following steps:

[0005] Step 1: Collect images of the concrete structure surface and construct a dataset containing three types of annotations: image classification, object detection, and semantic segmentation;

[0006] Step 2: Input the image to be detected into the preset lightweight image classification model ResNet_GL, and determine whether there are cracks in the image. If there are, proceed to step 3. The ResNet_GL model is constructed by compressing the depth of the ResNet18 network and introducing the lightweight convolutional structure LSConv.

[0007] Step 3: Input the image containing the crack into the preset lightweight object detection model YOLOv8n_EB to obtain the bounding box location information of the crack region; wherein, the YOLOv8n_EB model is constructed by introducing the EB-Block module, EMA attention mechanism and BiFPN feature fusion structure into YOLOv8n;

[0008] Step 4: Extract the crack region image based on the bounding box location information and input it into the preset semantic segmentation model U-Net_EC to obtain the pixel-level contour and morphological information of the crack; the U-Net_EC model is constructed by adding a CAA attention mechanism to the U-Net encoder, introducing a learnable upsampling structure EUCB in the decoder, and jointly optimizing the loss function.

[0009] Furthermore, in step 2, the lightweight image classification model ResNet_GL is specifically implemented as follows: Using ResNet18 as the baseline model, a Global Context attention mechanism is added after its last residual structure to enhance the model's attention to crack regions; one Basic Residual Block is removed from each of the third and fourth layers of the baseline model to construct a deeply compressed ResNet14 model, reducing the number of parameters and computational cost; a lightweight convolutional structure LSConv is designed to replace all standard convolutional modules in ResNet14; the LSConv structure is composed of grouped convolutions, a compression and activation SE attention mechanism, residual connections, batch normalization layers, and the SiLU activation function combined sequentially.

[0010] Furthermore, in step 3, the lightweight object detection model YOLOv8n_EB is specifically as follows:

[0011] An EB-Block structure was designed to replace the bottleneck structure of the C2f module in the YOLOv8n backbone network. The EB-Block structure first uses 1×1 convolutions for channel compression, followed by 5×5 depthwise convolutions to extract spatial features. Then, global and local features are extracted through parallel average pooling and max pooling branches, respectively. Finally, the results of the two branches are fused and added to the input through residual connections. An efficient multi-scale attention (EMA) mechanism is added to the YOLOv8n neck network. Through channel grouping, directional pooling, and multi-branch structure, the model's attention to small cracks is enhanced. The original FPN+PAN feature fusion structure of YOLOv8n is replaced with a weighted bidirectional feature pyramid network BiFPN. Through bidirectional cross-scale connections and learnable weights, the multi-scale feature fusion effect is optimized.

[0012] Furthermore, in step 4, the semantic segmentation model U-Net_EC is implemented as follows: A channel affinity attention (CAA) mechanism is added to the end of the U-Net encoder. The CAA mechanism calculates the similarity between channels and generates attention weights through a compact channel comparator (CCC) and a channel affinity estimator (CAE), which is used to enhance the semantic expressive power of shallow features and reduce their semantic deviation from deep features. The bilinear interpolation upsampling structure in the U-Net decoder is replaced with a learnable upsampling structure (EUCB). The EUCB structure first performs a 2x upsampling on the feature map, and then adjusts the number of channels sequentially through a 3×3 depthwise separable convolution, a batch normalization layer, a ReLU activation function, and a 1×1 convolution. The cross-entropy loss function and the Dice loss function are used together as the total loss function of the U-Net_EC model to alleviate the problem of imbalance between cracks and background pixels.

[0013] Further, step 1 is as follows: The original concrete structure crack image is subjected to image cropping, image cleaning, image denoising based on bilateral filtering, and normalization preprocessing in sequence;

[0014] Data augmentation is performed on the preprocessed images, including random rotation, flipping, brightness enhancement, and adding Gaussian noise to increase sample diversity. For image classification tasks, images are divided into two categories: cracks and non-cracks. For object detection tasks, the LabelImg tool is used to annotate crack regions with rectangular boxes and generate YOLO format label files. For semantic segmentation tasks, the LabelMe tool is used to annotate crack regions with polygonal pixel-level labels and generate binary mask images.

[0015] Furthermore, the ResNet_GL model, YOLOv8n_EB model, and U-Net_EC model constitute a cascaded detection process, enabling collaborative work from rapid screening and region localization to fine segmentation.

[0016] The present invention discloses a deep learning-based concrete structure crack detection system for implementing any of the methods described herein, comprising: an image acquisition module, a data processing and storage module, a cascaded detection module, and a result output module.

[0017] An electronic device according to the present invention includes a processor and a memory, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements any of the methods described herein.

[0018] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements any of the methods described herein.

[0019] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: This invention constructs a classification-detection-segmentation linkage detection system, realizing automated and intelligent detection from coarse screening to fine measurement, which is more practically valuable in engineering than single-task methods. Through targeted lightweighting and structural optimization design of the three-stage models, the entire model significantly reduces the computational burden while ensuring high detection accuracy, making it suitable for deployment on mobile edge devices such as drones and inspection robots. By introducing attention mechanisms (Global Context, EMA, CAA) with different working mechanisms into multiple models, the system exhibits extremely high robustness to interference from lighting changes, shadows, stains, and complex background textures, especially excelling in the identification of small cracks and the restoration of blurred boundaries. The design of the learnable upsampling module EUCB and the combined loss function in the U-Net_EC model effectively solves the problems of blurred and discontinuous crack edges in traditional segmentation methods, laying a solid foundation for the accurate measurement of subsequent crack length, width, and other parameters. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0022] like Figure 1 As shown in the figure, this embodiment of the invention provides a method for detecting cracks in concrete structures based on deep learning, including the following steps:

[0023] Step 1: Preparation of the multi-task dataset: Acquiring images of cracks in concrete structures. The open-source dataset CCIC used in this embodiment contains 20,000 crack images and 20,000 non-crack images. The self-made dataset contains 540 crack images taken with a smartphone in scenes such as buildings, roads, and bridges on campus.

[0024] Preprocess all the above images: (1) According to the different model input requirements, the images are uniformly cropped to 224×224 or 640×640 pixels; (2) The Laplacian variance method is used to remove blurry images, and the perceptual hash and Euclidean distance algorithms are used to remove duplicate images; (3) The bilateral filtering algorithm is used to remove image noise while retaining crack edge details; (4) The Min-Max normalization method is used to linearly map all pixel values ​​to the range [0-1].

[0025] Data augmentation was performed on the preprocessed images to increase sample diversity. Specific operations included random rotation (0-360°), horizontal and vertical flipping, brightness enhancement (simulating different lighting conditions), and adding Gaussian noise with an intensity of σ=15 (simulating electronic interference). Finally, three datasets were constructed according to task requirements: Classification dataset: 15,000 CCIC images and 2,500 self-made images were selected, divided into crack (positive) and non-crack (negative) categories, and split into training and validation sets in an 8:2 ratio. Object detection dataset: 4,000 images were selected from the 6,250 augmented images. The crack regions were labeled with rectangular boxes using the LabelImg tool, and the category was uniformly named "crack". YOLO-formatted normalized coordinate labels were generated, and the training and validation sets were split in an 8:2 ratio. Semantic segmentation dataset: 2,250 images were selected from the 6,250 augmented images. The crack regions were labeled with polygonal vertices using the LabelMe tool and converted into binary mask images, and the training and validation sets were split in a 9:1 ratio.

[0026] Step 2: Train the lightweight classification model ResNet_GL. A Global Context attention mechanism module is inserted at the output of the last residual block in Layer 4 of the baseline ResNet18 network to capture the global visual context. One Basic Block is removed from each of Layers 3 and 4, reducing the network to a lightweight 14-layer skeleton. The standard 3×3 convolutions in all residual blocks are replaced with a custom LSConv module. The internal computation flow of LSConv is: input features → grouped convolutions with 4 groups → SE channel attention weighting → residual connections added to input features → batch normalization layer → SiLU activation function → output features. The model is trained for 50 epochs using the classification training set and cross-entropy loss function, using the Adam optimizer, a learning rate of 1e-4, and a batch size of 32. After training, the final model ResNet_GL is obtained, with 1.66M parameters and 0.4 GFLOPs computation.

[0027] The specific process is as follows: Global context modeling is integrated. A Global Context attention mechanism is added after the last residual stage of ResNet18. This mechanism can overcome the limitations of the local receptive field of the convolutional kernel, establish the dependency relationship between global spatial location and channels, integrate discrete crack features into global information, and significantly improve the model's overall perception ability of slender cracks with varying shapes. Analysis shows that the crack classification task has a relatively low dependence on deep semantics. This invention prunes the network structure, removing one basic residual block each from the third and fourth layers of the original ResNet18, constructing a lighter ResNet14 skeleton, fundamentally reducing parameter redundancy and overfitting risks. To further reduce the number of parameters and computational cost, this invention designs a novel lightweight convolutional structure, LSConv, to replace all standard convolutions in the network. The core of LSConv's structure lies in: first, using grouped convolution to reduce parameter connections, then using a concatenated SE channel attention mechanism to recalibrate the channel dimensions of the feature maps after grouped convolution, thereby enhancing key crack features and suppressing redundant background noise; finally, using residual connections, batch normalization (BN), and the SiLU activation function to ensure smooth information flow and stable training.

[0028] Step 3: Train the lightweight detection model YOLOv8n_EB. In the C2f module of the backbone network, the original Bottleneck structure is completely replaced with the designed EB-Block. In the neck feature fusion network, an EMA attention mechanism module is inserted before or after each C2f module to enhance the feature response to small cracks. The entire feature fusion part (Neck) is reconstructed from the original FPN+PAN structure to a BiFPN structure with learnable weights. The model is trained for 200 epochs using the object detection training set and YOLOv8's composite loss function (including bounding box loss, classification loss, and DFL loss), using the SGD optimizer with a learning rate of 0.01, momentum of 0.937, and a batch size of 32. After training, the final model YOLOv8n_EB is obtained, with 1.57M parameters and 5.7 GFLOPs of computation.

[0029] The specific process is as follows: In response to the problem that the Bottleneck structure of the C2f module in the YOLOv8n backbone network has a large number of parameters and limited multi-scale feature extraction capability, this invention designs EB-Block to replace it.

[0030] The input features are first compressed through 1×1 convolutions to reduce computational burden. Then, 5×5 depthwise convolutions are used to extract local spatial details of the cracks channel by channel. Next, the features are split into two parallel branches: one performs average pooling to capture the global context, and the other performs max pooling to highlight the salient edge features of the cracks. Finally, the features from the two branches are fused and added to the original input via residual connections. This "compression-depth extraction-multi-pooling-fusion" structure enables the simultaneous capture of crack thickness variations and global orientation with extremely low parameter counts, significantly improving the feature representation capability for multi-scale cracks.

[0031] To address the challenge of detecting subtle cracks in complex backgrounds, an EMA attention mechanism is embedded in the neck network (feature fusion part) of the model. The EMA module first groups the feature maps along the channel dimension to enhance feature diversity. Then, it performs one-dimensional pooling encoding on each group of features along the height and width directions to obtain orientation-aware spatial feature maps, generating spatial attention weights through convolution and the sigmoid function. Simultaneously, another branch performs 3×3 convolution on the spatially enhanced features to generate channel attention weights. By jointly modeling in both spatial and channel dimensions, the model can better focus on subtle, discontinuous crack features, effectively suppressing interference from complex textured backgrounds and reducing false positives and false negatives. The FPN+PAN structure used for feature fusion in the original YOLOv8n is replaced with a more efficient BiFPN structure. BiFPN constructs denser bidirectional cross-scale connection paths and introduces a trainable scalar weight at each feature fusion node, enabling the network to adaptively learn the importance of input feature maps at different scales. This design allows for a more complete and efficient fusion of high-level semantic information and low-level detailed information, greatly improving the model's accuracy in locating crack targets with significant scale differences and the completeness of bounding box regression.

[0032] Step 4: Train the semantic segmentation model U-Net_EC. Add a CAA attention mechanism to the final output of the U-Net encoder (i.e., the bottom of the U-shaped structure) to recalibrate channel features based on similarity. Replace all upsampling operations in the decoder (originally bilinear interpolation) with EUCB learnable upsampling modules. Define a joint loss function to calculate the error between the predicted mask and the ground truth mask. Train the model for 100 epochs using the semantic segmentation training set and the aforementioned joint loss function, using the SGD optimizer with a learning rate of 1e-4, momentum of 0.9, and batch size of 2. After training, the final model U-Net_EC is obtained.

[0033] The specific process is as follows: A CAA (Channel Affinity Attention) mechanism is introduced at the end of the U-Net encoder. By calculating the similarity matrix between all channels, the intrinsic connections between channels are mined, and attention weights are generated accordingly to reweight the original features. Applying CAA at the end of the encoder enhances the semantic information content of the shallow features after downsampling, resulting in a smaller semantic gap when they are spliced ​​and fused with the deep features of the decoder. The fused features are more coordinated, effectively improving the recognition of crack targets during subsequent upsampling. Learnable EUCB upsampling is adopted. The traditional fixed bilinear interpolation upsampling in the U-Net decoder is completely replaced with a learnable EUCB upsampling module. After performing a 2x upsampling to expand the feature map size, the EUCB module immediately processes the data through a convolutional block consisting of a 3×3 depthwise separable convolution, a batch normalization (BN) layer, and a ReLU activation function, and finally adjusts the number of channels with a 1×1 convolution. This combination of "upsampling + feature refinement" gives the model the ability to dynamically adjust reconstruction parameters based on current semantic and detailed information when reconstructing high-resolution feature maps. Compared to rigid bilinear interpolation, EUCB can recover the sharp edges and fine tips of cracks to a great extent, reducing the blurring and breakage of the segmentation results.

[0034] The cross-entropy loss function and the Dice loss function are used together as the total loss for model training. The cross-entropy loss function is responsible for classifying each pixel independently, ensuring the stability of the overall segmentation; while the Dice loss function directly optimizes the overlap between the predicted crack region and the real crack region, which is highly effective in addressing the class imbalance problem caused by the small proportion of crack regions. The weighted combination of the two guides the model to focus on both the accuracy of overall pixel classification and the morphology and integrity of the crack region itself during training, ultimately achieving a segmentation effect with high recall and accurate edges.

[0035] Dice Loss is more robust in semantic segmentation tasks with class imbalance and small target regions.

[0036] The Dice coefficient measures the degree of overlap between the model's predicted region and the actual region. The calculation formula is shown below:

[0037]

[0038] in, This represents the number of pixels that the model predicts to be cracks. This represents the number of pixels whose actual label is a crack. This indicates the number of pixels where both the predicted result and the actual label show a crack.

[0039] The Dice Loss function constructed based on the Dice coefficients is shown below:

[0040]

[0041] in, Indicates the first The probability that a pixel is predicted to be a crack. This represents the actual label of the pixel. It is a very small constant used to avoid cases where the denominator is 0.

[0042]

[0043] in, This indicates the total number of intermediate output branches. Indicates the first The weighting coefficients corresponding to each branch and These represent the Dice loss and cross-entropy loss for this branch, respectively.

[0044] Step 5: Cascaded System Deployment and Inference. Deploy the three trained models in series. Input a concrete image to be detected, and the system will automatically and sequentially complete the following steps: determining whether cracks are present (ResNet_GL), locating the cracks (YOLOv8n_EB), and extracting the specific shape and size of the cracks at the pixel level (U-Net_EC). Finally, the system will overlay and display the classification labels, bounding boxes, and crack mask image, and output the results.

[0045] In summary, the deep learning-based concrete structure crack detection method constructed in this invention achieves high-precision, fully automated crack detection with low computational cost through the cascading of three lightweight models. It solves the problems of low efficiency and poor robustness of traditional methods and the difficulty of balancing accuracy and speed in existing deep learning methods, and has extremely high engineering application value.

Claims

1. A method for detecting cracks in concrete structures based on deep learning, characterized in that, Includes the following steps: Step 1: Collect images of the concrete structure surface and construct a dataset containing three types of annotations: image classification, object detection, and semantic segmentation; Step 2: Input the image to be detected into the preset lightweight image classification model ResNet_GL, and determine whether there are cracks in the image. If there are, proceed to step 3. The ResNet_GL model is constructed by compressing the depth of the ResNet18 network and introducing the lightweight convolutional structure LSConv. Step 3: Input the image containing the crack into the preset lightweight object detection model YOLOv8n_EB to obtain the bounding box location information of the crack region; wherein, the YOLOv8n_EB model is constructed by introducing the EB-Block module, EMA attention mechanism and BiFPN feature fusion structure into YOLOv8n; Step 4: Extract the crack region image based on the bounding box location information and input it into the preset semantic segmentation model U-Net_EC to obtain the pixel-level contour and morphological information of the crack; the U-Net_EC model is constructed by adding a CAA attention mechanism to the U-Net encoder, introducing a learnable upsampling structure EUCB in the decoder, and jointly optimizing the loss function.

2. The method for detecting cracks in concrete structures based on deep learning according to claim 1, characterized in that, In step 2, the lightweight image classification model ResNet_GL is specifically implemented as follows: using ResNet18 as the baseline model, a global context attention mechanism is added after its last residual structure to enhance the model's attention to crack regions; One basic residual block each from the third and fourth layers of the baseline model is removed to construct a deeply compressed ResNet14 model to reduce the number of parameters and computational cost. A lightweight convolutional structure LSConv is designed to replace all standard convolutional modules in ResNet14. The LSConv structure is composed of grouped convolutions, compression and activation SE attention mechanisms, residual connections, batch normalization layers, and SiLU activation functions combined in sequence.

3. The method for detecting cracks in concrete structures based on deep learning according to claim 1, characterized in that, In step 3, the lightweight object detection model YOLOv8n_EB is specifically as follows: The EB-Block structure is designed to replace the bottleneck structure of the C2f module in the YOLOv8n backbone network. The EB-Block structure first uses 1×1 convolution for channel compression, followed by 5×5 depth convolution to extract spatial features. Then, it extracts global and local features through parallel average pooling and max pooling branches respectively. Finally, the results of the two branches are fused and added to the input through residual connection. An efficient multi-scale attention (EMA) mechanism is added to the YOLOv8n neck network. Through channel grouping, directional pooling, and multi-branch structure, the model’s attention to small cracks is enhanced. The original FPN+PAN feature fusion structure of YOLOv8n is replaced with a weighted bidirectional feature pyramid network BiFPN. Through bidirectional cross-scale connections and learnable weights, the multi-scale feature fusion effect is optimized.

4. The method for detecting cracks in concrete structures based on deep learning according to claim 1, characterized in that, In step 4, the semantic segmentation model U-Net_EC is implemented as follows: A channel affinity attention (CAA) mechanism is added to the end of the U-Net encoder. The CAA mechanism calculates the similarity between channels and generates attention weights through a compact channel comparator (CCC) and a channel affinity estimator (CAE), which is used to enhance the semantic expressive power of shallow features and reduce their semantic deviation from deep features. The bilinear interpolation upsampling structure in the U-Net decoder is replaced with a learnable upsampling structure (EUCB). The EUCB structure first performs a 2x upsampling on the feature map, and then adjusts the number of channels sequentially through a 3×3 depthwise separable convolution, a batch normalization layer, a ReLU activation function, and a 1×1 convolution. The cross-entropy loss function and the Dice loss function are used together as the total loss function of the U-Net_EC model to alleviate the problem of imbalance between cracks and background pixels.

5. The method for detecting cracks in concrete structures based on deep learning according to claim 1, characterized in that, Step 1 is as follows: The original concrete structure crack image is processed sequentially by image cropping, image cleaning, image denoising based on bilateral filtering, and normalization preprocessing. Data augmentation is performed on the preprocessed images, including random rotation, flipping, brightness enhancement, and adding Gaussian noise to increase sample diversity. For image classification tasks, images are divided into two categories: cracks and non-cracks. For object detection tasks, the LabelImg tool is used to annotate crack regions with rectangular boxes and generate YOLO format label files. For semantic segmentation tasks, the LabelMe tool is used to annotate crack regions with polygonal pixel-level labels and generate binary mask images.

6. The method for detecting cracks in concrete structures based on deep learning according to claim 1, characterized in that, The ResNet_GL model, YOLOv8n_EB model, and U-Net_EC model constitute a cascaded detection process, enabling collaborative work from rapid screening and region localization to fine segmentation.

7. A deep learning-based concrete structure crack detection system, characterized in that, The method for implementing any one of claims 1-6 includes: an image acquisition module, a data processing and storage module, a cascade detection module, and a result output module.

8. An electronic device comprising a processor and a memory, characterized in that, The memory stores a computer program, which, when executed by the processor, implements the method described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

Images