A lightweight UNet crack image segmentation method incorporating CBAM

Abstract

This invention discloses a lightweight UNet crack segmentation method integrating CBAM, comprising: preprocessing the crack image dataset to generate the dataset required for model training; using the original UNet as the backbone network and optimizing the model based on it; modifying the encoder in the model and adding spatial attention and channel attention for training; inputting the preprocessed crack image dataset into the model to train the final model weights; replacing the traditional convolutions of the original UNet encoder with separable convolutions, and replacing large convolutions with small convolutions in the attention layer, thereby reducing the number of model parameters. Since the method of reducing the number of parameters in this invention does not change the original receptive field, it will not affect the model training. This invention optimizes the encoder and network structure on the original segmentation model, which can improve the segmentation effect and reduce the number of model parameters on the original basis, which is beneficial for the deployment of the model in practical engineering.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and architectural engineering, and in particular to a lightweight UNet crack image segmentation method that integrates CBAM. Background Technology

[0002] my country's infrastructure construction projects have developed rapidly, and the increased volume of projects has also brought about an increase in engineering safety risks. Numerous engineering accidents have been caused by cracks in the structures. These accidents occur frequently, resulting in casualties and economic losses. In practice, if defects in engineering structures can be monitored in advance, potential disasters can be warned, thus avoiding unnecessary losses. Therefore, the development of a rapid and effective crack segmentation method is extremely urgent.

[0003] Current research methods provide basic predictions of cracks in engineering projects. Most safety inspections rely on manual, static methods, with dynamic inspections serving only as a supplementary monitoring method in practice. The traditional approach involves inspectors visually observing the shape and trend of cracks and measuring relevant characteristic values ​​using instruments. The hazard factor is then determined based on the inspector's experience. This method is slow, highly subjective, and lacks objective judgment. While deep learning methods are also being researched, the complexity of the model significantly impacts its practical deployment, requiring substantial computational resources that some embedded devices often lack. Therefore, given these two technological approaches, finding an easily deployable and high-performance crack segmentation model is crucial. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of existing technologies and provide a crack segmentation method that balances model complexity and model accuracy. This invention can be trained on a lightweight model using a small training set. This reduces the number of model parameters without a significant decrease in accuracy, making it more suitable for deployment and application in practical engineering verification. It also does not require sophisticated testing equipment.

[0005] The present invention solves the above problems by adopting the following technical solution:

[0006] A lightweight UNet crack image segmentation method incorporating CBAM includes the following steps:

[0007] Step 1: Preprocess the collected crack image data as sample data;

[0008] Step 2: Establish an improved UNet network model, using the sample data preprocessed in Step 1 as input;

[0009] Step 3: Obtain the optimized CBAM module;

[0010] Step 4: During the downsampling process of the improved UNet network model, an optimized CBAM module is added after each encoder, so that the output of each encoder enters the optimized CBAM module for weight allocation to obtain the feature map.

[0011] Step 5: Perform four upsampling operations on the last feature map obtained in Step 4. In each upsampling process, it is necessary to fuse the feature map with the previously optimized CBAM module weight allocation of each layer. The feature map after feature fusion is input into the decoder of the corresponding layer.

[0012] Step 6: Perform two convolution operations on the feature map after feature fusion at each layer, and adjust the number of channels to generate a weight file;

[0013] Step 7: Load the weight file obtained in Step 6 into the improved UNet network model for prediction, and use the improved UNet network model to segment and detect cracks in the image.

[0014] Furthermore, step 1 includes:

[0015] Step 1.1: Perform image smoothing on each sample data to reduce noise interference;

[0016] Step 1.2: Compress the number of image channels for each sample data to convert the sample data into single-channel images, thereby reducing the dimension of the matrix and reducing the amount of computation involved in the model to some extent.

[0017] Furthermore, the image smoothing process in step 1.1 is specifically a Gaussian weighted average.

[0018] Furthermore, step 1.2, which compresses the number of image channels, specifically involves processing the image as a grayscale image.

[0019] Furthermore, the improved UNet network model in step 2 involves replacing the encoder in the original UNet network with depthwise separable convolutions from Mobile-Net. The replacement process involves setting the output channel count of the first convolution of the encoder in the original UNet network to equal its input channel count, and setting the `groups` parameter of the first convolution to the input channel count as well. In the second convolution, the input channel count is equal to the output channel count of the first convolution, and the output channel count of the second convolution is equal to the final output channel count of the replaced traditional convolution. Finally, the `kernel_size` of the second convolution is set to 1. The operation in step 2 is the main way this invention reduces the number of parameters by replacing the original traditional convolution with two convolutions.

[0020] Furthermore, step 3 specifically involves replacing the 7×7 convolution in the original CBAM module of ResNet50 with three 3×3 small convolution kernels. After the replacement, the size of the output feature map remains unchanged. On the one hand, the number of parameters is reduced, and on the other hand, the receptive field is not reduced, thereby obtaining an optimized CBAM module.

[0021] Furthermore, in step 4, the feature maps extracted by the encoder at each downsampling stage are fed into the optimized CBAM module, which performs weight allocation. The feature maps that have learned the weights are then used in each subsequent downsampling operation, allowing the network to focus more on the crack region and thus suppress irrelevant regions.

[0022] Furthermore, step 6, which adjusts the number of channels in the final output image of the model, depends on the number of categories. For the previous convolution of the final output layer, a 1×1 convolution is used to adjust the number of channels to 1. Using a 1×1 convolution does not damage the features and allows for adjustment of the output channel number.

[0023] Beneficial effects:

[0024] (1) This invention provides a lightweight UNet crack segmentation method that integrates CBAM, which provides a new technical route for crack segmentation and detection. It retains the advantages of previous models and introduces new ideas, which is of great significance for deploying related models in engineering.

[0025] (2) The UNet structure proposed in this invention redesigns the encoder of the original UNet. The redesign of the structure results in a lightweight model, which has important engineering significance for the later actual engineering deployment.

[0026] (3) The final result of this invention is to achieve segmentation and detection of crack images. If other experimental factors are considered, such as the quality and size of the dataset, and the selection of the optimizer during model training, the model training effect can be improved. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0028] Figure 2 This is a schematic diagram of depthwise separable convolution.

[0029] Figure 3 This is a schematic diagram of the CBAM model structure.

[0030] Figure 4 This is a schematic diagram of the final segmentation model structure. Detailed Implementation

[0031] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0032] This invention optimizes the original UNet structure by replacing all convolutions in the original encoder with separable convolutions, and then appending CBAM to the encoder for weight allocation, thereby improving feature extraction for learning. This benefits both model training and final prediction.

[0033] See appendix Figure 1 The proposed lightweight UNet crack segmentation method, which integrates CBAM, includes the following steps:

[0034] Step 1: First, the crack image dataset needs to be preprocessed. The weights generated from the preprocessed dataset through model training are the core of the final model.

[0035] Step 2, see appendix Figure 2 This paper replaces the encoder in the original UNet network with depthwise separable convolutions from Mobile-Net. The process of replacing traditional convolutions with separable convolutions to reduce the number of parameters is accomplished through combination operations. First, single-channel convolution kernels are used to convolve the input feature map; the number of kernels must match the number of input channels. Then, the number of channels in subsequent convolution kernels must match the number of output channels from the previous convolution. Finally, the number of feature maps depends on the number of kernels used in the second convolution.

[0036] Step 3, see appendix Figure 3 The original CBAM module's 7×7 convolution is replaced with three 3×3 small convolution kernels, which reduces the number of parameters without reducing the receptive field.

[0037] In spatial attention and channel attention, the original large 7×7 convolution is replaced with a small 3×3 convolution, thereby further reducing the number of model parameters without affecting the receptive field.

[0038] Step 4, see appendix Figure 4 During downsampling, an optimized CBAM module is added after each encoder. The output of each encoder is fed into the CBAM for weight allocation. Attention weights are assigned to the features extracted by each encoder layer to achieve better feature extraction results.

[0039] Step 5, see appendix Figure 4 The resulting feature map undergoes a four-step upsampling operation. During each upsampling process, the feature map is fused with the output feature map from each previous CBAM layer. This fusion of the features from the encoding stage and the upsampled features helps recover some details lost during downsampling.

[0040] Step 6, see appendix Figure 4 For each fused feature map, two convolution operations are performed to adjust the number of channels. Before the final output layer, a 1×1 convolution is used to adjust the number of channels. For segmentation tasks, the final segmentation result is represented by cracks and background. Adjusting the number of channels in the final output convolution to 1 is sufficient.

[0041] Step 7: Load the trained weight file into the model for detection. The weight file is loaded into the model to predict crack images and improve detection performance.

[0042] This invention discloses a lightweight UNet crack segmentation method integrating CBAM, addressing a problem in image segmentation. First, the crack image dataset is preprocessed to generate the dataset required for model training. Then, the original UNet is used as the backbone network, and the model is optimized based on this. Modifications are made to the encoder, and spatial attention and channel attention are added for training. The preprocessed crack image dataset is input into the model to train the final model weights. Replacing the traditional convolutions of the original UNet encoder with separable convolutions, and replacing large convolutions with smaller ones in the attention layer, significantly reduces the number of model parameters. Since the parameter reduction method of this invention does not change the original receptive field, it does not affect model training. Attention is used to further extract features during the encoding stage, and these features are fused with upsampled features during the later decoding stage to improve model performance. This invention optimizes the encoder and network structure on the original segmentation model, improving segmentation performance while reducing the number of model parameters, which is beneficial for the deployment of the model in practical engineering.

[0043] The above description, in conjunction with specific simulated embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, substitutions and modifications can be made without departing from the concept of the present invention, and all such modifications and modifications should be considered within the scope of protection of the present invention.

Claims

1. A lightweight UNet crack image segmentation method integrating CBAM, characterized in that, Includes the following steps: Step 1: Preprocess the collected crack image data as sample data; Step 2: Establish an improved UNet network model, using the sample data preprocessed in Step 1 as input; Step 3: Obtain the optimized CBAM module; Step 4: During the downsampling process of the improved UNet network model, an optimized CBAM module is added after each encoder, so that the output of each encoder enters the optimized CBAM module for weight allocation to obtain the feature map. Step 5: Upsample the feature map obtained in Step 4. In each upsampling process, it is necessary to fuse the feature map with the previously optimized CBAM module weight allocation of each layer. The feature map after feature fusion is input into the decoder of the corresponding layer. Step 6: Perform two convolution operations on the feature map after feature fusion at each layer, and adjust the number of channels to generate a weight file; Step 7: Load the weight file obtained in Step 6 into the improved UNet network model for prediction, and use the improved UNet network model to segment and detect cracks in the image. The improved UNet network model in step 2 is to replace the encoder in the original UNet network with depthwise separable convolutions from Mobile-Net. The replacement process is to set the number of output channels of the first convolution of the encoder in the original UNet network to be equal to the number of input channels, and set the groups parameter of the first convolution to be the number of input channels as well. In the second convolution, the number of input channels is the number of output channels of the first convolution, and the number of output channels of the second convolution is the number of the last output channels of the replaced traditional convolution. Step 3 specifically involves replacing the 7×7 convolution in the original CBAM module of ResNet50 with three 3×3 small convolution kernels. The size of the output feature map remains unchanged after the replacement, thus obtaining an optimized CBAM module. Step 4: The feature maps extracted by the encoder in each downsampling stage are fed into the optimized CBAM module, which performs weight allocation. The feature maps that have learned the weights are then used in each subsequent downsampling operation, so that the network focuses more on the crack region and suppresses irrelevant regions.

2. The lightweight UNet crack image segmentation method fused with CBAM according to claim 1, characterized in that, Step 1 includes: Step 1.1: Perform image smoothing processing on each sample data; Step 1.2: Compress the number of image channels for each sample data so that the sample data are converted into single-channel images.

3. The lightweight UNet crack image segmentation method fused with CBAM according to claim 2, characterized in that, The image smoothing process in step 1.1 is specifically a Gaussian weighted average.

4. The lightweight UNet crack image segmentation method fused with CBAM according to claim 2, characterized in that, Step 1.2, which compresses the number of image channels, specifically involves processing the image as a grayscale image.

5. The lightweight UNet crack image segmentation method fused with CBAM according to claim 1, characterized in that, The adjustment of the number of channels in the final output image of the model in step 6 depends on the number of categories. For the previous convolution of the final output layer, the number of channels is adjusted using an 1:1 convolution.

Images