Underwater bridge crack defect detection method and system based on U-Net algorithm

Abstract

The invention discloses an underwater bridge crack defect detection method and system based on a U-Net algorithm, and the method comprises the steps: enabling information to be directly transmitted from a shallow layer to a deep layer through quick connection of an added residual network module on the basis of a conventional U-Net algorithm, and effectively solving the problem of gradient disappearance; therefore, the U-Net can construct a deeper network structure, so that richer and higher-level features are extracted, and the ability of understanding complex images is improved. And secondly, the added RFB module can extract scale features of receptive fields of different sizes at the same time, more comprehensive information is provided for the model, and the detection capability of cracks of different sizes is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and particularly relates to a method and system for underwater bridge crack defect detection based on the U-Net algorithm. Background Art

[0002] During the use of bridges, they will experience a large amount of damage caused by passing vehicles, human damage, and natural environmental erosion, resulting in different degrees of damage on the bridge surface. At the same time, due to the possible degradation of the material properties used in bridge construction, cracks will appear at the bottom of the bridge, the side of the bridge, etc., and even situations such as exposed reinforcement may occur, posing serious safety hazards. Although the cracks that appear on the bridge surface are difficult to distinguish, they are the direct cause of bridge structure damage and failure. The appearance of cracks means that there are defects in the bridge structure, which will affect the overall performance and stability of the bridge. Cracks will cause a decrease in the load-bearing capacity of the bridge structure, which may lead to serious safety accidents. Cracks will accelerate the aging process of the bridge structure and shorten the service life of the bridge.

[0003] With the wide application of deep learning technology in the field of image processing, it can play a huge role in the field of bridge crack detection. However, there are still many deficiencies in the existing bridge crack detection methods based on convolutional neural networks. For example, the fixed receptive field of traditional convolutional kernels limits the adaptability of the model to cracks of different scales and morphologies, making it difficult to accurately identify small, broken or blurred crack structures; at the same time, it has poor robustness to interference information such as textures and shadows in complex backgrounds, which is prone to false detection and missed detection. In addition, the fusion of multi-layer convolutional features is insufficient, and it is easy to ignore shallow details and edge information, especially in the detection of tiny cracks. Existing models generally rely strongly on labeled data and have limited generalization ability, making it difficult to adapt to the diversity of bridge structures and environments; at the same time, the lack of effective modeling of prior information such as crack topological structures and material properties affects the interpretability and stability of detection results. Therefore, there is an urgent need to design a crack detection method with stronger structure perception ability, multi-scale adaptability and migration ability to further improve the accuracy and engineering applicability of bridge crack identification. Deep learning models can learn complex feature representations from a large amount of data, which can help identify and detect cracks of different types and sizes, and it is easier to obtain higher detection accuracy compared with traditional methods. Deep learning technology can automate crack detection, complete the processing of a huge amount of data without manual intervention, improve the detection efficiency and reduce the labor cost. Deep learning models can effectively fuse multiple data sources, such as image, video, acoustic wave and other information, improve the ability to comprehensively analyze and detect cracks, and make the detection results more comprehensive and accurate. Through transfer learning, the model trained in other fields can be transferred to the bridge crack detection task to speed up the training process and improve the detection accuracy. At the same time, continuously optimize the deep learning model structure and algorithm to further improve the effect of bridge crack detection. Summary of the Invention

[0004] The purpose of the present invention is to provide an underwater bridge crack defect detection method and system based on the U-Net algorithm, so as to solve the problems of insufficient receptive field and poor accuracy in the process of underwater bridge crack defect detection.

[0005] To achieve the technical solution of the purpose of the present invention: on the one hand, the present invention proposes a method for underwater bridge crack defect detection based on the U-Net algorithm, including the following steps:

[0006] Step 1, perform image preprocessing operations on the captured pictures;

[0007] Step 2, extract crack features through the VGG16 feature extraction network;

[0008] Step 3, increase the receptive field through the multi-branch receptive field convolution module;

[0009] Step 4: Implement low-dimensional embedding of the input through convolution, converting 256 channels into 4 channels, with a total of 32 degrees of freedom; transform each branch through transformation operations, scaling the single-dimensional subspace obtained by segmentation; aggregate the transformation results obtained from the 32 branches;

[0010] Step 5: Output the underwater bridge crack segmentation model.

[0011] On the other hand, the present invention proposes an underwater bridge crack defect detection system based on the U-Net algorithm, comprising:

[0012] A preprocessing module is used to perform image preprocessing operations on the captured images;

[0013] Crack feature extraction module, used to extract crack features through VGG16 feature extraction network;

[0014] Multi-branch receptive field convolution module, used to increase the receptive field;

[0015] The aggregation module implements low-dimensional embedding of the input through convolution, converting 256 channels into 4 channels with a total of 32 degrees of freedom; transforms each branch through transformation operations to scale the single-dimensional subspace obtained by segmentation; and aggregates the transformation results obtained by 32 branches;

[0016] Output module, used to output underwater bridge crack segmentation model.

[0017] Compared with the existing technology, the significant advantages of the present invention are: (1) Based on the traditional U-Net algorithm, the present invention adds a residual network module, which uses shortcut connections to enable information to be directly transferred from shallow layers to deep layers, effectively alleviating the gradient vanishing problem; this enables U-Net to build a deeper network structure, thereby extracting richer and more advanced features and improving the ability to understand complex images. (2) The RFB module added by the present invention can simultaneously extract scale features of receptive fields of different sizes, providing more comprehensive information for the model and improving the ability to detect cracks of different sizes.

[0018] The present invention is further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the underwater bridge crack defect detection method based on the U-Net algorithm of the present invention.

[0020] Figure 2 This is a picture of the detection effect of crack defects in underwater bridges. DETAILED DESCRIPTION

[0021] Combine Figure 1, An underwater bridge crack defect detection method based on an improved U-Net algorithm, comprising the following steps:

[0022] Step 1, perform image preprocessing operations on the captured images, and use the relevant functions in the python-opencv installation package to perform grayscale processing, corner extraction, and boundary detection on the images in sequence to achieve camera calibration.

[0023] Step 2, through the VGG16 feature extraction network, specifically:

[0024] Step 2-1, through the first convolutional layer conv1: perform two convolutions of 64 channels with [3,3] to obtain a preliminary effective feature layer of [512,512,64], and then perform 2X2 max pooling to obtain a feature layer of [256,256,64];

[0025] Step 2-2, through the second convolutional layer conv2: perform two convolutions of 128 channels with [3,3] to obtain a preliminary effective feature layer of [256,256,128], and then perform 2X2 max pooling to obtain a feature layer of [128,128,128];

[0026] Step 2-3, through the third convolutional layer conv3: perform three convolutions of 256 channels with [3,3] to obtain a preliminary effective feature layer of [128,128,256], and then perform 2X2 max pooling to obtain a feature layer of [64,64,256];

[0027] Step 2-4, through the fourth convolutional layer conv4: perform three convolutions of 512 channels with [3,3] to obtain a preliminary effective feature layer of [64,64,512], and then perform 2X2 max pooling to obtain a feature layer of [32,32,512];

[0028] Step 2-5, through the fifth convolutional layer conv5: perform three convolutions of 512 channels with [3,3] to obtain a preliminary effective feature layer of [32,32,512].

[0029] Step 3, through the multi-branch receptive field convolution module, specifically:

[0030] Step 3-1, perform dimensionality reduction through a 1×1Conv convolutional layer with a dilation rate of 1, the size of the convolutional kernel is 3×3, the dilation rate of the dilated convolution is 1, and the size of the receptive field at this time is 3×3;

[0031] The calculation formula for the receptive field is:

[0032] r n-1 ,

[0032] , n , =r n-1+(k - 1)×d×s (1)

[0033] Among them, r n-1 represents the receptive field size of the upper layer, r n represents the size of the current convolution kernel, k represents the size of the convolution kernel (kernel) of the current convolution layer, that is, the size of each convolution kernel in the spatial dimensions (height, width); the dilation rate of the dilated convolution is d, and the stride is s;

[0034] Step 3 - 2: Through a 3×3 Conv convolution layer with a dilation rate of 3, the dilation rate is 3, and the size of the original convolution kernel is 3×3. According to the formula r n = r n-1 +(k - 1)×d×s, the size of the receptive field can be obtained as 9;

[0035] Step 3 - 3: Through a convolution layer with a dilation rate of 5 and an original convolution kernel size of 3×3, according to the formula r n = r n-1 +(k - 1)×d×s, the size of the receptive field can be obtained as 19.

[0036] Step 4: Implement low - dimensional embedding on the input through convolution, change 256 channels to 4 channels, with a total of 32 degrees of freedom. Then perform a transformation on each branch through a transformation operation, and simply scale the segmented one - dimensional subspace x i to obtain w i x i , w i represents the weight coefficient corresponding to the i - th branch. Finally, aggregate the transformation results - feature maps obtained from 32 branches to obtain where x represents the input feature, which is the feature map extracted from the original image or the previous network layer, τ i (x) represents the transformation function of the i - th branch, indicating a series of non - linear transformations performed on the input feature x on branch i; c is the number of branches, taking 32.

[0037] This aggregated feature serves as the final high - order representation, which can better capture the local and global semantic information in the input image and be used as the input for the subsequent underwater bridge crack segmentation module to improve the model's perception and recognition ability of crack targets.

[0038] Step 5: Output the underwater bridge crack segmentation model.

[0039] Based on the same inventive concept, the present invention also proposes an underwater bridge crack defect detection system based on the U - Net algorithm, including:

[0040] A pre - processing module for performing image pre - processing operations on the captured pictures;

[0041] The crack feature extraction module is used to extract crack features through the VGG16 feature extraction network;

[0042] The multi-branch receptive field convolution module is used to increase the receptive field;

[0043] The aggregation module realizes low-dimensional embedding of the input through convolution, changes 256 channels into 4 channels, with a total of 32 degrees of freedom; performs transformation on each branch through transformation operations and scales the single-dimensional subspaces obtained by segmentation; aggregates the transformation results obtained from 32 branches;

[0044] The output module is used to output the underwater bridge crack segmentation model.

[0045] The specific implementation methods of the above modules are the same as the steps of the aforementioned underwater bridge crack defect detection method, and will not be elaborated here.

[0046] The results of the method for underwater bridge crack defect detection based on the improved U-Net algorithm are shown in Table 1:

[0047] Table 1

[0048]

[0049] Combined with Figure 2It can be seen that from the comparison charts of the crack segmentation effects of various algorithms, the algorithm of the present invention can better capture targets of different scales in segmentation compared with other algorithms. For the reticular cracks with complex shapes and unclear boundaries, the algorithm of the present invention can perform more refined segmentation, better retain the edge information and details of the image, and accurately outline the contour of the target. The DeepLab V3 algorithm has over-segmentation at the details of the cracks, resulting in an incomplete match with the boundary of the actual target. PSPNet has poor detail processing for reticular cracks, is interfered by the shadows and background noises of underwater pictures, and there will be fractures in the segmentation of cracks, and the extracted cracks are not continuous enough, missing some edge information and details. It can be seen from the figure that the original U-Net network can basically separate the cracks from the image, but there is a certain degree of blurring in some areas with rich details, making the segmentation result not sharp enough. The processing of the background noise of the picture is not ideal enough, and there will also be over-segmentation. As shown in Table 1, the indicators of the Deeplabv3 model in bridge crack recognition are all the lowest. The reason may be that the output result of DeepLabV3 is usually a pixel-level segmentation mask, and it is relatively difficult to evaluate specific indicators such as the width, length, and depth of the cracks, and it is difficult to directly meet the requirements of accurate measurement and evaluation of cracks in engineering practice. The PSPNet model has improved in various evaluation indicators compared with the Deeplabv3 model, but it is still difficult to meet the accuracy requirements of bridge crack detection. Compared with the PSPNet model and the Deeplabv3 model, the U-Net network can more accurately segment the boundary of the cracks when processing crack images with complex backgrounds and fuzzy boundaries. For small cracks and cracks similar to the background, it can also have good detection effects, can effectively distinguish the cracks from the surrounding background noises, improve the detection accuracy and reliability, and has a large lead in various indicators. The improved algorithm of the present invention has a relatively obvious improvement compared with the original U-Net network. It not only has a more excellent crack segmentation effect subjectively, but also has more obvious advantages in the four objective evaluation indicators of MIou, MPA, Precision, and Accuracy. It can be seen that the modification of the network structure by the algorithm of the present invention significantly improves the detection performance and accuracy of the network.

Claims

1. An underwater bridge crack defect detection method based on the U-Net algorithm, characterized in that It includes the following steps: Step 1: Perform image preprocessing operations on the captured images; Step 2: Extract crack features through the VGG16 feature extraction network; Step 3: Increase the receptive field through the multi-branch receptive field convolution module; Step 4: Achieve low-dimensional embedding of the input through convolution, changing 256 channels to 4 channels, with a total of 32 degrees of freedom; perform transformation operations on each branch to scale the single-dimensional subspaces obtained by segmentation; aggregate the transformation results obtained from 32 branches; Step 5: Output the underwater bridge crack segmentation model.

2. The underwater bridge crack defect detection method according to claim 1, characterized in that, In step 1 and step 4, each branch is transformed through a transformation operation, and the single-dimensional subspace x obtained by segmentation i is scaled to obtain w i x i , w i represents the weight coefficient corresponding to the i-th branch; the transformation results obtained from 32 branches are aggregated to obtain where x represents the input feature, which is the feature map extracted from the original image or the previous network layer, and τ i (x) represents the transformation function of the i-th branch, which represents a series of non-linear transformations performed on the input feature x on branch i, and c is the number of branches.

3. The underwater bridge crack defect detection method according to claim 1, characterized in that Specifically, Step 2 is as follows: Step 2-1: Through the first convolutional layer conv1: Perform two convolutions with 64 channels of [3,3] to obtain a preliminary effective feature layer of [512,512,64], and then perform 2X2 max pooling to obtain a feature layer of [256,256,64]; Step 2-2: Through the second convolutional layer conv2: Perform two convolutions with 128 channels of [3,3] to obtain a preliminary effective feature layer of [256,256,128], and then perform 2X2 max pooling to obtain a feature layer of [128,128,128]; Step 2-3: Through the third convolutional layer conv3: Perform three convolutions with 256 channels of [3,3] to obtain a preliminary effective feature layer of [128,128,256], and then perform 2X2 max pooling to obtain a feature layer of [64,64,256]; Step 2-4: Through the fourth convolutional layer conv4: Perform three convolutions with 512 channels of [3,3] to obtain a preliminary effective feature layer of [64,64,512], and then perform 2X2 max pooling to obtain a feature layer of [32,32,512]; Step 2-5: Through the fifth convolutional layer conv5: Perform three convolutions with 512 channels of [3,3] to obtain a preliminary effective feature layer of [32,32,512].

4. The underwater bridge crack defect detection method according to claim 1, characterized in that Specifically, Step 3 is as follows: Step 3-1: Perform dimensionality reduction through a 1×1 Conv convolutional layer with a dilation rate of 1. The size of the convolutional kernel is 3×3, and the dilation rate of the dilated convolution is 1. At this time, the size of the receptive field is 3×3; The calculation formula for the receptive field is: r n = r n-1 + (k - 1) × d × s (1) Among them, r n-1 represents the receptive field size of the upper layer, and r n represents the size of the current convolutional kernel, and k represents the convolutional kernel size of the current convolutional layer; the dilation rate of the dilated convolution is d, and the stride is s; Step 3-2: Through a 3×3 Conv convolutional layer with a dilation rate of 3, the dilation rate is 3, and the size of the original convolutional kernel is 3×3. According to the formula r n = r n-1 +(k - 1)×d×s, the receptive field size can be obtained as 9; Step 3-3: With a porosity of 5 and an original convolution kernel size of 3×3, according to the formula r n = r n-1 +(k - 1)×d×s, the receptive field size is 19.

5. The underwater bridge crack defect detection method according to claim 1, characterized in that, In Step 1, functions in the python-opencv installation package are used to perform grayscale processing, corner extraction, and boundary detection on the images in sequence to achieve camera calibration.

6. An underwater bridge crack defect detection system based on the U-Net algorithm, characterized in that, It includes: A preprocessing module for performing image preprocessing operations on the captured images; A crack feature extraction module for extracting crack features through the VGG16 feature extraction network; A multi-branch receptive field convolution module for increasing the receptive field; An aggregation module that achieves low-dimensional embedding of the input through convolution, changes 256 channels to 4 channels, with a total of 32 degrees of freedom; performs transformation operations on each branch to scale the single-dimensional subspaces obtained by segmentation; aggregates the transformation results obtained from 32 branches; An output module for outputting the underwater bridge crack segmentation model.

7. The underwater bridge crack defect detection system based on the U-Net algorithm according to claim 6, wherein, The described aggregation module realizes low-dimensional embedding of the input through convolution, changing 256 channels into 4 channels, with a total of 32 degrees of freedom; it performs transformation on each branch through a transformation operation, and scales the single-dimensional subspace x obtained by segmentation to get w i to obtain w i x i , w i represents the weight coefficient corresponding to the i-th branch; the transformation results obtained from 32 branches are aggregated to obtain where x represents the input feature, which is the feature map extracted from the original image or the previous network layer, and τ i (x) represents the transformation function of the i-th branch, which represents a series of non-linear transformations performed on the input feature x on branch i, and c is the number of branches.

8. The underwater bridge crack defect detection system based on the U-Net algorithm according to claim 6, characterized in that, The extraction of crack features through the VGG16 feature extraction network is specifically as follows: Through the first convolutional layer conv1: Perform two convolutions with 64 channels of [3,3] to obtain a preliminary effective feature layer of [512,512,64], and then perform 2X2 max pooling to obtain a feature layer of [256,256,64]; Through the second convolutional layer conv2: Perform two convolutions with 128 channels of [3,3] to obtain a preliminary effective feature layer of [256,256,128], and then perform 2X2 max pooling to obtain a feature layer of [128,128,128]; Through the third convolutional layer conv3: Perform three convolutions with 256 channels of [3,3] to obtain a preliminary effective feature layer of [128,128,256], and then perform 2X2 max pooling to obtain a feature layer of [64,64,256]; Through the fourth convolutional layer conv4: Perform three convolutions with 512 channels of [3,3] to obtain a preliminary effective feature layer of [64,64,512], and then perform 2X2 max pooling to obtain a feature layer of [32,32,512]; Through the fifth convolutional layer conv5: Perform three convolutions with 512 channels of [3,3] to obtain a preliminary effective feature layer of [32,32,512].

9. The underwater bridge crack defect detection system based on the U-Net algorithm according to claim 6, characterized in that, The extraction of crack features through the VGG16 feature extraction network is specifically as follows: Perform dimensionality reduction through a 1×1Conv convolutional layer with a dilation rate of 1. The size of the convolutional kernel is 3×3, and the dilation rate of the dilated convolution is 1. At this time, the size of the receptive field is 3×3; The calculation formula for the receptive field is: r n = r n-1 + (k - 1) × d × s (1) Among them, r n-1 represents the receptive field size of the previous layer, and r n represents the size of the current convolutional kernel. k represents the size of the convolutional kernel (kernel) of the current convolutional layer, that is, the size of each convolutional kernel in the spatial dimensions (height, width); the dilation rate of the dilated convolution is d, and the stride is s; Through a 3×3 Conv convolutional layer with a dilation rate of 3, the dilation rate is 3, and the size of the original convolutional kernel is 3×3. According to the formula r n = r n-1 +(k - 1)×d×s, the receptive field size can be obtained as 9; With a porosity of 5 and an original convolution kernel size of 3×3, according to the formula r n = r n-1 +(k - 1)×d×s, the receptive field size can be obtained as 19.

10. The underwater bridge crack defect detection system based on the U-Net algorithm according to claim 6, wherein, The image preprocessing operations include: sequentially performing grayscale processing, corner extraction, and boundary detection on the picture using the functions in the python-opencv installation package to achieve calibration at the camera.

Images