Crack detection method based on improved VGG network model

Abstract

The invention discloses a crack detection method based on an improved VGG network model, belongs to the technical field of crack detection, and aims to improve the precision and efficiency of image crack segmentation. The method comprises the following steps: acquiring a crack image with a label, converting the crack image into a grayscale image, and performing preprocessing by adopting fuzzy morphological filtering so as to reduce the influence of noise on crack detection; in a network structure, a receptive field is expanded through a cavity convolution module, so that the network can capture richer context information; meanwhile, the transmission and fusion capabilities of different level features are enhanced by using a residual structure; the path enhancement module further optimizes the feature path and improves the segmentation performance of the model; and finally, the image size is recovered through a deconvolution and up-sampling module, and crack analysis is realized. According to the invention, the accuracy of crack segmentation is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of crack detection, and in particular to a crack detection method based on an improved VGG network model. Background Art

[0002] The aging of infrastructure and the increasing traffic load make crack detection a key link to ensure the safety of buildings and roads. Accurate image crack segmentation helps to timely discover and repair structural defects and prevent potential safety hazards. In recent years, convolutional neural networks (CNNs) have been widely used in the field of image crack detection due to their powerful feature extraction and representation capabilities. Crack segmentation technology based on CNN and VGG has become a hot topic of research. Among them, Wu et al. combined the Retinex image enhancement algorithm with the VGG_19 model to study efficient crack image segmentation, focusing on crack length and width information, and the crack detection accuracy still needs to be improved. Tang et al. proposed a multi-task enhancement method based on Faster-RCNN, focusing on small crack detection, but the model needs to be further enhanced in crack detection effect. Vishal et al. proposed an automatic pavement damage analysis system based on yolov2 deep learning architecture, which has a high running speed, but poor performance in crack detection quantity and clarity. Sukhad et al. proposed an algorithm based on SqueezeNet and texture coding layer, which has good robustness and effectiveness, but poor real-time detection performance. Madiha et al. proposed a transfer learning method for the VGG19 network, which can improve the generalization ability of the model but consumes a lot of computing resources.

[0003] In 2015, a fully convolutional network architecture (UNet) was applied to image segmentation. This architecture combines high-resolution features and low-resolution contextual information, and uses symmetric structures and jump links to make it perform well in crack segmentation tasks. Based on this UNet architecture, Vladimir et al. proposed the DAUNet model, which has advantages in lightweight and training time, but poor noise processing performance. Fan et al. proposed the UNet convolutional neural network for automated crack segmentation. The algorithm can focus on the efficiency of irregular images and noisy images and achieve automated segmentation. However, the effect of detail segmentation is not good. Zhou et al. proposed the UNet++ architecture, which improves the performance of crack segmentation tasks through dense jump connections and deep supervision mechanisms.

[0004] Furthermore, researchers applied the attention mechanism to image crack segmentation. Among them, the STrans-YOLOX model proposed by Luo et al. combines CNN and SwinTransformer and introduces a global attention guidance module. Although it enhances the ability of multi-scale feature extraction, it still needs to be improved in terms of crack detection accuracy and denoising effect. Zhang et al. proposed the APLCNet algorithm, which effectively improved the crack detection accuracy through channel attention mechanism and spatial attention mechanism, but the effect of crack contour processing was poor. Saberironaghi et al. proposed a crack segmentation model combining multi-scale attention mechanism with efficient UNet. This model has significant advantages in processing complex backgrounds and small cracks, but the computing resource requirements are large and the model training is complex. Katsamenis et al. proposed an R2AU-NeT model that combines attention mechanism and few-shot learning. Through dynamic fine-tuning, it can optimize the crack segmentation performance according to user feedback. However, the computing resource requirements are large and the model training is complex. In addition, the FPHBN network proposed by Fan et al. can effectively extract image features, but the detection efficiency is poor. The fully convolutional neural network and deep supervision network (HED) proposed by Xie et al. realize image-to-image edge detection, optimize the computing speed, but the detail processing effect is poor.

[0005] In order to overcome the shortcomings of existing methods in noise processing and small crack detection, this study proposed a crack detection method based on an improved VGG network model, aiming to improve the accuracy and efficiency of image crack segmentation. Summary of the invention

[0006] The purpose of the present invention is to provide a crack detection method based on an improved VGG network model to solve the following technical problems:

[0007] Overcome the shortcomings of existing methods in noise processing and small crack detection, and improve the accuracy and efficiency of image crack segmentation.

[0008] The purpose of the present invention can be achieved through the following technical solutions:

[0009] A crack detection method based on an improved VGG network model comprises the following steps:

[0010] Acquire a crack image, convert the crack image into a grayscale image, normalize the grayscale image, and then perform fuzzy morphological filtering on the normalized grayscale image to obtain an input image;

[0011] Construct an improved model, wherein the improved model uses the VGG13 network as the basic network, combines residual hole convolution and path enhancement modules, and adopts an encoder and decoder architecture;

[0012] Input the input image into the improved model, the improved model includes 5 layers of convolution blocks, wherein the first layer, the third layer and the fourth layer extract the basic features of the crack image through standard convolution operations, the second layer captures the multi-scale features of the crack image through the residual hole convolution module, the fifth layer is a hole convolution module, and the hole convolution module is followed by a path enhancement module;

[0013] The path enhancement module uses the information of shallow features of the input image to enhance the representation capability of deep features through top-down feature transfer;

[0014] The Dice coefficient is used as the loss function to measure the performance of the improved model. The gradient formula of the Dice loss function to the model prediction value is calculated. The gradient is calculated using the gradient descent method, the network weights are updated, and the loss function is minimized until the model converges.

[0015] As a further solution of the present invention: the specific process of the fuzzy morphological filtering process is:

[0016] The filtering process includes fuzzy expansion, fuzzy erosion, fuzzy opening and closing operations. Fuzzy expansion enhances the brightness of the image by combining each pixel in the image with the maximum value of its neighborhood. The calculation formula is:

[0017]

[0018] Among them, U A The fuzzy set representing the original crack image, U B (γ) represents the fuzzy set of structural elements, sup represents the supremum, and min represents the minimum value;

[0019] Fuzzy erosion combines each pixel in the image with the minimum value of its neighborhood to reduce the brightness of the image. The calculation formula is:

[0020]

[0021] Among them, inf represents the infimum, and max represents the maximum value. In the erosion operation, for each pixel in the image, the maximum value of its neighborhood and the corresponding position of the structural element is taken, and the minimum value is taken as the pixel value after erosion among the maximum values ​​obtained. In this way, the erosion operation retains the smaller pixel values ​​in the image and reduces the larger pixel values ​​in each local area;

[0022] The fuzzy opening operation is used to remove noise by first performing fuzzy erosion and then fuzzy dilation. The calculation formula is:

[0023]

[0024] The fuzzy closing operation first performs fuzzy expansion and then fuzzy erosion to fill small holes and cracks in the image. The calculation formula is:

[0025]

[0026] As a further solution of the present invention: the process of capturing features by the residual hole convolution module is:

[0027] The residual atrous convolution module consists of four branches, of which the convolution expansion rates of three branches are 1, 3, and 5, respectively, to capture features of different scales. The convolution operation formula is:

[0028] F i (Y) = σ(W i *Y+b i )

[0029] Among them, σ represents the activation function ReLU, W i and b i are the weight and bias of the i-th convolutional layer, respectively. * indicates the convolution operation. In the entire residual dilated convolution module, 1×1 convolution is used to capture local features and reduce dimensions. 3×3 convolution with dilation rate = 1 is used for standard convolution to extract detailed features. 3×3 convolution with dilation rate = 3 is used to capture the mid-range context by expanding the receptive field. 3×3 convolution with dilation rate = 5 is used to extract broader context information.

[0030] After processing through the dilated convolutions of different branches, the outputs of all branches are connected, and then the multi-scale features are fused through another 1×1 convolution. The formula is:

[0031] F(Y)=σ(W f *concat(F1(Y),F2(Y),F3(Y),F4(Y))+b f )

[0032] Among them, W f and b f are the weight and bias of the 1×1 convolution layer, respectively. F(Y) represents the fused features after multi-scale convolution. The final output is connected through the residual connection. The input feature map passes through the 1×1 convolution layer and is then added to the output of the improved residual hole convolution module to achieve feature extraction. The calculation formula is:

[0033] I=Y''+F(Y)

[0034] Among them, Y'' represents the result of processing through a 1×1 convolutional layer, F(Y) represents the features processed through a 1×1 convolutional layer, and the feature map processed by the hole residual module can enhance the multi-scale context while retaining the original features.

[0035] As a further solution of the present invention: the residual hole convolution module performs convolution processing by adjusting the receptive field size and the convolution kernel size fusion method, and adjusts the receptive field of the convolution kernel with a hole rate of 2 and 5 to 5 and 10, and adopts spatially separable convolution;

[0036] Use 5×1 and 1×5 concatenated convolution modules instead of a 5×5 module, add 1×1 convolution after each branch to adjust the linear excitation and dimension, and use global average pooling to add and average all pixel values ​​in the feature map to obtain the feature vector. The calculation formula is:

[0037]

[0038] Among them, H represents the height of the feature map, M represents the width of the feature map, and Y ab represents the pixel value in the ath row and bth column of the feature map, Represents the sum of all pixel values ​​in the feature map. By performing a double summation operation on all pixel values ​​in the feature map, the sum of all pixel values ​​in the feature map is calculated. Represents the normalization factor.

[0039] As a further solution of the present invention: the process of enhancing the features of the path enhancement module is as follows:

[0040] The path enhancement module extracts feature maps from the shallow and deep layers of the network respectively. The shallow feature map contains details and positioning information, and the deep feature map contains semantic information. low Upsample and add it element by element with the deep feature map to get the fused feature result, and then use the convolution layer to process the fused result. The calculation formula is:

[0041]

[0042] i∈[1,G hight ],j∈[1,G hight ]

[0043] The fusion result is further processed using the convolution layer, the formula is:

[0044]

[0045] Among them, c is the size of the convolution kernel, ω nm is the convolution kernel weight, and b is the bias term.

[0046] As a further solution of the present invention: the formula of the Dice loss function is:

[0047]

[0048] in, Indicates The predicted value of pixels, Indicates The true label of pixels, O represents the total number of pixels,

[0049] In the back propagation, the loss function is calculated for the predicted value The gradient of is:

[0050]

[0051] make and The simplified formula of Dice loss function is:

[0052]

[0053] Then, yes Derivative, use the chain rule, calculate the numerator and denominator, and get the Dice loss function for the predicted value The gradient formula is:

[0054]

[0055] By substitution and simplification, the final gradient formula is obtained, and the gradient is calculated using the gradient descent method to update the network weights and minimize the loss function until the model converges.

[0056] Beneficial effects of the present invention:

[0057] The present invention proposes an improved VGG network model for crack detection, which can effectively improve the accuracy of crack segmentation and the accuracy of detail segmentation. By using fuzzy morphological filtering to preprocess the input image, the influence of noise on crack detection is effectively reduced. By introducing void convolution and residual structure, this study significantly expands the receptive field of the network and improves the feature extraction capability. The introduction of the path enhancement module further improves the feature expression capability and detail capture capability of the network at different scales through top-down feature transfer and multi-scale feature fusion. Experimental verification on the data set has proved that the algorithm has significant advantages in detail processing and image segmentation accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The present invention will be further described below in conjunction with the accompanying drawings.

[0059] Figure 1 It is a schematic diagram of the structure of the VGG_13 network;

[0060] Figure 2 Schematic diagram of the process of dilated residual convolution;

[0061] Figure 3To improve the structural diagram of the VGG network model;

[0062] Figure 4 Schematic diagram of the residual hole convolution module;

[0063] Figure 5 The denoising effect diagram of different algorithms;

[0064] Figure 6 The segmentation effect diagram of different algorithms in the self-built data set;

[0065] Figure 7 The segmentation effect diagram of different algorithms on CFD data sets;

[0066] Figure 8 The following is a diagram showing the segmentation effects of different algorithms on the Crack500 dataset. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] See also Figure 1-Figure 8 As shown, the present invention is a crack detection method based on an improved VGG network model, comprising the following steps:

[0069] 1. Image preprocessing

[0070] Crack images are collected, including public data sets and self-built data sets. The self-built data sets are unlabeled, while the public data sets are labeled. In order to effectively reduce crack noise and better enhance crack edge and detail information, this study proposes to use fuzzy morphological filtering to preprocess cracks. Fuzzy morphological filtering combines the advantages of fuzzy logic and mathematical morphology. Mathematical morphology can enhance crack edge information and fill and connect cracks. At the same time, the grayscale image itself is fuzzy. Fuzzy logic can effectively classify and filter out noise, thereby avoiding noise amplification during expansion or corrosion.

[0071] The input image is converted into a grayscale image to facilitate fuzzy morphological operations. Then the image is normalized and the image size is adjusted to 224*224 to unify the data set size to adapt to the neural network. Finally, fuzzy morphological filtering is performed. The filtering process is divided into fuzzy expansion, fuzzy erosion, fuzzy opening and closing operations. Fuzzy expansion enhances the brightness of the image by combining each pixel in the image with the maximum value of its neighborhood. The calculation is as follows:

[0072]

[0073] Among them, U A The fuzzy set representing the original crack image, U B (γ) represents the fuzzy set of the structural element, sup represents the supremum, and min represents the minimum value.

[0074] Fuzzy erosion reduces the brightness of an image by combining each pixel in the image with the minimum value of its neighborhood. The formula is as follows:

[0075]

[0076] Among them, inf represents the infimum, and max represents the maximum value. In the erosion operation, for each pixel in the image, the maximum value of its neighborhood and the corresponding position of the structural element is taken, and the minimum value among the maximum values ​​obtained is taken as the pixel value after erosion. In this way, the erosion operation retains the smaller pixel values ​​in the image and reduces the larger pixel values ​​in each local area.

[0077] The fuzzy opening operation is used to remove noise by first performing fuzzy erosion and then fuzzy dilation. The calculation is as follows:

[0078]

[0079] The fuzzy closing operation first performs fuzzy expansion and then fuzzy erosion to fill small holes and cracks in the image. The calculation is as follows:

[0080]

[0081] Through fuzzy morphological filtering processing, the crack image noise is significantly reduced, while the crack detail information is retained and enhanced.

[0082] 2. Improve the VGG_13 model

[0083] In order to improve the accuracy and detail processing ability of crack segmentation, this study proposes an improved VGG crack segmentation network. The model is based on the VGG_13 network and combines residual hole convolution and path enhancement modules to enhance image segmentation performance. The improved model adopts an encoder and decoder architecture.

[0084] 3. Schematic diagram of improved VGG network model

[0085] Depend on Figure 3 It can be seen that the encoder retains the first four convolutional layers of VGG_13, among which the first, third and fourth layers extract the basic features of the crack image through standard convolution operations. The second layer uses Figure 2The dilated residual convolution module shown in the figure uses convolution layers with different dilation rates to capture multi-scale features, thereby expanding the receptive field and enhancing the ability to capture complex edge and detail features without significantly increasing the computational cost. The improved dilated convolution module (DCM) is used in the fifth convolution block to further enhance the network's ability to capture features at different scales. RDC effectively improves the feature extraction ability and robustness of the model by introducing a more advanced dilated convolution structure and a multi-scale feature fusion mechanism. The path enhancement module is introduced after RDC to improve the feature expression ability of the model at different levels. The path enhancement module uses the information of shallow features to enhance the representation ability of deep features through top-down feature transfer. The decoder part restores the image size through deconvolution and upsampling operations to achieve crack segmentation.

[0086] 4. RDC module

[0087] In order to enhance the feature extraction capability of the network, an improved residual dilated convolution module (RDC) is proposed. This module introduces convolution layers with different dilation rates and a multi-scale feature fusion mechanism, which enables RDC to capture feature information in a larger range of images and improves the ability to capture complex scenes and detailed features. Figure 4 As shown:

[0088] 5. Residual Dilated Convolution Module

[0089] As can be seen from the figure above, the input feature map Y is transformed through a series of convolutional layers with different dilation rates. This configuration ensures the effective capture and combination of multi-scale features. The improved residual hole convolution module consists of four branches, each with a convolution dilation rate of 1, 3, and 5, respectively. These branches are designed to capture features of different scales. The convolution operation is as follows:

[0090] F i (Y) = σ(W i *Y+b i )

[0091] Where σ represents the activation function ReLU, W i and b i are the weights and biases of the ith convolutional layer, respectively, and * denotes the convolution operation. In the entire residual atrous convolution module, 1×1 convolution is used to capture local features and reduce dimensions. 3×3 convolution (dilation rate = 1) is used for standard convolution to extract detailed features. 3×3 convolution (dilation rate = 3) captures mid-range context by expanding the receptive field. 3×3 convolution (dilation rate = 5) is used to extract broader context information.

[0092] After processing through the dilated convolutions of different branches, the outputs of all branches are connected, and then the multi-scale features are fused through another 1×1 convolution. The calculation is as follows:

[0093] F(Y)=σ(W f *concat(F1(Y), F2(Y), F3(Y), F4(Y))+b f )

[0094] Among them, W f and b f are the weight and bias of the 1×1 convolution layer, and F(Y) represents the fused features after multi-scale convolution. The final output is connected through a residual connection, and the input feature map is passed through a 1×1 convolution layer and then added to the output of the improved residual hole convolution module to achieve feature extraction. The calculation is as follows:

[0095] I=Y''+F(Y)

[0096] Among them, Y'' represents the result of processing through a 1×1 convolutional layer. F(Y) represents the feature after processing through a 1×1 convolutional layer. The feature map processed by the hole residual module can enhance the multi-scale context while retaining the original features.

[0097] In order to reduce the problems of insufficient local detail preservation and increased computational complexity caused by high receptive fields due to high voiding rates, this study performs convolution processing by adjusting the receptive field size and the convolution kernel size fusion method. The receptive fields of convolution kernels with voiding rates of 2 and 5 are adjusted to 5 and 10. At the same time, spatially separable convolution is used, and 5×1 and 1×5 series convolution modules are used instead of a 5×5 module to ensure the receptive field while reducing network parameters and increasing nonlinearity and model expression capabilities. Adding 1×1 convolution after each branch adjusts linear excitation and dimension, reduces parameters and computation, and improves computational efficiency. In addition, this study uses global average pooling to make the feature layer easier to convert into classification probabilities and avoid overfitting. Global average pooling adds and averages all pixel values ​​in the feature map to obtain the feature vector. The calculation is as follows:

[0098]

[0099] Among them, H represents the height of the feature map, M represents the width of the feature map, and Y ab Represents the pixel value in row a and column b of the feature map

[0100] Represents the sum of all pixel values ​​in the feature map. The sum of all pixel values ​​in the feature map is calculated by performing a double summation operation on all pixel values ​​in the feature map. Represents the normalization factor.

[0101] 6. Path Enhancement Module

[0102] In order to compensate for the information loss caused by continuous pooling and convolution operations and improve the accuracy and efficiency of image crack segmentation, this study proposes to use a path augmentation module (PAM) to optimize the model. The path augmentation module uses the information of shallow features to enhance the representation ability of deep features through top-down feature transfer.

[0103] The path enhancement module extracts feature maps from the shallow (low-level) and deep (high-level) layers of the network. The shallow feature maps contain details and positioning information, and the deep feature maps contain semantic information. low Upsample and add it element by element with the deep feature map to get the fused feature result, and then use the convolution layer to process the fused result. The calculation is as follows:

[0104]

[0105] i∈[1,G hight ],j∈[1,G hight ]

[0106] Finally, the convolutional layer is used to further process the fusion result, as shown below:

[0107]

[0108] Among them, c is the size of the convolution kernel, ω nm is the convolution kernel weight, and b is the bias term.

[0109] The path enhancement module (PAM) can effectively utilize the information of shallow features, enhance the representation ability of deep features, and improve the accuracy of crack segmentation and the ability to capture details.

[0110] 7. Loss Function

[0111] To ensure that the model focuses on important details and improves the ability to detect and segment small cracks, this study uses the Dice coefficient loss function to optimize network parameters. The Dice coefficient is a statistical indicator that measures the similarity of two sample sets. Through forward propagation, the predicted value and the true value are calculated, and the Dice loss function is calculated. The Dice loss function is shown below:

[0112]

[0113] in, Indicates The predicted value of pixels, Indicates The true label of pixels, O represents the total number of pixels.

[0114] In the back propagation, the loss function is calculated for the predicted value The gradient is as follows:

[0115]

[0116] make and The Dice loss function is simplified to the following formula:

[0117]

[0118] Then, yes Derivative, use the chain rule, calculate the numerator and denominator, and get the Dice loss function for the predicted value The gradient formula is as follows:

[0119]

[0120] By substituting and simplifying, we get the final gradient formula. We use the gradient descent method to calculate the gradient, update the network weights, minimize the loss function, and improve the segmentation performance of the model.

[0121] In another embodiment of the present invention:

[0122] 1. Experimental environment

[0123] The study used the deep learning framework tensorflow to implement the network, and the experiment was conducted on a HP Z8G4 graphics workstation. The device is equipped with two Intel Xeon 5218R CPUs, each with 20 cores and running at 2.1 GHz, for a total of 40 cores. Graphics processing power is provided by an A4000 series 16GB graphics card. The system memory is 128GB, and the storage solution includes a 512GB solid-state drive (SSD) and a 2TB mechanical hard drive. The operating system used is Linux, and self-built datasets, SDNET2018, Crack500 and CFD datasets are used for testing. The dataset information is shown in Table 1:

[0124] Table 1 Dataset information

[0125]

[0126]

[0127] The dataset contains 68,618 images in total. To expand the data resources, this study uses rotation, translation and flipping methods to augment the data to increase the diversity of the data. The augmented dataset contains 117,236 images.

[0128] To test the effectiveness of the model, this study used SDNET2018 and a random 70% self-built dataset for model training, and verified it on the self-built dataset, Crack500, and CFD datasets.

[0129] 2 Evaluation indicators

[0130] In order to quantitatively analyze the differences between algorithms, common image segmentation evaluation indicators are used to measure the advantages and disadvantages of the algorithm in this paper and other algorithms, including accuracy, recall, mean pixel accuracy (MPA) and mean intersection over union (MIoU). Accuracy is used to evaluate the overall accuracy of the segmentation results, and the calculation is as follows:

[0131]

[0132] Among them, TP is true positive, TN is true negative, FP is false positive, and FN is false negative. The accuracy reflects the proportion of correct classification, and its value range is [0,1]. The larger the value, the better the segmentation effect.

[0133] The recall rate is used to evaluate the detection ability of the model and is calculated as follows:

[0134]

[0135] The recall rate reflects the proportion of actual positive examples that are correctly identified. Its value range is [0,1]. The larger the value, the better the segmentation effect.

[0136] The average pixel accuracy is used to evaluate the pixel classification accuracy of each category and is calculated as follows:

[0137]

[0138] Among them, L is the number of categories, TP q and FN q are the true positives and false negatives of the qth category, respectively. MPA reflects the average accuracy of each category, and its value range is [0,1]. The larger the value, the better the segmentation effect.

[0139] The average intersection-over-union ratio is used to evaluate the overlap between the segmentation result and the actual result, and the calculation is as follows:

[0140]

[0141] Among them, L is the number of categories, FN q ,TP q and FN qare the true positive, false positive, and false negative of the qth category, respectively. MIoU reflects the average intersection-over-union ratio of each category, and its value range is [0,1]. The larger the value, the better the segmentation effect.

[0142] 4 Image denoising analysis using different algorithms

[0143] In order to verify the effect of image preprocessing, the study used fuzzy mathematical morphological filtering, bilateral filtering, Gaussian filtering and wavelet filtering for noise reduction. Figure 5 As shown: from left to right: (a) original image, (b) fuzzy mathematical morphology filtering, (c) bilateral filtering, (d) Gaussian filtering, and (e) wavelet filtering.

[0144] Depend on Figure 5 ,Fuzzy mathematical morphology filtering can effectively filter the white point noise in the image, but does not affect the detailed features of the cracks in the original image. ,While wavelet filtering has poor noise processing effect during the noise reduction process, it also causes a large loss of details.

[0145] In order to objectively verify the denoising performance of the algorithm, 200 crack data set images were randomly selected and denoised using fuzzy mathematical morphology filtering, bilateral filtering, Gaussian filtering, and wavelet filtering. The structural similarity index (SSIM), mean square error (MSE), and normalized correlation coefficient (NC) were used for evaluation. The evaluation results were retained to 2 decimal places, as shown in Table 2:

[0146] Table 2 Performance evaluation of different filters

[0147]

[0148]

[0149] As can be seen from Table 2, the best processing results of SSIM, MSE and NC are all obtained by fuzzy mathematical morphology filtering, indicating that fuzzy mathematical morphology filtering performs well in processing complex structures and retaining details. This is due to the fact that fuzzy mathematical morphology filtering is based on fuzzy set theory, which allows for effective processing of uncertainty and ambiguity. However, the edges of cracks are often fuzzy and irregular. Fuzzy set theory can better describe and retain the fuzzy edges of cracks by fuzzifying the grayscale of pixels. At the same time, fuzzy mathematical morphology filtering uses multi-scale structural elements and can process images at different scales, thereby effectively capturing structural features at different scales. Therefore, fuzzy mathematical morphology filtering can well retain the structure and details of the image while removing noise, thereby improving the accuracy and reliability of the model for image segmentation.

[0150] 4 Analysis of segmentation effects of different algorithms

[0151] To verify the effectiveness of the algorithm of the present invention, the present algorithm, UNet++, VGG_19, HED and CNN algorithms were used to train the self-built dataset and SDNET2018 dataset under the same experimental environment, and the segmentation effect was verified on the self-built dataset, Crack500 and CFD crack data. Figures 6 to 8 As shown;

[0152] Figure 6 From left to right in the figure are (a) original image, (b) label, (c) algorithm of the present invention, (d) UNet++, (e) VGG_19, (f) HED, and (g) CNN.

[0153] Figure 7 From left to right in the figure are (a) original image, (b) label image, (c) algorithm of the present invention, (d) UNet++, (e) VGG_19, (f) HED, and (g) CNN.

[0154] Figure 8 From left to right in the figure are (a) original image, (b) label image, (c) algorithm of the present invention, (d) UNet++, (e) VGG_19, (f) HED, and (g) CNN.

[0155] Figure 8 Segmentation effects of different algorithms on the Crack500 dataset.

[0156] Depend on Figures 6 to 8 It can be seen that HED and CNN have the worst effects in the self-built data set, CDF data set and Crack500 data set. In the self-built data set, the imprint is identified as a crack, which affects the image segmentation result. In the CFD data set, HED and CNN are seriously disturbed by the road particle noise, and the noise particle points are identified as cracks for output, which affects the accuracy of image segmentation. On the Crack500 data set, the segmentation clarity of the two algorithms is not good. VGG_19 is second. Although it can eliminate noise interference to a certain extent on the CFD data set, it can exclude imprints on the self-built data set, and the segmentation of obvious areas on the Crack500 data set is relatively clear, there is still room for improvement. UNet++ has better segmentation effect than the first three due to the U-shaped structure setting and jump connection, but its processing clarity and accuracy are still lacking in fine crack processing. The algorithm of the present invention can better eliminate the interference of road particles and imprint noise, accurately identify the fine boundaries of cracks, and achieve better crack detail segmentation. The reason for this effect is that the algorithm of the present invention uses fuzzy mathematical morphological filtering to achieve noise reduction while retaining detail characteristics. The use of multi-scale hole convolution modules and path enhancement modules can effectively retain small features while paying attention to contextual information.

[0157] In order to objectively evaluate the accuracy of the proposed algorithm in image crack segmentation, this study used self-built data sets, Crack500 and CFD data sets for evaluation and verification. The experiment used accuracy (ACC), average pixel accuracy (MPA), average intersection over union (MIoU) and recall (Recall) for evaluation, and took the average value of each indicator to eliminate machine errors. The evaluation results were retained with 2 as the decimal point, as shown in Tables 3 to 5.

[0158] Table 3 Segmentation performance analysis of different algorithms on self-built datasets

[0159]

[0160] Table 4 Segmentation performance analysis of different algorithms on the crack500 dataset

[0161]

[0162]

[0163] Table 5 Segmentation performance analysis of different algorithms on CFD datasets

[0164]

[0165] As can be seen from Tables 3 to 5, CNN performs relatively poorly on all datasets, especially in complex scenes, where it is difficult to capture detailed crack features. The HED algorithm is better than the CNN algorithm, but it is still inferior to VGG_19 and UNet++. This shows that there is still room for improvement in the HED algorithm in feature extraction and detail recovery. Although it performs well in accuracy, it is slightly insufficient in average intersection-over-union and recall, which may be due to its over-reliance on deep networks for feature extraction, resulting in insufficient detail recovery. Unet++, as an improved version of Unet, shows better segmentation performance than CNN, HED and VGG_19, but it still does not surpass the algorithm in this paper, showing further optimization of the algorithm in architectural design.

[0166] The algorithm of the present invention has excellent segmentation performance on the self-built dataset, CRACK500 dataset and CFD dataset. On all datasets, the accuracy (96.97%) and average pixel accuracy (96.86%) of the algorithm of the present invention are the highest among all algorithms, indicating that it has a good fitting ability for the training data and is very accurate in extracting and segmenting crack features in the image. On the CRACK500 dataset, the various indicators of the algorithm of the present invention are also leading, especially the accuracy (96.37%) and average pixel accuracy (97.36%). This shows that the algorithm of this paper not only performs well on the self-built dataset, but also has strong generalization ability and can maintain high performance on different datasets. CFD datasets usually contain more complex crack scenes. On this dataset, the average intersection-over-union ratio (91.89%) and recall rate (919%) of the algorithm of the present invention are significantly higher than those of other algorithms, showing its advantages in dealing with complex and high-noise environments.

[0167] The excellent performance of this algorithm in the crack segmentation task is due to its innovative encoder and decoder design, which combines the idea of ​​hole residual and multi-scale feature extraction to better capture the details and features of the cracks. By integrating deconvolution and upsampling, the algorithm can effectively restore image details during the decoding process. In addition, its excellent performance on different datasets also demonstrates its strong generalization ability and robustness.

[0168] 5 Comparison of different cycle times

[0169] In order to explore the law of network training, the present invention conducts experiments on a self-built data set to study how different numbers of cycles correspond to different experimental results, as shown in Table 6:

[0170] Table 6 Effect analysis of different iteration times

[0171]

[0172] As shown in Table 6, with the increase of the number of network cycles, various detection effect indicators (Accuracy, MPA, MIoU, Recall) are significantly improved, especially in the first four cycles. When the number of cycles is 4, each indicator reaches a peak. As the number of cycles increases, the detection effect gradually deteriorates. Experiments show that the algorithm in this study has the best detection effect when the number of cycles is 4. Moderately increasing the number of cycles can help improve the detection effect of the model, but too many cycles may cause the performance improvement to be flat or even slightly decreased. Choosing an appropriate number of cycles is crucial for model optimization.

[0173] 6. Comparison of running time and efficiency

[0174] In order to analyze the real-time performance of various algorithms in image crack segmentation, the segmentation time statistics of CNN, HED, VGG19, Unet++ and the algorithm of the present invention are conducted on the self-built data set, Crack500 and CFD data set, and the average of the three results is taken for comparison experiments. At the same time, the number of frames per second (FPS) and the peak speed per second (FLOPS) of the image processing are observed. The experimental results are shown in Table 4:

[0175] Table 6 Comparison of running time and efficiency of different algorithms

[0176]

[0177] As can be seen from Table 6, in terms of average detection time, the algorithm of the present invention leads with a result of 0.09 seconds. In terms of FPS, the algorithm of the present invention reaches 9.39. The algorithm of the present invention performs well in terms of processing speed and real-time performance. This efficient segmentation time is due to the fact that the encoder part quickly extracts multi-scale features through multi-layer convolution and hole residual convolution, expands the receptive field without increasing the amount of calculation, improves the effect of feature extraction, and reduces the time to process each frame of the image. However, the higher FPS is attributed to the efficient decoder design, which quickly restores the image through upsampling and deconvolution operations, reducing processing delays. The second is UNet++, which is slightly different from the algorithm of the present invention, but still shows good real-time performance. The other three algorithms are relatively slow in processing speed. In terms of FLOPS, the algorithm of the present invention has a significant advantage in processing speed per second compared with the other four algorithms, reaching 2338. The reason is that the multi-scale residual hole module and the path enhancement module capture features of different scales through multi-scale convolution operations, significantly improving the FLOPS value, while restoring high-quality segmented images. Experiments show that the algorithm in this paper finds a good balance between processing speed, computational efficiency and segmentation accuracy, and is suitable for a variety of practical application scenarios.

[0178] 7 Ablation experiments

[0179] To further verify the superiority of the algorithm proposed in this study in image segmentation, comparative ablation experiments were conducted on self-built datasets and CFD datasets. The algorithm of the present invention was trained with VGG13, VGG+denoising, VGG+residual hole+denoising, VGG+enhanced path+denoising module, and objectively evaluated using ACC, accuracy, MPA, MioU, and average segmentation time. The results are shown in Table 7:

[0180] Table 7 Ablation experiment results

[0181]

[0182] As can be seen from Table 7, the algorithm of the present invention is effective. The algorithm of the present invention introduces a hole residual module and a path enhancement module into the VGG network for feature extraction, so the algorithm of the present invention can capture more image details and complex crack features, thereby improving the accuracy of segmentation. The decoder uses deconvolution and upsampling fusion processing to restore the original resolution of the image while retaining high-level feature information to ensure that the details of the cracks are retained and accurately segmented. The filtering method can retain and enhance important details while smoothing the image, which is why ACC, MPA, MioU and Recall are better. In terms of average segmentation time and FPS, the advantages of the algorithm in this paper in terms of operating efficiency are reflected. Thanks to the efficient computing framework of the algorithm, the use of more optimized algorithm flow, upsampling and deconvolution parallelism and GPU acceleration, these technologies can improve the processing speed of the algorithm and the number of frames processed per second. The data of the algorithm in this paper on FLOPS indicates its high efficiency in the number of floating-point operations per second, which comes from the careful balance between computational complexity and execution efficiency in the algorithm design. By reasonably designing the network structure and optimizing the calculation process, the algorithm in this paper can achieve efficient image segmentation processing without sacrificing accuracy.

[0183] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A crack detection method based on an improved VGG network model, characterized in that: The following steps are involved: Acquire a crack image, convert the crack image into a grayscale image, normalize the grayscale image, and then perform fuzzy morphological filtering on the normalized grayscale image to obtain an input image; Construct an improved model, wherein the improved model uses the VGG13 network as the basic network, combines residual hole convolution and path enhancement modules, and adopts an encoder and decoder architecture; Input the input image into the improved model, the improved model includes 5 layers of convolution blocks, wherein the first layer, the third layer and the fourth layer extract the basic features of the crack image through standard convolution operations, the second layer captures the multi-scale features of the crack image through the residual hole convolution module, the fifth layer is a hole convolution module, and the hole convolution module is followed by a path enhancement module; The path enhancement module uses the information of shallow features of the input image to enhance the representation capability of deep features through top-down feature transfer; The Dice coefficient is used as the loss function to measure the performance of the improved model. The gradient formula of the Dice loss function to the model prediction value is calculated. The gradient is calculated using the gradient descent method, the network weights are updated, and the loss function is minimized until the model converges.

2. The crack detection method based on the improved VGG network model according to claim 1, characterized in that: The specific process of the fuzzy morphological filtering process is as follows: The filtering process includes fuzzy expansion, fuzzy erosion, fuzzy opening and closing operations. Fuzzy expansion enhances the brightness of the image by combining each pixel in the image with the maximum value of its neighborhood. The calculation formula is: IN A⊕B (s)=sup p∈B min(U A (μ-γ),U B (γ)) Among them, U A The fuzzy set representing the original crack image, U B (γ) represents the fuzzy set of structural elements, sup represents the supremum, and min represents the minimum value; Fuzzy erosion combines each pixel in the image with the minimum value of its neighborhood to reduce the brightness of the image. The calculation formula is: Among them, inf represents the infimum, and max represents the maximum value. In the erosion operation, for each pixel in the image, the maximum value of its neighborhood and the corresponding position of the structural element is taken, and the minimum value is taken as the pixel value after erosion among the maximum values ​​obtained. In this way, the erosion operation retains the smaller pixel values ​​in the image and reduces the larger pixel values ​​in each local area; The fuzzy opening operation is used to remove noise by first performing fuzzy erosion and then fuzzy dilation. The calculation formula is: The fuzzy closing operation first performs fuzzy expansion and then fuzzy erosion to fill small holes and cracks in the image. The calculation formula is:

3. The crack detection method based on the improved VGG network model according to claim 1, characterized in that: The process of capturing features by the residual hole convolution module is as follows: The residual atrous convolution module consists of four branches, of which the convolution expansion rates of three branches are 1, 3, and 5, respectively, to capture features of different scales. The convolution operation formula is: F i (Y)=σ(W i *Y+b i ) Among them, σ represents the activation function ReLU, W i and b i are the weight and bias of the i-th convolutional layer, respectively. * indicates the convolution operation. In the entire residual dilated convolution module, 1×1 convolution is used to capture local features and reduce dimensions. 3×3 convolution with dilation rate = 1 is used for standard convolution to extract detailed features. 3×3 convolution with dilation rate = 3 is used to capture the mid-range context by expanding the receptive field. 3×3 convolution with dilation rate = 5 is used to extract broader context information. After processing through the dilated convolutions of different branches, the outputs of all branches are connected, and then the multi-scale features are fused through another 1×1 convolution. The formula is: F(Y)=σ(W f *concat(F1(Y),F2(Y),F3(Y),F4(Y))+b f ) Among them, W f and b f are the weight and bias of the 1×1 convolution layer, respectively. F(Y) represents the fused features after multi-scale convolution. The final output is connected through the residual connection. The input feature map passes through the 1×1 convolution layer and is then added to the output of the improved residual hole convolution module to achieve feature extraction. The calculation formula is: I=Y''+F(Y) Among them, Y'' represents the result of processing through a 1×1 convolutional layer, F(Y) represents the features processed through a 1×1 convolutional layer, and the feature map processed by the hole residual module can enhance the multi-scale context while retaining the original features.

4. The crack detection method based on the improved VGG network model according to claim 3 is characterized in that: The residual hole convolution module performs convolution processing by adjusting the receptive field size and the convolution kernel size fusion method, adjusting the receptive field of the convolution kernel with a hole rate of 2 and 5 to 5 and 10, and adopting spatially separable convolution; Use 5×1 and 1×5 concatenated convolution modules instead of a 5×5 module, add 1×1 convolution after each branch to adjust the linear excitation and dimension, and use global average pooling to add and average all pixel values ​​in the feature map to obtain the feature vector. The calculation formula is: Among them, H represents the height of the feature map, M represents the width of the feature map, and Y ab represents the pixel value in the ath row and bth column of the feature map, Represents the sum of all pixel values ​​in the feature map. By performing a double summation operation on all pixel values ​​in the feature map, the sum of all pixel values ​​in the feature map is calculated. Represents the normalization factor.

5. The crack detection method based on the improved VGG network model according to claim 1, characterized in that: The process of enhancing the features of the path enhancement module is as follows: The path enhancement module extracts feature maps from the shallow and deep layers of the network respectively. The shallow feature map contains details and positioning information, and the deep feature map contains semantic information. low Upsample and add it element by element with the deep feature map to get the fused feature result, and then use the convolution layer to process the fused result. The calculation formula is: i∈[1,G hight ],j∈[1,G hight ] The fusion result is further processed using the convolutional layer. The formula is: Among them, c is the size of the convolution kernel, ω nm is the convolution kernel weight, and b is the bias term.

6. The crack detection method based on the improved VGG network model according to claim 1, characterized in that: The formula for the Dice loss function is: in, Indicates The predicted value of pixels, Indicates The true label of pixels, O represents the total number of pixels, In the back propagation, the loss function is calculated for the predicted value The gradient of is: make and The simplified formula of Dice loss function is: Then, yes Derivative, use the chain rule, calculate the numerator and denominator, and get the Dice loss function for the predicted value The gradient formula is: By substitution and simplification, the final gradient formula is obtained, and the gradient is calculated using the gradient descent method to update the network weights and minimize the loss function until the model converges.

Images