Lightweight tunnel crack segmentation identification method based on improved YOLOv8

Abstract

The invention discloses a lightweight tunnel crack segmentation identification method based on improved YOLOv8, and the method specifically comprises the following steps: 1, obtaining an image data set of a tunnel crack, and carrying out the preprocessing of the data set; step 2, carrying out expansion and enhancement on the preprocessed data set; step 3, labeling the data set processed in the step 2; 4, the YOLOv8 model is improved; 5, training the model improved in the step 4; and step 6, carrying out training test verification on the model trained in the step 5. According to the invention, the problems of low segmentation efficiency and low crack detection precision of the existing crack segmentation method are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision image segmentation and relates to a lightweight tunnel fissure segmentation and recognition method based on improved YOLOv8. Background Art

[0002] The segmentation and identification of tunnel cracks is a key task in tunnel engineering, which is directly related to the safety and long-term performance of the tunnel structure. At present, tunnel crack segmentation usually relies on manual inspection or automated methods based on traditional image processing. However, these methods show obvious limitations in practical applications, especially when the crack background is complex and the illumination is uneven, the crack edges are prone to blur, resulting in inaccurate segmentation results. Traditional methods based on edge detection, threshold segmentation and other technologies are difficult to cope with the diversity of cracks in complex environments, and often have problems of missed detection and false detection.

[0003] With the development of deep learning technology, image segmentation methods based on convolutional neural networks (CNN) have gradually been applied to crack segmentation. In particular, instance segmentation models based on the YOLO series can accurately locate crack positions in complex scenes, making the crack segmentation process no longer require manual intervention. However, existing methods have also exposed some problems in practical applications. First, the amount of calculation is large and the running speed is slow. Many deep learning models (such as YOLOv8) have high computational complexity, especially when processing high-resolution tunnel images, which is slow and not conducive to real-time application; second, the accuracy is low. The segmentation accuracy of existing methods is still not ideal and cannot meet the needs of efficient and accurate crack detection. Summary of the invention

[0004] The purpose of the present invention is to provide a lightweight tunnel crack segmentation and recognition method based on improved YOLOv8, which solves the problems of low segmentation efficiency and low crack detection accuracy in existing crack segmentation methods.

[0005] The technical solution adopted by the present invention is a lightweight tunnel crack segmentation and recognition method based on improved YOLOv8, which specifically includes the following steps: Step 1, obtaining an image dataset of tunnel cracks and preprocessing the dataset; Step 2, expanding and enhancing the preprocessed data set; Step 3, labeling the data set processed in step 2; Step 4, improve the YOLOv8 model; Step 5, training the improved model in step 4; Step 6: Perform training, testing and verification on the model trained in step 5.

[0006] The present invention is also characterized in that: The specific process of step 1 is: Step 1.1, download the crack segmentation dataset from the Roboflow platform; Step 1.2, using the Gaussian filter denoising algorithm to process the image in the data set of step 1.1, removing random noise in the image while retaining the clarity of the crack edge; Step 1.3, resize the images processed in step 1.2 to the same size.

[0007] The specific process of step 2 is: Step 2.1, input the preprocessed data set into the WGAN-RE model for iterative training, and use the Fre´chet Inception Distance and Inception Score performance indicators to determine whether the model converges during the training process; Step 2.2: Based on the judgment result of step 2.1, when the generator and discriminator of the WGAN-RE model converge, the data set is expanded with an expansion ratio of 1:2 to form an enhanced data set.

[0008] In step 2.1, FID is expressed as:

[0009] in, and Respectively represent the mean vectors of the real image and the generated image in the feature space; and Represent the covariance matrix of the real image and the generated image in the feature space respectively; x Represents the distribution of real images; g Represents the distribution of real images; Among them, IS is expressed as:

[0010] Among them, X~Pg represents the samples drawn from the distribution of generated images x ; Indicates the Inception network for the image x The predicted class distribution; is the average category distribution of the generated images as a whole; is the category distribution With the overall category distribution The KL divergence between Represents the expectation for generating image samples.

[0011] The specific process of step 3 is: Step 3.1: Use the “Polygon” tool in the image annotation software Labelme to annotate the cracks, save them in JSON format, and then use the Python script to convert the annotated data into a TXT file in YOLO format; Step 3.2: For the dataset annotated in step 3.1, divide the dataset into a training set and a validation set in a ratio of 8:2. The divided datasets are stored in different folders according to categories, and each folder contains corresponding images and annotation files.

[0012] The specific process of step 4 is: Step 4.1, Initialize the module. When the Faster_Block_EMA class is initialized, set the module's input channel number, output channel number, MLP ratio, DropPath probability, and layer scaling value; Step 4.2, adjust the number of channels of the feature map obtained after the convolution operation in the FasterNet_EMA module; if the number of input channels is inconsistent with the number of channels processed inside the module, the number of channels will be adjusted through 1x1 convolution to ensure that the number of channels of the input feature map is consistent with the expected number and matches the processing requirements of the module; Step 4.3, the input feature map first passes through the Partial_conv3 module for spatial mixing, and then the feature map enters the MLP module for nonlinear transformation to further enhance the feature representation; Step 4.4, the feature map after MLP processing is regularized by DropPath, and then the EMA attention mechanism is applied for weighted processing; Step 4.5, adding the feature map processed by step 4.4 to the original input feature to retain the input feature and output the final result; Step 4.6, based on the feature map output in step 4.5, use the LADH detection head (Lightweight Asymmetric Detection Head) to replace the original detection head in YOLOv8 for the final segmentation task.

[0013] Step 4.1 The specific process is: Step 4.1.1, set the core properties of the module, including the output feature dimension dim, the MLP ratio mlp_ratio and the number of divisions of the partial convolution n_div; In step 4.1.2, we initialize the multi-layer perceptron (MLP), partial convolution (Partial_conv3), and EMA attention mechanism, which constitute the main computational components of the module.

[0014] The specific process of step 4.3 is: Step 4.3.1, the input feature map is spatially mixed by the Partial_conv3 convolution operation. The convolution operation is performed in different regions according to the parameters n_div and pconv_fw_type to achieve spatial mixing of feature maps. Among them, pconv_fw_type is a parameter used to control the forward propagation behavior of partial convolution. In step 4.3.2, the spatially mixed feature map enters the MLP module for nonlinear transformation, and two 1x1 convolutions expand the feature dimension to mlp_hidden_dim and then reduce it back to dim.

[0015] The specific process of step 4.4 is: Step 4.4.1, the feature map processed by MLP is discarded through DropPath, and then enters the EMA attention mechanism for further weighting; Step 4.4.2, define input parameters and group them, accept the input channel number channels and grouping factor factor, which is used to determine the number of groups of input data. Use assert statement to verify whether the number of channels in each group is greater than 0; Step 4.4.3, initialize the components used to process the feature map, including the Softmax layer, the adaptive average pooling layer, the group normalization layer, and the 1x1 and 3x3 convolution layers; Step 4.4.4, in the forward propagation, first obtain the shape of the input feature map, and then rearrange the feature map according to the number of groups; Step 4.4.5, perform adaptive average pooling in the horizontal and vertical directions on the reconstructed feature map to obtain feature map representations in two directions, and fuse them through a 1x1 convolutional layer; Step 4.4.6, multiply the feature map after pooling in step 4.4.5 and the feature map reconstructed in step 4.4.4 element by element, and process it through the Sigmoid activation function and the group normalization layer, and then use 3x3 convolution to further extract features, and the generated feature map , the feature extraction process is expressed as:

[0016] in, x h It is the feature map after the pooling operation; x w is the reconstructed feature map; Step 4.4.7, perform Softmax operation on the first set of feature maps processed in step 4.4.6 after using adaptive average pooling to obtain the attention weight matrix of the first set of feature maps x 11； Step 4.4.8, use adaptive average pooling and Softmax operation on the second set of feature maps processed in step 4.4.6 to obtain the attention weight matrix of the second set of feature maps x 21 , and the final attention weight is obtained by the attention weights of the two sets of feature maps. The calculation formula is as follows:

[0017] Among them, A is the final attention weight; x 11 is the attention weight matrix of the first set of feature maps obtained in step 4.4.7; x 21 is the attention weight matrix of the second set of feature maps in step 4.4.8; x 12 is the feature matrix obtained by flattening the second set of feature maps; x 22 is the feature matrix obtained by flattening the first set of feature maps.

[0018] Step 4.4.9, use the final attention weight obtained in step 4.4.8 to multiply the feature map obtained in step 4.4.1, and then resize the result to the original shape, return the feature map processed by the attention mechanism, and use it for subsequent network layers to enhance the feature extraction ability of the model. The formula is expressed as:

[0019] in, x It is the feature map after adjusting the shape; A is the final attention weight obtained in step 4.4.8; DropPath (x) It is the feature map after the path is discarded; b is the batch size; groups is the number of groups; h and w The height and width of the feature map after adjusting the shape; Step 4.4.10, add the feature map processed by step 4.4.9 and the feature map after spatial mixing in step 4.3.1 to form the final output, which is expressed as:

[0020] The specific process of step 4.5 is: Step 4.5.1, define a Segment_LADH module; Step 4.5.2, define the basic properties of the model in the module initialization function, including the number of categories nc, the number of masks nm, the number of prototypes npr, and the number of channels ch of the input feature map; Step 4.5.3, calculate the number of output channels c4 of each convolutional layer and depth convolutional layer. The specific formula is as follows:

[0021] Step 4.5.4, construct a convolution module group cv4 for processing the input feature map, each module includes two depth-separable convolutions DSConv and one standard convolution Conv; Step 4.5.5, during the forward propagation process, use the Proto module to process the first layer of the input feature map to generate the mask prototype p; Step 4.5.6, use the corresponding cv4 module to process each input feature map, combine the depth separable convolution DSConv and the ordinary convolution Conv, perform step-by-step feature extraction on the feature map, and finally generate the mask coefficient. Use the torch.cat function to concatenate the mask coefficients of all layers in the third dimension to obtain the final mask coefficient mc; Step 4.5.7, call the LADH detection head to process the input feature map and generate the final prediction result x; Step 4.5.8, determine whether the model is in training mode according to the internal attribute self.training of nn.Module in PyTorch. If self.training is true, return the prediction result x, mask coefficient mc and mask prototype p; If self.training is false, different spliced ​​outputs are returned depending on whether self.export is in export mode. The spliced ​​output contains the prediction results and mask coefficients for the final image segmentation result generation.

[0022] The specific process of step 5 is: Use the training set divided in step 3.2 to train the improved YOLOv8 model in step 4. During the training process, Adam is used as the optimizer and CIoU is used as the loss function to ensure that the model converges quickly.

[0023] The beneficial effects of the present invention are that, compared with the classic tunnel crack segmentation method, the present invention expands the data set by adopting the Wasserstein generative adversarial network enhanced by residual blocks (WGAN-RE), enriches the data set, and enhances the generalization ability of the model; the lightweight module of FasterNet is used to partially replace the standard convolution in the original C2f Bottleneck; in the improved C2f_Faster module, the EMA attention mechanism is added; finally, the LADH lightweight detection head is used to replace the original detection head in YOLOv8; thereby improving the detection accuracy of the model for the crack area, reducing the computational complexity while maintaining the detection accuracy, and having strong robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a model overall structure diagram of the lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 of the present invention; Figure 2 It is a structural diagram of the C2f_Faster_EMA module in the lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 of the present invention; Figure 3 It is a structural diagram of the PConv module in the lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 of the present invention; Figure 4 It is a structural diagram of the EMA module in the lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 of the present invention; Figure 5 It is a structural diagram of the LADH module in the lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 of the present invention; Figure 6 This is a photo of the results of the YOLOv8n model in the tunnel crack segmentation task; Figure 7 This is a photo of the results of the YOLOv8s model in the tunnel crack segmentation task; Figure 8 This is a photo of the results of the YOLOv9c model in the tunnel crack segmentation task; Fig. 9 This is a photo of the results of the lightweight tunnel crack segmentation and recognition method based on the improved YOLOv8 in the tunnel crack segmentation task. DETAILED DESCRIPTION

[0025] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1 The present invention is based on an improved YOLOv8 lightweight tunnel crack segmentation and recognition method, comprising the following steps: Step 1, obtaining an image dataset of tunnel cracks and preprocessing the dataset, the preprocessing includes image denoising and size standardization; Step 1.1, download the cosmic crack segmentation dataset from the Roboflow platform. These images show different crack types, including longitudinal cracks, transverse cracks, and cracks in complex backgrounds; Step 1.2, use the Gaussian filter denoising algorithm to process the image to remove random noise in the image while retaining the clarity of the crack edge; In step 1.3, all images are resized to 640x640 to ensure data consistency to meet the model input requirements.

[0027] Step 2: For the images in the preprocessed dataset, the Wasserstein generative adversarial network enhanced with residual blocks (WGAN-RE) is used to expand the dataset, enrich the dataset, and enhance the generalization ability of the model; WGAN mainly consists of a generator and a discriminator. In the generator, 100-dimensional random noise is first generated. Then, the noise is reshaped into data of size 4 x 4 x 512 using a deconvolution operation with a kernel size of 4 x 4 and a stride of 1. The data is sequentially input into four deconvolution layers with a kernel size of 4 and a stride of 2, and a new image sample of size 64 x 64 x 3 is output. In the convolution layer, the image of size 64 x 64 x 3 is sequentially input into four convolution layers with a kernel size of 4 and a stride of 2 to obtain data of size 4 x 4 x 512. Finally, the data is converted into a scalar for judging the authenticity of the input image. The generator and discriminator continue to optimize themselves during the training process until the discriminator cannot tell whether the input is from a real or generated image.

[0028] WGAN is suitable for generating crack images of size 640 x 640. However, when the size of the generated image increases, there will be problems such as the convergence speed of the network decreases and the generated crack images are not realistic enough. To solve the above problems, WGAN-RE is formed by introducing residual blocks and efficient channel attention modules (ECA) to WGAN.

[0029] The core parts of the generator and discriminator of WGAN-RE are the DRE (Deconvolution enhanced by Residual Block and ECA) module and the CRE (Convolution enhanced by Residual Block and ECA) module, respectively. In the CRE module, the feature map F of size H x W x C is first converted to F1 of size H / 2 x W / 2 x 2C through a convolution layer with a kernel size of 4, and then F1 is input into the ECA module. In ECA, F1 passes through the global average pooling layer, the adaptive one-dimensional convolution layer, and the Sigmoid activation function in sequence to obtain the attention vector. The modified feature map F2 is obtained by multiplying the attention vector with F1. Finally, F2 is input into the convolution layer with a kernel size of 3 and added to the output of the residual edge to obtain the output feature map F C The structures of DRE and CRE are basically the same, the only difference is that the first convolutional layer with kernel size 4 is replaced by a deconvolutional layer in DRE.

[0030] Step 2.1, input the preprocessed data set into the generator of the WGAN-RE model for iterative training, and use the Fre´chet Inception Distance (FID) and Inception Score (IS) performance indicators to determine whether the generator converges during the training process. FID can be expressed as:

[0031] in and Respectively represent the mean vectors of the real image and the generated image in the feature space; and Represent the covariance matrix of the real image and the generated image in the feature space respectively; x Represents the distribution of real images; g Represents the distribution of real images.

[0032] The IS can be expressed as:

[0033] Where X~Pg represents the samples drawn from the distribution of generated images x ; Indicates the Inception network for the image x The predicted class distribution; is the average category distribution of the generated images as a whole; is the category distribution With the overall category distribution The Kullback-Leibler divergence (KL divergence) between Represents the expectation for generating image samples.

[0034] By observing the logs after training, if FID tends to decrease and stabilize, and IS tends to increase and stabilize, it can be judged that the generator has basically converged; if the discriminator loss tends to 0.5, it means that the discriminator maintains a balance in the classification of real images and generated images, that is, it cannot completely distinguish between generated images and real images, and it can be judged that the discriminator has basically converged. If both the generator and the discriminator have reached convergence, the crack dataset can be expanded through the trained WGAN-RE network; if the generator or the discriminator has not reached convergence, increase the number of training iterations and restart the training.

[0035] Step 2.2: According to the judgment criteria of step 2.1, when the generator and discriminator of the WGAN-RE model converge, the data set is expanded with an expansion ratio of 1:2 (that is, the ratio of the number of images preprocessed in step 1.3 to the number of images generated in step 2.1 is 1:2) to form an enhanced data set.

[0036] Step 3, annotating the tunnel crack dataset enhanced in step 2.2 and dividing the dataset; Step 3.1: Use the “Polygon” tool in the image annotation software Labelme to annotate the cracks, save them in JSON format, and then use a Python script to convert the annotated data into a TXT file in YOLO format for model training.

[0037] Step 3.2: For the dataset annotated in step 3.1, divide the dataset into a training set and a validation set at a ratio of 8:2. The divided datasets are stored in different folders by category, and each folder contains corresponding images and annotation files for subsequent model training and evaluation.

[0038] Step 4: Improve YOLOv8 using FasterNet. Replace the standard convolution in the original C2f Bottleneck with some convolutions in the lightweight module of FasterNet; add the EMA (Exponential Moving Average) attention mechanism to the improved C2f_Faster module; The present invention introduces a lightweight Faster_EMA network structure in YOLOv8, which has the advantages of faster detection speed and higher detection accuracy.

[0039] like Figure 1As shown in the figure, the overall network structure can be divided into three parts: backbone network (Backbone), neck (Neck) and head (Head). The backbone network accepts an input image of size 640×640, and the input image is gradually extracted through the multi-layer convolution operation of the backbone network. The backbone network uses multiple C2f_Faster_EMA modules, which combine the lightweight design of FasterNet and the EMA attention mechanism to efficiently extract the spatial and channel features of the image. After passing through the backbone network, the feature maps of different layers are fused through "splicing" and "upsampling" operations. Through the fusion of features of different scales, the model can capture contextual information more comprehensively, thereby improving the final prediction performance. The C2f_Faster_EMA module is continued to be used in the neck structure, which can not only reduce the amount of calculation through a lightweight structure, but also improve the global context perception ability of the model through the EMA mechanism, helping the model to better understand the image content. The last part of the network is the LADH detection head, which is used to process the feature map and generate the final detection or segmentation output. In the segmentation task, the LADH detection head generates mask coefficients and segmentation results based on the feature map from the Neck part. The design of the LADH module reduces the computational complexity through an asymmetric lightweight structure, making it suitable for real-time applications and resource-constrained devices.

[0040] The C2f module of the improved YOLOv8 model backbone is integrated with the Faster_EMA module to form the C2f_Faster_EMA module. In addition, the FasterNet_EMA module is the main feature extraction module. Each FasterNet_EMA block is composed of a partial convolution (PConv) and two subsequent point-wise convolutions (PWConv). These three parts are combined to form an inverted residual block, and an EMA attention mechanism module is added at the end to enhance the network's feature extraction capabilities. See the C2f_Faster_EMA module structure. Figure 2 .

[0041] like Figure 3 As shown in Figure 1, PConv is a new convolution operation proposed by FasterNet. Unlike previous convolution methods that perform convolution on all channels of the input feature, PConv only performs conventional convolution on a part of the input feature channels to extract features, and does not process other channels. For the continuity of memory access, the first or last continuous c p channels to represent the entire feature map. Therefore, PConv greatly compresses the number of parameters and computations, and still maintains good feature extraction capabilities. As shown in Formula 1, when c pWhen it is 1 / 4 of c, compared with conventional convolution, the FLOPs (Floating Point Operations) of PConv is only 1 / 16 of that of conventional convolution.

[0042]

[0043] in, is the computational complexity or complexity of the traditional convolution operation; is the computational complexity or complexity of the partial convolution; is the height of the input feature map; is the width of the input feature map; is the size of the convolution kernel; is the number of channels of the input feature map; c p is the number of channels of the input feature map in the partial convolution.

[0044] This paper introduces the EMA attention mechanism in the FasterNet Block module, which improves the pixel-level attention of high-level feature maps without reducing the channel dimension. For any given input feature map X, the EMA channel dimension is divided into G sub-features , G≪C is taken, and the learned attention weights strengthen the representation of sub-features in all attention mechanisms. EMA uses three parallel paths to extract grouped feature maps, two parallel paths are 1×1 convolution branches, and the third parallel path is 3×3 convolution branches. In the cross-space learning stage, two-dimensional global average pooling is used to encode global spatial information. The pooling operation formula is:

[0045] in, Represents the global average value of the cth channel of the feature map, which is the output value of the channel after the pooling operation; is the height of the input feature map; is the width of the input feature map; is the cth channel of the feature map at position The pixel value of The Softmax function is used to fit the output of the average pooling, and the output of each branch is multiplied by the matrix dot product operation to obtain the spatial attention maps of the two branches. Finally, the output feature information of the two branches is mapped to two sets of spatial attention weight values, and the Sigmoid function sends it to the subsequent module.

[0046] The three branches of the EMA attention mechanism integrate the advantages of channel attention and spatial attention, while capturing global channel dependencies and local spatial features, and obtaining more comprehensive feature information between channel and spatial dimensions. Its structure is as follows: Figure 4As shown in the figure, the attention mechanism EMA is introduced and fused into the FasterNet module, which weakens the weight of background information to improve the model's ability to focus on multi-scale target features and more accurately obtain the feature information of the target to be detected in the image.

[0047] Step 4.1, Initialize the module. When the Faster_Block_EMA class is initialized, set the key parameters of the module, such as the number of input channels, the number of output channels, the MLP ratio, the DropPath probability, and the layer scaling value. The module includes partial convolution, MLP, and EMA attention mechanisms, which together constitute the core architecture of the module; Step 4.1 The specific steps are: Step 4.1.1, set the core properties of the module, including the output feature dimension dim, the MLP ratio mlp_ratio and the number of divisions of the partial convolution n_div; In step 4.1.2, we initialize the multi-layer perceptron (MLP), partial convolution (Partial_conv3), and EMA attention mechanism, which constitute the main computational components of the module.

[0048] Step 4.2, adjust the number of channels of the feature map obtained after the convolution operation in the FasterNet_EMA module. If the number of input channels is inconsistent with the number of channels processed inside the module, the number of channels will be adjusted through 1x1 convolution to ensure that the number of channels of the input feature map is consistent with the expected number and matches the processing requirements of the module; Step 4.3, spatial mixing and feature processing. The input feature map first passes through the Partial_conv3 module for spatial mixing. This step enhances the interaction of spatial information through partial convolution of the feature map. Then the feature map enters the MLP module for nonlinear transformation to further enhance the feature representation. The specific steps of step 4.3 are: Step 4.3.1, the input feature map is spatially mixed by the Partial_conv3 convolution operation. The partial convolution here performs convolution operations in different regions according to the parameters n_div and pconv_fw_type to achieve spatial mixing of feature maps; Among them, pconv_fw_type is a parameter used to control the forward propagation behavior of partial convolution. In partial_conv3, it determines the specific way of partial convolution operation. Depending on the set pconv_fw_type value, partial convolution may adopt different strategies when processing input feature maps, such as how to divide the input feature map, how to perform convolution operations on the divided parts, etc.

[0049] In step 4.3.2, the spatially mixed feature map enters the MLP module for nonlinear transformation. These two 1x1 convolutions expand the feature dimension to mlp_hidden_dim and then reduce it back to dim.

[0050] Step 4.4, path drop and attention mechanism. The feature map processed by MLP is regularized by DropPath and then weighted by EMA attention mechanism. These steps aim to improve the generalization ability of the model by path drop and focus on important features by attention mechanism.

[0051] The specific steps of step 4.4 are: Step 4.4.1, the feature map processed by MLP is discarded through DropPath, and then enters the EMA attention mechanism for further weighting; Step 4.4.2, define input parameters and group them, accept the input channel number channels and grouping factor factor, which is used to determine the number of groups of input data. Use assert statement to verify whether the number of channels in each group is greater than 0; In step 4.4.3, various components for processing feature maps are initialized, including the Softmax layer, the adaptive average pooling layer, the group normalization layer, and the 1x1 and 3x3 convolution layers.

[0052] Step 4.4.4, in the forward propagation, first obtain the shape of the input feature map, and then rearrange the feature map according to the number of groups so that each group of feature maps can be processed independently.

[0053] In step 4.4.5, adaptive average pooling is performed on the reconstructed feature map in the horizontal and vertical directions respectively to obtain the feature map representation in two directions, and then fused through a 1x1 convolutional layer.

[0054] In step 4.4.6, the feature map after pooling in step 4.4.5 and the feature map reconstructed in step 4.4.4 are multiplied element by element, and then processed by the Sigmoid activation function and the group normalization layer, and then the 3x3 convolution is used to further extract features. The feature extraction process can be expressed as:

[0055] in x h It is the feature map after the pooling operation; x w is the reconstructed feature map; x conv is the feature map generated after processing in step 4.4.6.

[0056] Step 4.4.7, perform Softmax operation on the first set of feature maps processed in step 4.4.6 after using adaptive average pooling to obtain the attention weight matrix of the first set of feature maps x 11 .

[0057] Step 4.4.8, use adaptive average pooling and Softmax operation on the second set of feature maps processed in step 4.4.6 to obtain the attention weight matrix of the second set of feature maps x 21 , and the final attention weight is obtained by the attention weights of the two sets of feature maps. The calculation formula is as follows:

[0058] Where A is the final attention weight; x 11 is the attention weight matrix of the first set of feature maps obtained in step 4.4.7; x 21 is the attention weight matrix of the second set of feature maps in step 4.4.8; x 12 is the feature matrix obtained by flattening the second set of feature maps; x 22 is the feature matrix obtained by flattening the first set of feature maps.

[0059] Step 4.4.9, use the final attention weight obtained in step 4.4.8 to multiply the feature map obtained in step 4.4.1, and then resize the result to the original shape. Return the feature map processed by the attention mechanism for use by subsequent network layers to enhance the feature extraction ability of the model. Its formula is expressed as:

[0060] in, x It is the feature map after adjusting the shape; A is the final attention weight obtained in step 4.4.8; DropPath (x) It is the feature map after the path is discarded; b is the batch size; groups is the number of groups; h and w is the height and width of the feature map after reshaping.

[0061] Step 4.4.10, add the feature map processed by step 4.4.9 and the feature map after spatial mixing in step 4.3.1 to form the final output. The formula can be expressed as:

[0062] In step 4.5, the processed feature map is added to the original input to preserve the input features and output the final result. If layer scaling is enabled, the channels of the feature map are also scaled, resulting in more detailed feature control. The final result is passed to the next layer of the network for further processing.

[0063] Step 5: Based on the feature map output in step 4.5, the LADH lightweight detection head is used to replace the original detection head in YOLOv8 for the final segmentation task. The feature map is compressed and enhanced through the LADH module, and asymmetric compression is used to maintain key features while reducing computational complexity; Figure 5 The LADH detection head structure is outlined. By separating the network based on the task type, three different channels are utilized to perform related tasks. In order to expand the receptive field and increase the task parameters of the IoU branch, three types of convolution are adopted to reduce the features along the channel dimension. In the LADH network, 3×3 depthwise convolution (DWConv) is used instead of the traditional 3×3 convolution. The advantage of DWConv over standard convolution is that it significantly reduces the number of parameters by decomposing the convolution operation into depthwise separable convolution and pointwise convolution.

[0064] This method further reduces model parameters and improves model inference speed. Using 3×3 depthwise separable convolutional layers to separate classification and bounding box tasks prevents over-extension of the two tasks, because positive samples matching the two tasks have relatively small correlation losses. By replacing the decoupling head in the YOLOv8 network with LADH-Head, the model parameters are significantly reduced while improving detection accuracy.

[0065] Step 5.1, define a Segment_LADH module, which inherits from Detect_LADH. Based on the LADH design, create a new segmentation detection head based on the existing detection head to improve the crack image segmentation task; Step 5.2, define the basic properties of the model in the module initialization function, including the number of categories (nc), the number of masks (nm), the number of prototypes (npr), and the number of channels of the input feature map (ch). These parameters determine the overall architecture and function of the model; Step 5.4, calculate the number of output channels c4 of each convolutional layer and depth convolutional layer. This value depends on the number of channels ch[0] of the input feature map and the number of masks nm to ensure that the generated feature map is suitable for both subsequent processing and mask generation; the number of output channels c4 can be expressed as:

[0066] Step 5.5, build a convolution module group cv4 for processing the input feature map. Each module contains multiple convolution operations, including two depth-separable convolutions (DSConv) and one standard convolution (Conv). The design goal of these modules is to reduce computational complexity while maintaining model performance; In step 5.6, during the forward propagation, the first layer of the input feature map is processed using the Proto module to generate mask prototypes p. These prototypes are used for subsequent mask generation to ensure that the model can learn more diverse features.

[0067] In step 5.7, each input feature map is processed using the corresponding cv4 module, combined with deep separable convolution (DSConv) and ordinary convolution to gradually extract the feature map and finally generate the mask coefficient. The torch.cat function is used to concatenate the mask coefficients of all layers in the third dimension to obtain the final mask coefficient mc.

[0068] Step 5.8, call the LADH detection head to process the input feature map and generate the final prediction result x.

[0069] Step 5.9, determine whether the model is in training mode based on the internal attribute self.training of nn.Module in PyTorch. If self.training is true, return the predicted result x, mask coefficient mc and mask prototype p. These outputs will be used for loss calculation in training; If self.training is false, different concatenated outputs are returned depending on whether it is in export mode (self.export). The concatenated output contains the prediction results and mask coefficients for the final image segmentation result generation.

[0070] Step 6: Use the training set divided in step 3.2 to train the improved YOLOv8 model in step 5. During the training process, Adam is used as the optimizer and CIoU is used as the loss function to ensure that the model converges quickly; Step 6.1, load the configuration file of the YOLOv8 improved model and initialize the model parameters. Load the pre-trained weights YOLOv8n.pt to improve the training efficiency of the model. Then, specify the dataset path for crack image segmentation, run the training script, start the model training, and save the generated results in the specified project path for subsequent model evaluation and use; Step 7: Test the model on the validation set divided in step 3.2 to compare the crack segmentation performance of different models under the same configuration conditions.

[0071] Example 2 The results of the model comparison experiment are shown in Table 1. According to the results shown in Table 1, the improved model proposed in the present invention is significantly better than other algorithm models and has the highest detection accuracy among all models. The bounding box mAP and mask mAP reach 82.1% and 67.6% respectively. While ensuring the accuracy, the number of parameters and the amount of calculation of the model are significantly reduced.

[0072] Table 1 Model comparison experimental results

[0073] Figure 6 This is a photo of the results of the YOLOv8n model in the tunnel crack segmentation task; Figure 7 This is a photo of the results of the YOLOv8s model in the tunnel crack segmentation task; Figure 8 This is a photo of the results of the YOLOv9c model in the tunnel crack segmentation task; Fig. 9 This is a photo of the results of the lightweight tunnel crack segmentation and recognition method based on the improved YOLOv8 in the tunnel crack segmentation task. Figure 6~Figure 9 It can be seen that the method of the present invention performs well in actual processing of crack morphology and background interference. The outline of the crack is finer, especially the curved part of the crack, which is closer to the real crack morphology.

[0074] Example 3 In order to verify the optimization effect of the improved content on the model, the present invention carried out 4 groups of ablation experiments on the verification set divided in step 3.2 without changing the experimental environment. The ablation experiment results are shown in Table 2, where A represents the use of FasterNet Block to replace the Bottleneck module of C2f in the original YOLOv8 network; B represents the addition of the EMA attention mechanism to the FasterNet Block on the basis of A; C represents the use of the LADH lightweight detection head to replace the original detection head on the basis of B.

[0075] Table 2 shows the performance comparison of the improved YOLOv8 crack segmentation network on the same dataset after performance enhancement. It is not difficult to see that after replacing the Bottleneck module of C2f in the original YOLOv8 network with FasterNet Block, although the bounding box mAP and mask mAP decreased by 1.2% and 1.3% respectively, the computational complexity decreased by 2.9GFLOPs; on this basis, the EMA attention mechanism was added to FasterNet Block, and the bounding box mAP and mask mAP increased by 2.4% and 2.5% respectively, and the computational complexity of the model only increased by 0.2GFLOPs; on this basis, after replacing the original detection head with the LADH lightweight detection head, the bounding box mAP and mask mAP increased by 1.2% and 0.8% respectively, and the model computational complexity was further reduced by 2.9FLOPs; through performance enhancement, the bounding box mAP and mask mAP were finally increased by 2.4% and 2% respectively, and the model computational complexity was reduced by 4.5GFLOPs.

[0076] The experimental environment uses the Linux Ubuntu 22.04 operating system, the graphics card model is NVIDIA GeForce RTX4090, 24GB video memory, the processor is 12 vCPU Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz, the deep learning framework is PyTorch 2.4.0, the GPU acceleration library is CUDA 12.1, the programming language is Python, and the interpreter is Python 3.9. During training, the input image size is 640 x 640, each model training cycle is 200 rounds, the Batch-Size is 64, the SGD optimizer is used, the initial learning rate is 0.01, and the final learning rate is 0.0001.

[0077] Table 2 Ablation experiment results

Claims

1. A lightweight tunnel crack segmentation and recognition method based on improved YOLOv8, characterized in that: The specific steps include: Step 1, obtaining an image dataset of tunnel cracks and preprocessing the dataset; Step 2, expanding and enhancing the preprocessed data set; Step 3, labeling the data set processed in step 2; Step 4, improve the YOLOv8 model; Step 5, training the improved model in step 4; Step 6: Perform training, testing and verification on the model trained in step 5.

2. According to the lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 1, it is characterized in that: The specific process of step 1 is as follows: Step 1.1, download the crack segmentation dataset from the Roboflow platform; Step 1.2, using the Gaussian filter denoising algorithm to process the image in the data set of step 1.1, removing random noise in the image while retaining the clarity of the crack edge; Step 1.3, resize the images processed in step 1.2 to the same size.

3. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 2 is characterized in that: The specific process of step 2 is: Step 2.1, input the preprocessed data set into the WGAN-RE model for iterative training, and use the Fre´chet Inception Distance and Inception Score performance indicators to determine whether the model converges during the training process; Step 2.2: Based on the judgment result of step 2.1, when the generator and discriminator of the WGAN-RE model converge, the data set is expanded with an expansion ratio of 1:2 to form an enhanced data set.

4. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 3 is characterized in that: In step 2.1, FID is expressed as: in, and Respectively represent the mean vectors of the real image and the generated image in the feature space; and Represent the covariance matrix of the real image and the generated image in the feature space respectively; x Represents the distribution of real images; g Represents the distribution of real images; Among them, IS is expressed as: Among them, X~Pg represents the samples drawn from the distribution of generated images x ; Indicates the Inception network for the image x The predicted class distribution; is the average category distribution of the generated images as a whole; is the category distribution With the overall category distribution The KL divergence between Represents the expectation for generating image samples.

5. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 4 is characterized in that: The specific process of step 3 is as follows: Step 3.1: Use the "Polygon" tool in the image annotation software Labelme to annotate the cracks, save them in JSON format, and then use a Python script to convert the annotated data into a TXT file in YOLO format; Step 3.2: For the dataset annotated in step 3.1, divide the dataset into a training set and a validation set in a ratio of 8:

2. The divided datasets are stored in different folders according to categories, and each folder contains corresponding images and annotation files.

6. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 5 is characterized in that: The specific process of step 4 is as follows: Step 4.1, Initialize the module. When the Faster_Block_EMA class is initialized, set the module's input channel number, output channel number, MLP ratio, DropPath probability, and layer scaling value; Step 4.2, adjust the number of channels of the feature map obtained after the convolution operation in the FasterNet_EMA module; if the number of input channels is inconsistent with the number of channels processed inside the module, the number of channels will be adjusted through 1x1 convolution to ensure that the number of channels of the input feature map is consistent with the expected number and matches the processing requirements of the module; Step 4.3, the input feature map first passes through the Partial_conv3 module for spatial mixing, and then the feature map enters the MLP module for nonlinear transformation to further enhance the feature representation; Step 4.4, the feature map after MLP processing is regularized by DropPath, and then the EMA attention mechanism is applied for weighted processing; Step 4.5, adding the feature map processed by step 4.4 to the original input feature to retain the input feature and output the final result; In step 4.6, based on the feature map output in step 4.5, the LADH lightweight detection head is used to replace the original detection head in YOLOv8 for the final segmentation task.

7. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 6 is characterized in that: The specific process of step 4.3 is as follows: Step 4.3.1, the input feature map is spatially mixed through the Partial_conv3 convolution operation. The convolution operation is performed in different regions according to the parameters n_div and pconv_fw_type to achieve spatial mixing of feature maps. Among them, pconv_fw_type is a parameter used to control the forward propagation behavior of partial convolution. In step 4.3.2, the spatially mixed feature map enters the MLP module for nonlinear transformation, and two 1x1 convolutions expand the feature dimension to mlp_hidden_dim and then reduce it back to dim.

8. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 7, characterized in that: The specific process of step 4.4 is as follows: Step 4.4.1, the feature map processed by MLP is discarded through DropPath, and then enters the EMA attention mechanism for further weighting; Step 4.4.2, define the input parameters and group them, accept the input channel number channels and grouping factor factor, which are used to determine the number of groups of the input data; verify whether the number of channels in each group is greater than 0 through the assert statement; Step 4.4.3, initialize the components used to process the feature map, including the Softmax layer, the adaptive average pooling layer, the group normalization layer, and the 1x1 and 3x3 convolution layers; Step 4.4.4, in the forward propagation, first obtain the shape of the input feature map, and then rearrange the feature map according to the number of groups; Step 4.4.5, perform adaptive average pooling in the horizontal and vertical directions on the reconstructed feature map to obtain feature map representations in two directions, and fuse them through a 1x1 convolutional layer; Step 4.4.6, multiply the feature map after pooling in step 4.4.5 and the feature map reconstructed in step 4.4.4 element by element, and process it through the Sigmoid activation function and the group normalization layer, and then use 3x3 convolution to further extract features. The generated feature map , the feature extraction process is expressed as: in, x h It is the feature map after the pooling operation; x w is the reconstructed feature map; Step 4.4.7, perform Softmax operation on the first set of feature maps processed in step 4.4.6 after using adaptive average pooling to obtain the attention weight matrix of the first set of feature maps x 11 ; Step 4.4.8, use adaptive average pooling and Softmax operation on the second set of feature maps processed in step 4.4.6 to obtain the attention weight matrix of the second set of feature maps x 21 , and the final attention weight is obtained by the attention weights of the two sets of feature maps. The calculation formula is as follows: Among them, A is the final attention weight; x 11 is the attention weight matrix of the first set of feature maps obtained in step 4.4.7; x 21 is the attention weight matrix of the second set of feature maps in step 4.4.8; x 12 is the feature matrix obtained by flattening the second set of feature maps; x 22 is the feature matrix obtained by flattening the first set of feature maps; Step 4.4.9, use the final attention weight obtained in step 4.4.8 to multiply the feature map obtained in step 4.4.1, and then resize the result to the original shape, return the feature map processed by the attention mechanism, and use it for subsequent network layers to enhance the feature extraction ability of the model. The formula is expressed as: in, x It is the feature map after adjusting the shape; A is the final attention weight obtained in step 4.4.8; DropPath(x) It is the feature map after the path is discarded; b is the batch size; groups is the number of groups; h and w The height and width of the feature map after adjusting the shape; Step 4.4.10, add the feature map processed by step 4.4.9 and the feature map after spatial mixing in step 4.3.1 to form the final output, which is expressed as: 。 9. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 8, characterized in that: The specific process of step 4.5 is as follows: Step 4.5.1, define a Segment_LADH module; Step 4.5.2, define the basic properties of the model in the module initialization function, including the number of categories nc, the number of masks nm, the number of prototypes npr, and the number of channels ch of the input feature map; Step 4.5.3, calculate the number of output channels c4 of each convolutional layer and depth convolutional layer. The specific formula is as follows: Step 4.5.4, construct a convolution module group cv4 for processing the input feature map, each module includes two depth-separable convolutions DSConv and one standard convolution Conv; Step 4.5.5, during the forward propagation process, use the Proto module to process the first layer of the input feature map to generate the mask prototype p; Step 4.5.6, use the corresponding cv4 module to process each input feature map, combine the depth separable convolution DSConv and the ordinary convolution Conv, perform step-by-step feature extraction on the feature map, and finally generate the mask coefficient. Use the torch.cat function to concatenate the mask coefficients of all layers in the third dimension to obtain the final mask coefficient mc; Step 4.5.7, call the LADH detection head to process the input feature map and generate the final prediction result x; Step 4.5.8, determine whether the model is in training mode according to the internal attribute self.training of nn.Module in PyTorch. If self.training is true, return the prediction result x, mask coefficient mc and mask prototype p; If self.training is false, different spliced ​​outputs are returned depending on whether self.export is in export mode. The spliced ​​output contains the prediction results and mask coefficients for the final image segmentation result generation.

10. The lightweight tunnel crack segmentation and recognition method based on improved YOLOv8 according to claim 9, characterized in that: The specific process of step 5 is as follows: Use the training set divided in step 3.2 to train the improved YOLOv8 model in step 4. During the training process, Adam is used as the optimizer and CIoU is used as the loss function to ensure that the model converges quickly.

Images