Lightweight tunnel crack segmentation and recognition method based on improved YOLOv8

Abstract

The application discloses a light-weight tunnel crack segmentation and identification method based on improved YOLOv8, and specifically comprises the following steps: step 1, acquiring image data set of a tunnel crack, and pre-processing the data set; step 2, expanding and enhancing the pre-processed data set; step 3, labeling the data set processed in step 2; step 4, improving a YOLOv8 model; step 5, training the model improved in step 4; and step 6, training, testing and verifying the model trained in step 5. The application solves the problems of low segmentation efficiency and low crack detection precision of the existing crack segmentation method.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision image segmentation and relates to a lightweight tunnel fissure segmentation and recognition method based on improved YOLOv8. Background Art

[0002] The segmentation and identification of tunnel cracks is a critical task in tunnel engineering, directly impacting the safety and long-term performance of tunnel structures. Currently, tunnel crack segmentation typically relies on manual inspection or automated methods based on traditional image processing. However, these methods exhibit significant limitations in practical applications, particularly in complex crack backgrounds and uneven illumination, where crack edges can become blurred, leading to inaccurate segmentation results. Traditional methods based on edge detection and threshold segmentation techniques struggle to cope with the diversity of cracks in complex environments, often resulting in missed detections and false detections.

[0003] With the development of deep learning technology, image segmentation methods based on convolutional neural networks (CNNs) have gradually been applied to crack segmentation. In particular, instance segmentation models based on the YOLO family can accurately locate cracks in complex scenarios, eliminating the need for manual intervention in the crack segmentation process. However, existing methods have also exposed some problems in practical applications. First, they are computationally intensive and slow. Many deep learning models (such as YOLOv8) have high computational complexity, especially when processing high-resolution tunnel images, which is not conducive to real-time application. Second, the segmentation accuracy of existing methods is still not ideal and cannot meet the requirements of efficient and accurate crack detection. Summary of the Invention

[0004] The purpose of the present invention is to provide a lightweight tunnel crack segmentation and recognition method based on improved YOLOv8, which solves the problems of low segmentation efficiency and low crack detection accuracy in existing crack segmentation methods.

[0005] The technical solution adopted by the present invention is a lightweight tunnel crack segmentation and recognition method based on an improved YOLOv8, which specifically includes the following steps:

[0006] Step 1: Obtain an image dataset of tunnel cracks and preprocess the dataset;

[0007] Step 2: Expand and enhance the preprocessed dataset;

[0008] Step 3: Label the data set processed in step 2;

[0009] Step 4: Improve the YOLOv8 model;

[0010] Step 5: train the improved model in step 4;

[0011] Step 6, training test verification is performed on the model trained in step 5.

[0012] The application is also characterized in that:

[0013] The specific process of step 1 is:

[0014] Step 1.1, download the crack segmentation dataset from the Roboflow platform;

[0015] Step 1.2, use the Gaussian filter denoising algorithm to process the images in the dataset of step 1.1, remove random noise in the images, and at the same time keep the clarity of the crack edges;

[0016] Step 1.3, uniformly adjust the image size processed in step 1.2 to the same size.

[0017] The specific process of step 2 is:

[0018] Step 2.1, input the preprocessed dataset into the WGAN-RE model for iterative training, and judge whether the model converges in the training process through the Frechet Inception Distance and Inception Score performance indicators;

[0019] Step 2.2, based on the judgment result of step 2.1, when the generator and discriminator of the WGAN-RE model converge, expand the dataset with a 1:2 expansion ratio to form an enhanced dataset.

[0020] In step 2.1, FID is expressed as:

[0021]

[0022] Wherein, and respectively represent the mean vector of the real image and the generated image in the feature space; and respectively represent the covariance matrix of the real image and the generated image in the feature space; x represents the distribution of the real image; g represents the distribution of the real image;

[0023] Wherein, IS is expressed as:

[0024]

[0025] Wherein, X~Pg represents a sample drawn from the distribution of the generated image x . represents the class distribution predicted by the Inception network for the image x . is the average class distribution of the generated image as a whole; is the class distribution KL divergence between the overall class distribution represents the expectation of the generated image sample.

[0026] The specific process of step 3 is as follows:

[0027] Step 3.1, use the "Polygon" tool in the image annotation software Labelme to label the cracks, save it as JSON format, and then convert the labeled data to YOLO format TXT file through Python script;

[0028] Step 3.2, divide the data set labeled in step 3.1 into training set and validation set in the ratio of 8:2, and store the divided data set in different folders according to the category, each folder contains the corresponding image and label file.

[0029] The specific process of step 4 is as follows:

[0030] Step 4.1, initialize the module, when the Faster_Block_EMA class is initialized, set the input channel number, output channel number, MLP ratio, DropPath probability, and layer scaling value of the module;

[0031] Step 4.2, adjust the channel number of the feature map obtained after the convolution operation in the FasterNet_EMA module; if the input channel number is inconsistent with the channel number processed inside the module, adjust the channel number through 1x1 convolution to ensure that the channel number of the input feature map is consistent with the expectation and matches the processing requirements of the module;

[0032] Step 4.3, the input feature map first passes through the Partial_conv3 module for spatial mixing, and then the feature map enters the MLP module for nonlinear transformation to further enhance the feature representation;

[0033] Step 4.4, after the feature map processed by MLP is regularized by DropPath, EMA attention mechanism is applied for weighted processing;

[0034] Step 4.5, add the feature map processed in step 4.4 to the original input feature to retain the input feature and output the final result;

[0035] Step 4.6, based on the feature map output in step 4.5, use LADH detection head (Lightweight Asymmetric Detection Head) to replace the original detection head in YOLOv8 for the final segmentation task.​

[0036] The specific process of step 4.1 is as follows:

[0037] Step 4.1.1, set the core properties of the module, including the output feature dimension dim, the MLP ratio mlp_ratio, and the number of divisions of the partial convolution n_div;

[0038] In step 4.1.2, we initialize the multi-layer perceptron (MLP), partial convolution (Partial_conv3), and EMA attention mechanism, which constitute the main computational components of the module.

[0039] The specific process of step 4.3 is:

[0040] In step 4.3.1, the input feature map is spatially mixed by the partial_conv3 convolution operation. The convolution operation is performed in different regions according to the parameters n_div and pconv_fw_type to achieve spatial mixing of feature maps. Among them, pconv_fw_type is a parameter used to control the forward propagation behavior of the partial convolution.

[0041] In step 4.3.2, the spatially mixed feature map enters the MLP module for nonlinear transformation, and two 1x1 convolutions expand the feature dimension to mlp_hidden_dim and then reduce it back to dim.

[0042] The specific process of step 4.4 is as follows:

[0043] In step 4.4.1, the feature map processed by MLP is discarded through DropPath and then enters the EMA attention mechanism for further weighting;

[0044] Step 4.4.2 defines the input parameters and groups them. Accepts the input channel number channels and the grouping factor factor to determine the number of groups of input data. Use the assert statement to verify that the number of channels in each group is greater than 0.

[0045] In step 4.4.3, initialize the components used to process the feature map, including the Softmax layer, the adaptive average pooling layer, the group normalization layer, and the 1x1 and 3x3 convolution layers;

[0046] In step 4.4.4, in the forward propagation, first obtain the shape of the input feature map, and then rearrange the feature map according to the number of groups;

[0047] In step 4.4.5, adaptive average pooling is performed on the reconstructed feature map in the horizontal and vertical directions respectively to obtain feature map representations in two directions, and then fused through a 1x1 convolutional layer;

[0048] Step 4.4.6, element-wise multiplication is performed between the pooled feature map in step 4.4.5 and the reconstructed feature map in step 4.4.4, and then the result is processed by a Sigmoid activation function and a group normalization layer, followed by a 3x3 convolution to further extract features, generating a feature map The feature extraction process is represented as:

[0049]

[0050] wherein, x h is the feature map after pooling operation; x w is the reconstructed feature map;

[0051] Step 4.4.7, after using adaptive average pooling on the first group of feature maps processed in step 4.4.6, Softmax operation is performed to obtain the attention weight matrix of the first group of feature maps x 11；

[0052] Step 4.4.8, after using adaptive average pooling on the second group of feature maps processed in step 4.4.6, Softmax operation is performed to obtain the attention weight matrix of the second group of feature maps x 21 The final attention weight is obtained through the attention weights of the two groups of feature maps, and the calculation formula is as follows:

[0053]

[0054] wherein, A is the final attention weight; x 11 is the attention weight matrix of the first group of feature maps obtained in step 4.4.7; x 21 is the attention weight matrix of the second group of feature maps in step 4.4.8; x 12 is the feature matrix obtained by flattening the second group of feature maps; x 22 is the feature matrix obtained by flattening the first group of feature maps.

[0055] Step 4.4.9, the final attention weight obtained in step 4.4.8 is multiplied with the feature map obtained in step 4.4.1, and then the result is adjusted back to the original shape, returning the feature map processed by the attention mechanism for subsequent network layer use, to enhance the feature extraction capability of the model, and the formula is represented as:

[0056]

[0057] wherein, x is the reshaped feature map; A is the final attention weight obtained in step 4.4.8; DropPath (x) is the feature map after path dropping; b is the batch size; groups is the number of groups; h and w is the height and width of the reshaped feature map;

[0058] Step 4.4.10, add the feature map processed in step 4.4.9 to the feature map spatially mixed in step 4.3.1 to form the final output, which is represented by the formula:

[0059]

[0060] The specific process of step 4.5 is as follows:

[0061] Step 4.5.1, define a Segment_LADH module;

[0062] Step 4.5.2, define the basic properties of the model in the initialization function of the module, including the number of classes nc, the number of masks nm, the number of prototypes npr, and the number of channels ch of the input feature map;

[0063] Step 4.5.3, calculate the output channel number c4 of each convolutional layer and depth convolutional layer, and the specific formula is as follows:

[0064]

[0065] Step 4.5.4, construct a convolution module group cv4 for processing the input feature map, each module including two depth separable convolutions DSConv and one standard convolution Conv;

[0066] Step 4.5.5, in the forward propagation process, use the Proto module to process the first layer of the input feature map to generate the mask prototype p;

[0067] Step 4.5.6, process each input feature map using the corresponding cv4 module, combining depth separable convolution DSConv and ordinary convolution Conv, to gradually extract features from the feature map, and finally generate the mask coefficient. The mask coefficients of all layers are concatenated in the third dimension by the torch.cat function to obtain the final mask coefficient mc;

[0068] Step 4.5.7, call the LADH detection head to process the input feature map to generate the final prediction result x;

[0069] Step 4.5.8, according to the internal attribute self.training of nn.Module in PyTorch, whether the model is in training mode, if self.training is true, return the prediction result x, the mask coefficient mc and the mask prototype p;

[0070] If self.training is false, according to whether it is in the export mode self.export, return different splicing outputs, which contain the prediction result and the mask coefficient, for the final image segmentation result generation.

[0071] The specific process of step 5 is as follows:

[0072] Using the training set divided in step 3.2, the improved YOLOv8 model in step 4 is trained, and in the training process, Adam is used as the optimizer and CIoU is used as the loss function to ensure that the model converges quickly.

[0073] The beneficial effects of the present application are that, compared with the classical tunnel fracture segmentation method, the present application expands the data set by using the residual block enhanced Wasserstein generative adversarial network (WGAN-RE), enriches the data set, and enhances the generalization ability of the model; the light weight module of FasterNet is used to replace the standard convolution in the original C2f Bottleneck; the EMA attention mechanism is added in the improved C2f_Faster module; finally, the LADH light weight detection head is used to replace the original detection head in YOLOv8; thereby improving the detection accuracy of the model on the fracture area, reducing the calculation complexity while maintaining the detection accuracy, and having strong robustness. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1 is the model overall structure diagram of the improved YOLOv8 light weight tunnel fracture segmentation and identification method of the present application;

[0075] Figure 2 is the structure diagram of the C2f_Faster_EMA module in the improved YOLOv8 light weight tunnel fracture segmentation and identification method of the present application;

[0076] Figure 3 is the structure diagram of the PConv module in the improved YOLOv8 light weight tunnel fracture segmentation and identification method of the present application;

[0077] Figure 4 is the structure diagram of the EMA module in the improved YOLOv8 light weight tunnel fracture segmentation and identification method of the present application;

[0078] Figure 5is the LADH module structure diagram in the light-weight tunnel crack segmentation and identification method based on improved YOLOv8 of the present application;

[0079] Figure 6 is the result photo of the YOLOv8n model in the tunnel crack segmentation task;

[0080] Figure 7 is the result photo of the YOLOv8s model in the tunnel crack segmentation task;

[0081] Figure 8 is the result photo of the YOLOv9c model in the tunnel crack segmentation task;

[0082] Figure 9 is the result photo of the light-weight tunnel crack segmentation and identification method based on improved YOLOv8 of the present application in the tunnel crack segmentation task. DETAILED DESCRIPTION

[0083] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0084] Embodiment 1

[0085] The light-weight tunnel crack segmentation and identification method based on improved YOLOv8 of the present application comprises the following steps:

[0086] Step 1, obtain the image dataset of the tunnel crack, and pre-process the dataset, the pre-processing including denoising of the image and size standardization;

[0087] Step 1.1, download the crack segmentation dataset from the Roboflow platform, these images show different crack types, including longitudinal cracks, transverse cracks, and cracks in complex backgrounds;

[0088] Step 1.2, use a Gaussian filter denoising algorithm to process the image, remove random noise in the image, while preserving the clarity of the crack edges;

[0089] Step 1.3, adjust the size of all images to 640x640, ensure data consistency to meet the model input requirements.

[0090] Step 2, for the pictures in the pre-processed dataset, use the residual block enhanced Wasserstein generative adversarial network (WGAN-RE) to expand the dataset, enrich the dataset, and enhance the generalization ability of the model;

[0091] The WGAN mainly consists of a generator and a discriminator. In the generator, 100-dimensional random noise is first generated. Then, the noise is shaped into data with a size of 4 x 4 x 512 using a deconvolution operation with a kernel size of 4 x 4 and a step size of 1. The data is sequentially input into four deconvolution layers with a kernel size of 4 and a step size of 2, and a new image sample with an output size of 64 x 64 x 3 is obtained. In the convolution layer, the image with a size of 64 x 64 x 3 is sequentially input into four convolution layers with a kernel size of 4 and a step size of 2, and data with a size of 4 x 4 x 512 is obtained. Finally, the data is converted into a scalar for judging the authenticity of the input image. The generator and the discriminator continue to optimize themselves during the training process until the discriminator cannot judge whether the input is from a real or generated image.

[0092] The WGAN is suitable for generating crack images with a size of 640 x 640. However, when the size of the generated image increases, there will be a problem of reduced convergence speed of the network and the generated crack image is not realistic enough. In order to solve the above problems, the WGAN-RE is formed by introducing a residual block and an efficient channel attention module (ECA) into the WGAN.

[0093] The core part of the generator and the discriminator of the WGAN-RE is the DRE (Deconvolution enhanced by Residual Block and ECA) module and the CRE (Convolution enhanced by Residual Block and ECA) module, respectively. In the CRE module, the feature map F with a size of H x W x C is first converted into F1 with a size of H / 2 x W / 2 x 2C by a convolution layer with a kernel size of 4, and then F1 is input into the ECA module. In the ECA, F1 is sequentially passed through a global average pooling layer, an adaptive one-dimensional convolution layer, and a Sigmoid activation function to obtain an attention vector. The modified feature map F2 is obtained by multiplying the attention vector with F1. Finally, F2 is input into a convolution layer with a kernel size of 3, and the output of the residual edge is added to obtain the output feature map F C The structure of DRE and CRE is basically the same, and the only difference is that the first convolution layer with a kernel size of 4 is replaced by the deconvolution layer in DRE.

[0094] Step 2.1, input the preprocessed data set into the generator of the WGAN-RE model for iterative training, and judge whether the generator converges during the training process through the Frechet Inception Distance (FID) and Inception Score (IS) performance indicators. Wherein the FID can be expressed as:

[0095]

[0096] where and denote the mean vector of real and generated images in the feature space, respectively; and denote the covariance matrix of real and generated images in the feature space, respectively; x denotes the distribution of real images; g denotes the distribution of real images.

[0097] where IS can be represented as:

[0098]

[0099] where X~Pg denotes drawing a sample from the distribution of generated images x ; denotes the class distribution predicted by the Inception network for the image x ; is the average class distribution of the generated image as a whole; is the Kullback-Leibler divergence (KL divergence) between the class distribution and the overall class distribution ; denotes the expectation over the generated image samples.

[0100] By observing the logs after training, if the FID tends to decrease and stabilize, and the IS tends to increase and stabilize, it can be judged that the generator has basically converged; if the discriminator loss tends to 0.5, it means that the discriminator has maintained balance in the classification of real images and generated images, that is, it cannot completely distinguish between generated images and real images, and it can be judged that the discriminator has basically converged. If the generator and the discriminator have reached convergence, the trained WGAN-RE network can be used to expand the fissure data set; if the generator or the discriminator has not reached the state of convergence, the number of iterations of training is increased, and the training is restarted.

[0101] Step 2.2, according to the judgment standard of step 2.1, when the generator and the discriminator of the WGAN-RE model converge, expand the data set with an expansion ratio of 1:2 (i.e. the ratio of the number of images after preprocessing in step 1.3 to the number of generated images in step 2.1 is 1:2) to form an enhanced data set.

[0102] Step 3, label the tunnel fissure data set enhanced in step 2.2 and divide the data set;

[0103] Step 3.1, crack labeling is performed using the "Polygon" tool in the image labeling software Labelme, saved as JSON format, and then the labeling data is converted into TXT file in YOLO format through Python script, so as to be used for model training.

[0104] Step 3.2, the data set labeled in step 3.1 is divided into training set and validation set in the ratio of 8:2. The divided data set is stored in different folders according to the category, and each folder contains corresponding image and labeling file, so as to be used for subsequent model training and evaluation.

[0105] Step 4, YOLOv8 is improved by using FasterNet. The standard convolution in the original C2f Bottleneck is replaced by the lightweight module partial convolution of FasterNet; in the improved C2f_Faster module, the EMA (Exponential Moving Average) attention mechanism is added.

[0106] The application introduces a lightweight Faster_EMA network structure in YOLOv8, which has the advantages of faster detection speed and higher detection precision.

[0107] As shown in Figure 1 The overall network structure can be divided into three parts: backbone network, neck and head. The backbone network accepts an input image of 640x640 size. The input image is gradually extracted by the multi-layer convolution operation of the backbone network. The backbone network uses multiple C2f_Faster_EMA modules, which combine the lightweight design of FasterNet and the EMA attention mechanism, and can efficiently extract spatial and channel features of the image. After the backbone network, the feature maps of different layers are fused by "splicing" and "up-sampling" operations. Through the fusion of features of different scales, the model can capture more comprehensive context information, thereby improving the final prediction performance. The C2f_Faster_EMA module is used in the neck structure, which not only reduces the calculation amount through the lightweight structure, but also improves the global context awareness ability of the model through the EMA mechanism, helping the model better understand the image content. The last part of the network is the LADH detection head, which is used to process the feature map and generate the final detection or segmentation output. In the segmentation task, the LADH detection head will generate mask coefficients and segmentation results according to the feature map from the neck part. The design of the LADH module reduces the computational complexity through the asymmetric lightweight structure, making it suitable for real-time applications and resource-constrained devices.

[0108] The improved C2f module of the YOLOv8 model backbone part is fused with the Faster_EMA module to form a C2f_Faster_EMA module. In addition, the FasterNet_EMA module is a main feature extraction module, and each FasterNet_EMA block is composed of a partial convolution (PConv) and two subsequent point-wise convolutions (PWConv), which are combined into an inverted residual block, and an EMA attention mechanism module is added at the end to enhance the feature extraction capability of the network. The structure of the C2f_Faster_EMA module is shown in Figure 2 .

[0109] As shown in Figure 3 , PConv is a new convolution operation proposed by FasterNet, which is different from the previous various convolution methods that perform convolution on all channels of the input features. PConv only performs regular convolution on a part of the channels of the input features to extract features, and does not process the other channels. In order to ensure the continuity of memory access, the first or last c p channels are usually selected to represent the entire feature map. Therefore, PConv greatly reduces the parameter quantity and computational complexity while still maintaining good feature extraction capability. As shown in equation 1, when c p is 1 / 4 of c, the FLOPs (Floating Point Operations) of PConv is only 1 / 16 of that of regular convolution.

[0110]

[0111] wherein, is the computational complexity of the traditional convolution operation; is the computational complexity of the partial convolution; is the height of the input feature map; is the width of the input feature map; is the size of the convolution kernel; is the number of channels of the input feature map; c p is the number of channels of the input feature map in the partial convolution.

[0112] The EMA attention mechanism is introduced in the FasterNet Block module, which improves the pixel-level attention of the high-level feature map without reducing the channel dimension. For any given input feature map X, the EMA channel dimension is divided into G sub-feature , taking G << C, the learned attention weights reinforce the representation of sub-features in all attention mechanisms. EMA adopts three parallel paths to extract grouped feature maps, two parallel paths are 1x1 convolution branches, and the third parallel path is a 3x3 convolution branch. During the cross-space learning stage, the two-dimensional global average pooling is used to encode the global spatial information, and the pooling operation formula is:

[0113]

[0114] wherein, represents the global average value of the cth channel of the feature map, and is the output value of the channel after the pooling operation; is the height of the input feature map; is the width of the input feature map; is the pixel value of the cth channel of the feature map at position

[0115] The Softmax function is used at the output of the average pooling to fit, and the outputs of each branch are multiplied by matrix dot product operation to obtain the respective spatial attention maps on the two branches. Finally, the output feature information of the two branches is mapped into two sets of spatial attention weight values, and the Sigmoid function sends them to the subsequent modules.

[0116] The three branches of the EMA attention mechanism integrate the advantages of channel attention and spatial attention, while capturing global channel dependency and local spatial features, obtaining more comprehensive feature information between channel and spatial dimensions, as shown in Figure 4 The introduction of the attention mechanism EMA into the FasterNet module weakens the weight of background information to improve the focusing ability of the model on multi-scale target features, and more accurately obtains the feature information of the target to be detected in the image.

[0117] Step 4.1, initialization module, when the Faster_Block_EMA class is initialized, set the key parameters of the module such as the number of input channels, the number of output channels, the MLP ratio, the DropPath probability, and the hierarchical scaling value. The module includes partial convolution, MLP and EMA attention mechanism, and these components are combined to form the core architecture of the module;

[0118] The specific steps of step 4.1 are as follows:

[0119] Step 4.1.1, set the core attributes of the module, including the output feature dimension dim, the MLP ratio mlp_ratio and the segmentation number n_div of partial convolution;

[0120] ​Step 4.1.2, initialize the Multi-Layer Perceptron (MLP), Partial_conv3, and EMA attention mechanism, which constitute the main computational components of the module.

[0121] Step 4.2, adjust the channel number of the feature map obtained after the convolution operation in the FasterNet_EMA module. If the input channel number is inconsistent with the channel number processed inside the module, the channel number will be adjusted through 1x1 convolution to ensure that the channel number of the input feature map is consistent with the expected one, matching the processing requirements of the module;

[0122] Step 4.3, spatial mixing and feature processing. The input feature map first goes through the Partial_conv3 module for spatial mixing, which enhances the interaction of spatial information through partial convolution of the feature map, and then the feature map enters the MLP module for nonlinear transformation, further enhancing the feature representation;

[0123] The specific steps of step 4.3 are as follows:

[0124] Step 4.3.1, the input feature map is spatially mixed through Partial_conv3 convolution operation. The partial convolution in this step performs convolution operation in different regions according to the parameters n_div and pconv_fw_type, realizing the spatial mixing of the feature map;

[0125] where pconv_fw_type is a parameter used to control the forward propagation behavior of Partial Convolution. In Partial_conv3, it determines the specific way of partial convolution operation. According to the set pconv_fw_type value, partial convolution may adopt different strategies when processing input feature maps, such as how to divide input feature maps, how to perform convolution operation on the divided parts, etc.

[0126] Step 4.3.2, the spatially mixed feature map enters the MLP module for nonlinear transformation. The two 1x1 convolutions respectively expand the feature dimension to mlp_hidden_dim and then reduce it back to dim.

[0127] Step 4.4, path dropout and attention mechanism. After the feature map processed by MLP is regularized by DropPath, it is weighted by EMA attention mechanism. These steps aim to improve the generalization ability of the model through path dropout, and focus on important features through attention mechanism.

[0128] The specific steps of step 4.4 are as follows:

[0129] Step 4.4.1, the feature map processed by MLP is discarded by DropPath, and then enters the EMA attention mechanism for further weighting;

[0130] Step 4.4.2, define input parameters and group, accept input channel number channels and grouping factor factor, to determine the number of input data groups. Verify that the number of channels in each group is greater than 0 through the assert statement;

[0131] Step 4.4.3, initialize various components for processing feature maps, including Softmax layer, adaptive average pooling layer, group normalization layer, and 1x1 and 3x3 convolution layer.

[0132] Step 4.4.4, in the forward propagation, first get the shape of the input feature map, then rearrange the feature map according to the number of groups, so that each group of feature maps can be processed independently.

[0133] Step 4.4.5, perform adaptive average pooling in horizontal and vertical directions on the reconstructed feature map, respectively, to obtain feature map representations in both directions, and fuse them through a 1x1 convolution layer.

[0134] Step 4.4.6, multiply the feature map after pooling in step 4.4.5 and the reconstructed feature map in step 4.4.4 element by element, and then pass through the Sigmoid activation function, group normalization layer processing, and then use 3x3 convolution to further extract features. The feature extraction process can be represented as:

[0135]

[0136] Where x h is the feature map after pooling operation; x w is the reconstructed feature map; x conv is the feature map generated after step 4.4.6 processing.

[0137] Step 4.4.7, the first group of feature maps processed in step 4.4.6 are subjected to Softmax operation after adaptive average pooling to obtain the attention weight matrix of the first group of feature maps x 11 .

[0138] Step 4.4.8, the second group of feature maps processed in step 4.4.6 are subjected to Softmax operation after adaptive average pooling to obtain the attention weight matrix of the second group of feature maps x 21and the final attention weight is obtained through the attention weight of the two groups of feature maps. The calculation formula is as follows:

[0139]

[0140] wherein A is the final attention weight; x 11 is the attention weight matrix of the first group of feature maps obtained in step 4.4.7; x 21 is the attention weight matrix of the second group of feature maps obtained in step 4.4.8; x 12 is the feature matrix obtained after flattening the second group of feature maps; x 22 is the feature matrix obtained after flattening the first group of feature maps.

[0141] Step 4.4.9, multiply the final attention weight obtained in step 4.4.8 with the feature map obtained in step 4.4.1, and then adjust the result to the original shape. Return the feature map processed by the attention mechanism for subsequent network layers to enhance the feature extraction capability of the model. The formula is expressed as:

[0142]

[0143] wherein, x is the feature map after shape adjustment; A is the final attention weight obtained in step 4.4.8; DropPath (x) is the feature map after path dropping; b is the batch size; groups is the number of groups; h and w is the height and width of the feature map after shape adjustment.

[0144] Step 4.4.10, add the feature map processed in step 4.4.9 to the feature map mixed in space in step 4.3.1 to form the final output. The formula can be expressed as:

[0145]

[0146] Step 4.5, add the feature map processed by the above processing to the original input to retain the input features and output the final result. If hierarchical scaling is enabled, the channels of the feature map will also be scaled, resulting in more detailed feature control. The final result is passed to the next layer of the network for further processing.

[0147] Step 5, based on the feature map output in step 4.5, replace the original detection head in YOLOv8 with LADH lightweight detection head for the final segmentation task. Through the LADH module, the feature map is compressed and enhanced, and asymmetric compression is used to maintain key features while reducing computational complexity;

[0148] Figure 5 The structure of LADH detection head is summarized. By separating the network based on task type, three different channels are used to perform related tasks. In order to expand the receptive field and increase the task parameters of the IoU branch, three kinds of convolution are adopted to reduce the features along the channel dimension. In the LADH network, 3x3 deep convolution (DWConv) is used instead of traditional 3x3 convolution. The advantage of DWConv over standard convolution is that it significantly reduces the number of parameters by decomposing the convolution operation into depth separable convolution and pointwise convolution. This method further reduces the model parameters and improves the model inference speed. Using 3x3 depth separable convolution layer to separate classification and bounding box tasks can prevent the two tasks from over-expanding, as the positive samples matched with the two tasks have relatively small correlation loss. By replacing the decoupled head in YOLOv8 network with LADH-Head, the model parameters are significantly reduced while improving the detection accuracy.

[0149] Step 5.1, define a Segment_LADH module, which inherits from Detect_LADH. Based on LADH design. Create a new segmentation detection head based on the existing detection head to improve the segmentation task of the crack image; Step 5.2, define the basic properties of the model in the initialization function of the module, including the number of classes (nc), the number of masks (nm), the number of prototypes (npr), and the number of channels of the input feature map (ch). These parameters determine the overall architecture and function of the model; Step 5.4, calculate the output channel number c4 of each convolution layer and depth convolution layer. This value depends on the channel number ch[0] of the input feature map and the number of masks nm to ensure that the generated feature map is suitable for subsequent processing and generating masks; where the output channel number c4 can be represented as:

[0150]

[0151] Step 5.5, construct a convolution module group cv4 for processing the input feature map, each module contains multiple convolution operations, including two depth separable convolutions (DSConv) and one standard convolution (Conv). The design goal of these modules is to reduce the computational complexity while maintaining the model performance; Step 5.6, during the forward propagation process, the first layer of the input feature map is processed using the Proto module to generate mask prototypes p. These prototypes are used for subsequent mask generation to ensure that the model can learn more diverse features. Step 5.7, for each input feature map, use the corresponding cv4 module for processing, combining depth separable convolution (DSConv) and ordinary convolution to perform step-by-step feature extraction on the feature map, and finally generate mask coefficients. All layer mask coefficients are concatenated in the third dimension by the torch.cat function to obtain the final mask coefficients mc. Step 5.8, call the LADH detection head to process the input feature map to generate the final prediction result x. Step 5.9, according to the internal attribute self.training of nn.Module in PyTorch, judge whether the model is in training mode, if self.training is true, return the prediction result x, mask coefficient mc and mask prototype p. These outputs will be used for loss calculation in training; if self.training is false, according to whether it is in export mode (self.export), return different concatenation outputs. The concatenation output contains the prediction result and the mask coefficient, which is used for the final image segmentation result generation.

[0152] Step 6, use the training set divided in step 3.2 to train the improved YOLOv8 model in step 5. During training, Adam is used as the optimizer and CIoU is used as the loss function to ensure fast convergence of the model; Step 6.1, load the configuration file of the improved YOLOv8 model and initialize the model parameters. Load the pre-trained weight YOLOv8n.pt to improve the training efficiency of the model. Then, specify the dataset path for crack image segmentation, run the training script, start the training of the model, and save the results in the specified project path for subsequent model evaluation and use; Step 7, test the model on the validation set divided in step 3.2, and compare the crack segmentation performance of different models under the same configuration conditions.

[0153] Example 2

[0154] The results of the model comparison experiment are shown in Table 1; according to the results shown in Table 1, the improved model proposed in the present application is significantly better than other algorithm models, and has the highest detection accuracy among all models, with a boundary box mAP and mask mAP of 82.1% and 67.6%, respectively. While ensuring accuracy, the model significantly reduces the parameter quantity and computational quantity.

[0155] Table 1 Model comparison experimental results

[0156]

[0157] Figure 6 This is a photo of the results of the YOLOv8n model in the tunnel crack segmentation task; Figure 7 This is a photo of the results of the YOLOv8s model in the tunnel crack segmentation task; Figure 8 This is a photo of the results of the YOLOv9c model in the tunnel crack segmentation task; Figure 9 This is the result photo of the tunnel crack segmentation task based on the improved YOLOv8 lightweight tunnel crack segmentation and recognition method of the present invention. Figures 6 to 9 It can be seen that the method of the present invention performs well in processing crack morphology and background interference. The outline of the crack is finer, especially the curved part of the crack, which is closer to the real crack morphology.

[0158] Example 3

[0159] To verify the optimization effect of the improved content on the model, the present invention conducted four groups of ablation experiments on the validation set divided in step 3.2 without changing the experimental environment. The ablation test results are shown in Table 2, where A represents the use of FasterNet Block to replace the Bottleneck module of C2f in the original YOLOv8 network; B represents the addition of the EMA attention mechanism to the FasterNet Block based on A; C represents the use of the LADH lightweight detection head to replace the original detection head based on B.

[0160] Table 2 shows the performance comparison of the improved YOLOv8 crack segmentation network on the same dataset after performance enhancement. It is not difficult to see that after replacing the C2f Bottleneck module in the original YOLOv8 network with FasterNet Block, although the bounding box mAP and mask mAP decreased by 1.2% and 1.3% respectively, the computational load decreased by 2.9 GFLOPs. On this basis, adding the EMA attention mechanism to the FasterNet Block increased the bounding box mAP and mask mAP by 2.4% and 2.5% respectively, while the model computational load only increased by 0.2 GFLOPs. On this basis, after replacing the original detection head with the LADH lightweight detection head, the bounding box mAP and mask mAP increased by 1.2% and 0.8% respectively, and the model computational load was further reduced by 2.9 FLOPs. Through this performance enhancement, the bounding box mAP and mask mAP ultimately increased by 2.4% and 2%, respectively, and the model computational load was reduced by 4.5 GFLOPs.

[0161] The experimental environment uses a Linux Ubuntu 22.04 operating system, the graphics card model is NVIDIA GeForce RTX4090, 24GB video memory, the processor is 12 vCPU Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz, the deep learning framework is PyTorch 2.4.0, the GPU acceleration library is CUDA 12.1, the programming language is Python, the interpreter is Python 3.9, and during training, the input image size is 640 x 640, each model training period is 200 rounds, the Batch-Size is 64, the SGD optimizer is used, the initial learning rate is 0.01, and the final learning rate is 0.0001.

[0162] Table 2 ablation experiment results

[0163]

Claims

1. A lightweight tunnel crack segmentation and identification method based on improved YOLOv8, characterized in that, Specifically comprising the following steps: Step 1, obtaining an image data set of a tunnel fracture, preprocessing the data set; Step 2, extending and enhancing the preprocessed data set; the specific process of step 2 is as follows: Step 2.1, input the preprocessed data set into the WGAN-RE model enhanced by residual blocks for iterative training, and judge whether the model converges in the training process through the Frechet Inception Distance and Inception Score performance indicators; in step 2.1, FID is expressed as: （1） wherein, and and denote the mean vector of real and generated images in the feature space, respectively; and and denote the covariance matrix of real and generated images in the feature space, respectively; x denotes the distribution of real images; g denotes the distribution of real images; wherein, IS is denoted as: （2） Among them, X~Pg represents the samples drawn from the distribution of generated images x ; Indicates the Inception network for the image x Predicted class distribution; is the average category distribution of the generated images as a whole; is the category distribution and overall category distribution The KL divergence between Represents the expectation of generating image samples; Step 2.2, based on the judgment result of step 2.1, when the generator and discriminator of the WGAN-RE model converge, the data set is expanded with an expansion ratio of 1:2 to form an enhanced data set; Step 3, labeling the data set processed in step 2; Step 4, improving the YOLOv8 model; the specific process of step 4 is as follows: Step 4.1, initializing the module, when the Faster_Block_EMA class is initialized, setting the input channel number, output channel number, MLP ratio, DropPath probability, and layer scaling value of the module; Step 4.2, adjusting the channel number of the feature map obtained after adjusting the convolution operation in the FasterNet_EMA module; if the input channel number is inconsistent with the channel number processed inside the module, the channel number will be adjusted through 1x1 convolution; Step 4.3, the input feature map first passes through the Partial_conv3 module for spatial mixing, and then the feature map enters the MLP module for nonlinear transformation to further enhance the feature representation; Step 4.4, after the feature map processed by the MLP is regularized by DropPath, the EMA attention mechanism is applied for weighted processing; Step 4.5, adding the feature map processed in step 4.4 to the original input feature to retain the input feature and output the final result; the specific process of step 4.5 is as follows: Step 4.5.1, defining a Segment_LADH module; Step 4.5.2, defining the basic properties of the model in the initialization function of the module, including the number of classes nc, the number of masks nm, the number of prototypes npr, and the channel number ch of the input feature map; Step 4.5.3, calculating the output channel number c4 of each convolutional layer and depth convolutional layer, the specific formula is as follows: （3） Step 4.5.4, constructing a convolution module group cv4 for processing the input feature map, each module includes two depth separable convolutions DSConv and one standard convolution Conv; Step 4.5.5, in the forward propagation process, the Proto module is used to process the first layer of the input feature map to generate a mask prototype p; Step 4.5.6, each input feature map is processed using the corresponding cv4 module, combining the depth separable convolution DSConv and the ordinary convolution Conv, to gradually extract features from the feature map, and finally generate a mask coefficient, all layer mask coefficients are spliced in the third dimension through the torch.cat function to obtain the final mask coefficient mc; Step 4.5.7, the LADH detection head is called to process the input feature map, generating the final prediction result x; Step 4.5.8, according to the internal attribute self.training of nn.Module in PyTorch, it is judged whether the model is in training mode, if self.training is true, the prediction result x, mask coefficient mc and mask prototype p are returned; if self.training is false, according to whether it is in export mode self.export, different splicing outputs are returned, which contain the prediction result and the mask coefficient, which are used for the final image segmentation result generation; Step 4.6, based on the feature map output by step 4.5, the LADH lightweight detection head is used to replace the original detection head in YOLOv8 for the final segmentation task; Step 5, train the model improved in step 4; Step 6, train and test the model trained in step 5.

2. The improved YOLOv8-based lightweight tunnel crack segmentation and identification method according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1, download the crack segmentation dataset from the Roboflow platform; Step 1.2, use the Gaussian filter denoising algorithm to process the images in the dataset of step 1.1, remove random noise in the images, and at the same time preserve the clarity of the crack edges; Step 1.3, uniformly adjust the size of the images processed in step 1.2 to the same size; In step 2.1, FID is represented as: （4） wherein, and and denote the mean vector of real and generated images in the feature space, respectively; and and denote the covariance matrix of real and generated images in the feature space, respectively; x denotes the distribution of real images; g denotes the distribution of real images; wherein, IS is denoted as: （5） Among them, X~Pg represents the samples drawn from the distribution of generated images x ; Indicates the Inception network for the image x Predicted class distribution; is the average category distribution of the generated images as a whole; is the category distribution and overall category distribution The KL divergence between Represents the expectation of generating image samples.

3. The improved YOLOv8-based lightweight tunnel crack segmentation and identification method according to claim 2, characterized in that, The specific process of step 3 is as follows: Step 3.1, use the "Polygon" tool in the image annotation software Labelme to label the cracks, save it as JSON format, and then convert the labeled data to TXT file in YOLO format through Python script; Step 3.2, divide the dataset labeled in step 3.1 into training set and validation set in the ratio of 8:2, and store the divided dataset in different folders according to the category, each folder contains the corresponding image and annotation file.

4. The improved YOLOv8-based lightweight tunnel crack segmentation and identification method according to claim 3, characterized in that, The specific process of step 4.3 is as follows: Step 4.3.1, the input feature map is spatially mixed by Partial_conv3 convolution operation, and the convolution is performed in different regions according to the parameters n_div and pconv_fw_type, realizing the spatial mixing of feature map; Where pconv_fw_type is a parameter used to control the forward propagation behavior of partial convolution; Step 4.3.2, the spatially mixed feature map enters the MLP module for nonlinear transformation, and two 1x1 convolutions respectively expand the feature dimension to mlp_hidden_dim and then reduce it back to dim.

5. The improved YOLOv8-based lightweight tunnel crack segmentation and identification method according to claim 4, characterized in that, The specific process of step 4.4 is as follows: Step 4.4.1, the feature map processed by MLP passes through DropPath for path dropping, and then enters the EMA attention mechanism for further weighting; Step 4.4.2, define input parameters and grouping, accept input channel number channels and grouping factor factor to determine the number of input data groups; verify that the number of channels in each group is greater than 0 through the assert statement; Step 4.4.3, initialize components for processing feature maps, including Softmax layer, adaptive average pooling layer, group normalization layer, and 1x1 and 3x3 convolution layers; Step 4.4.4, in forward propagation, first get the shape of the input feature map, then rearrange the feature map according to the number of groups; Step 4.4.5, perform adaptive average pooling in horizontal and vertical directions on the reconstructed feature map, respectively, to obtain feature map representations in both directions, and fuse them through a 1x1 convolution layer; Step 4.4.6, element-wise multiply the pooled feature map in step 4.4.5 and the reconstructed feature map in step 4.4.4, and then pass through a Sigmoid activation function, a group normalization layer, and a 3x3 convolution to further extract features, and generate a feature map The feature extraction process is represented as: （6） wherein, x h is the feature map after the pooling operation; x w is the reconstructed feature map; Step 4.4.7, after the first group of feature maps processed in step 4.4.6 is subjected to the Softmax operation after using the adaptive average pooling, the attention weight matrix of the first group of feature maps is obtained x 11 ; Step 4.4.8, after adaptive average pooling is used on the second group of feature maps processed in step 4.4.6, a Softmax operation is performed to obtain the attention weight matrix of the second group of feature maps x 21 and the final attention weight is obtained through the attention weights of the two groups of feature maps, and the calculation formula is as follows: （7） wherein A is the final attention weight; x 11 the attention weight matrix for the first group of feature maps obtained in step 4.4.7; x 21 the attention weight matrix for the second group of feature maps obtained in step 4.4.8; x 12 the feature matrix obtained after flattening the second group of feature maps; x 22 the feature matrix obtained after flattening the first group of feature maps; Step 4.4.9, multiply the final attention weight obtained in step 4.4.8 with the feature map obtained in step 4.4.1, then adjust the result to the original shape, and return the feature map processed by the attention mechanism, the formula is: （8） wherein, x is the reshaped feature map; A is the final attention weight obtained from step 4.4.8; DropPath(x) is the feature map after path dropping; b is the batch size; groups is the number of groups; h and w is the height and width of the reshaped feature map; Step 4.4.10, add the feature map processed in step 4.4.9 to the feature map mixed in space in step 4.3.1 to form the final output, the formula is: （9） wherein, DropPath(x) is the feature map after path dropout.

6. The improved YOLOv8-based lightweight tunnel crack segmentation and identification method according to claim 5, characterized in that, The specific process of step 5 is: Use the training set divided in step 3.2 to train the improved YOLOv8 model in step 4, use Adam as the optimizer and CIoU as the loss function in the training process to ensure the fast convergence of the model.

Images