A fabric surface defect detection method fusing multi-scale feature maps

Abstract

A fabric surface defect detection method fusing multi-scale feature maps, comprising the following operation steps: improving a Yolov8 target detection algorithm, using an improved ECA-C2f module to replace the preset C2f module of the 2nd, 4th, 6th and 8th layers in the original backbone network; using an improved GA-SPPF module to replace the spatial pyramid pooling structure in the original backbone network; in the neck, a self-defined focus dispersion module (FDM) is used to aggregate feature maps of different scales; the improved target detection algorithm is defined as EGA-FDM-Yolov8. The model parameters are trained and determined using a public data set, and finally an improved Yolov8 algorithm fabric defect intelligent detection device is obtained. Without increasing the amount of parameters, the accuracy of the Yolov8 algorithm is improved while ensuring the detection rate; in the fabric defect detection, the average accuracy of defect recognition is improved by 15%, and the elongated defects with extreme width-height ratio and the stain defects with extremely similar background color can be detected more accurately.

Description

Technical Field

[0001] This invention relates to the field of visual target detection technology, and mainly to a method for detecting surface defects in fabrics by fusing multi-scale feature maps. Background Technology

[0002] With the rapid development of science and technology, and against the backdrop of the era of artificial intelligence and big data, deep learning has been further developed in the field of machine vision and has been widely applied in areas such as industrial defect detection, autonomous driving, and security technology. Nevertheless, some factories still rely on manual methods for detecting surface defects in fabrics. Manual inspection is susceptible to fatigue and distraction, especially after long working hours, leading to a high rate of missed detections. To address these challenges, this invention proposes an efficient method for detecting surface defects in fabrics by improving the Yolov8 algorithm, thus overcoming the shortcomings of existing technologies.

[0003] In the industrial production of fabrics, various fabric defects arise due to machine malfunctions, including numerous elongated defects with extreme aspect ratios and stains that closely resemble the background color. YOLOv8's performance deteriorates when handling these defects. Furthermore, YOLOv8 lacks robustness to variations in lighting and complex fabric textures, leading to false positives or false negatives. This makes fabric surface defect detection an extremely challenging task. Therefore, there is still significant room for improvement in fabric surface defect detection. Summary of the Invention

[0004] To address the aforementioned technical challenges in identifying fabric surface defects, this technical solution provides a fabric surface defect detection method that integrates multi-scale feature maps. Using a Yolov8 network as the baseline network, an improved version, EGA-FDM-Yolov8, is derived. This method proposes an effective target detection model for fabric surface defects, which can better distinguish between stains and fabric color, and more accurately detect slender defects with extreme aspect ratios. By improving detection accuracy without reducing detection speed, this method effectively solves the aforementioned problems.

[0005] This invention is achieved through the following technical solution:

[0006] A method for detecting surface defects in fabrics by fusing multi-scale feature maps includes the following steps:

[0007] Step 1: Optimize the Yolov8 object detection algorithm, build and improve the Yolov8 model to obtain a fabric surface defect detection model; specific steps include:

[0008] Step 1.1: Integrate the ECA attention mechanism into the C2f modules of layers 2, 4, 6, and 8 of the Yolov8 backbone network to generate the ECA-C2f module; the specific operation is as follows:

[0009] The ECA-C2f module first passes the input tensor x through the first convolutional layer Conv1 and splits it into two tensors along the channel dimension. It then performs n ECA loops on the split second tensor, using one module from the module list in each loop. Next, it concatenates all the tensors after the ECA loops along the channel dimension. Finally, it performs the final feature mapping through the second convolutional layer Conv2.

[0010] Step 1.2: The GA-SPPF module is used in the 9th layer of the Yolov8 backbone network to enhance the traditional spatial pyramid pooling structure SPPF; the specific operation is as follows:

[0011] The GA-SPPF module first passes the input tensor x through a GhostConv, then through two parallel MaxPools to obtain y1 and y2, and then performs AvgPool on the parallel results to obtain z; finally, x, y1, y2, and z are concatenated along the channel dimension, and the concatenated result is passed through a second convolutional layer GhostConv for feature mapping, and finally the output result is returned.

[0012] Step 1.3: Add a Focus on Dispersion Module (FDM) to the original neck network. FDM is used to acquire multi-scale feature maps. The specific content is as follows:

[0013] Using GhostConv convolution, the feature maps generated by the third, fourth, and fifth convolutions of the backbone network are aggregated and connected together, which allows each scale feature to have detailed contextual information, which is beneficial for defect detection. Then, parallel convolution is used to add the resulting features to capture rich cross-scale information.

[0014] Step 2: Obtain the publicly available sample dataset, convert the JSON file to YOLO training format, and divide the dataset into training, validation, and test sets according to the specified proportions;

[0015] Step 3: Obtain the optimal model parameters through debugging. The improved Yolov8 model using the ECA-C2f, GA-SPPF, and FDM modules is defined as EGA-FDM-Yolov8. The specific details are as follows:

[0016] The backbone network of the fabric surface defect detection model consists of the first convolutional module, the second convolutional module, the first ECA-C2f module, the third convolutional module, the second ECA-C2f module, the fourth convolutional module, the third ECA-C2f module, the fifth convolutional module, the fourth ECA-C2f module, and the GA-SPPF module.

[0017] The training set obtained in step two is input into the fabric surface defect detection model obtained by the improved EGA-FDM-Yolov8 device for training. On the validation set, the model with the highest average accuracy is saved and its weight file is named best.pt.

[0018] Step 4: Use the trained fabric surface defect detection model to detect the image to be detected, ultimately enhancing the model's detection accuracy, generalization ability, and robustness.

[0019] Furthermore, the training set, validation set, and test set mentioned in step two are divided in a ratio of 8:1:1.

[0020] Furthermore, the publicly available sample dataset uses the open-source Tianchi competition fabric defect detection dataset, totaling 7312 images; it includes 21 categories of fabric defect datasets, which include the following types: no defects, holes, water stains, oil stains, dirt, three threads, knots, pattern skips, 100 feet, fuzz, coarse warp, loose warp, broken warp, hanging warp, coarse fiber, weft shrinkage, sizing spots, warp knots, star skips, pattern skips, broken spandex, sparse and dense sections, wavy sections, color difference sections, abrasion marks, creases, dead wrinkles, and defective weft yarns.

[0021] Furthermore, in step three, the image size of the training set input is set to 640*640, the batch size is 8, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, the optimizer uses SGD, and after training is completed, the model will save the best weight file best.pt.

[0022] Furthermore, in step four, when detecting the image to be detected, the best.pt obtained in step three is used as the weight file for detecting defects on the fabric surface, and the fabric image is then detected.

[0023] Beneficial effects

[0024] The present invention proposes a method for detecting surface defects in fabrics by fusing multi-scale feature maps, which has the following advantages compared with existing technologies:

[0025] (1) This invention constructs a fabric defect detection device by improving the Yolov8 model, thereby increasing the accuracy of detecting defects with extreme aspect ratios and colors similar to the fabric. In improving the Yolov8 model, the C2f layers (layers 2, 4, 6, and 8) of the backbone network are integrated with an ECA attention mechanism to form a novel ECA-C2f module. By introducing the ECA attention mechanism into the backbone feature extraction network, the weights of the feature channels can be adaptively adjusted according to their importance, thus enhancing the network's ability to perceive defects.

[0026] (2) In this invention, the 9th layer of the improved Yolov8 model uses the GA-SPPF module, which enhances the traditional spatial pyramid pooling structure SPPF. Introducing average pooling into SPPF can effectively reduce the spatial size of the feature map, thereby improving computational efficiency. It also reduces the dependence on hardware devices and is better applied in fabric production environments.

[0027] (3) In improving the Yolov8 model, this invention uses a self-developed FDM module at the neck of the network to aggregate the feature maps generated by the backbone network. The self-developed FDM module, by using convolutional kernels of different sizes (5, 7, 9, 11), can capture features at different scales, helping the network to focus on details and contextual information while accepting three different scale input feature maps (x1, x2, x3). Size matching and channel number adjustment are performed through upsampling and convolution, allowing the module to be flexibly applied to different input features. By improving the Yolov8 structure, the model's performance is enhanced, improving the accuracy of the model in detecting surface defects in fabrics. Attached Figure Description

[0028] Figure 1 This is the overall flowchart of the present invention.

[0029] Figure 2 This is a diagram of the EGA-FDM-Yolov8 network structure in this invention.

[0030] Figure 3 This is a network structure diagram of the ECA-C2f module in this invention.

[0031] Figure 4 This is a network structure diagram of the GA-SPPF module in this invention.

[0032] Figure 5 This is a network structure diagram of the FDM module in this invention. Detailed Implementation

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. The described embodiments are merely some embodiments of the present invention, and not all embodiments. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the design concept of the present invention should fall within the protection scope of the present invention.

[0034] Example 1:

[0035] like Figure 1 As shown, a method for detecting surface defects in fabrics by fusing multi-scale feature maps includes the following steps:

[0036] Step 1: Optimize the Yolov8 object detection algorithm, build and improve the Yolov8 model to obtain a fabric surface defect detection model; the improved network structure is as follows. Figure 2 As shown. The specific steps for building and improving the Yolov8 model are as follows:

[0037] Step 1.1: Integrate the ECA attention mechanism into the C2f modules of layers 2, 4, 6, and 8 of the Yolov8 backbone network to generate the ECA-C2f module.

[0038] The network structure diagram of the ECA-C2f module is as follows: Figure 3 As shown, this module introduces the ECA attention mechanism, integrating ECA into Bottleneck to form ECA-Bottleneck. Based on this, the ECA-C2f module is built, which not only enhances the model's sensitivity to input features but also improves the detection accuracy of fabric surface defects.

[0039] ECA (Efficient Channel Attention) is a feature enhancement technique for convolutional neural networks. It improves the network's representational power by adaptively emphasizing important channel features and suppressing unimportant features. ECA is also a lightweight attention mechanism, making it particularly suitable for computationally limited environments while simultaneously improving model performance.

[0040] The workflow is as follows:

[0041] Global average pooling is applied to each channel of the input feature map to obtain a low-dimensional feature vector containing global statistical information. This feature vector is then passed through a 1D convolutional layer to generate an attention weight map. The attention weight map is non-linearly transformed and scaled using an activation function. The original feature map is multiplied by the attention weights processed by the ECA module to enhance important channel features. Finally, the attention-enhanced feature map is output for subsequent network layers or tasks. The module's formula is as follows:

[0042]

[0043] a = σ(conv(z));

[0044] F' c (i,j)=F c (i,j)×a c ;

[0045] Among them, Z c This is the global eigenvalue of the c-th channel after global average pooling; H and W are the height and width of the feature map, respectively; i is the row index of the feature map's spatial location; j is the column index of the feature map's spatial location; F c (i,j) is the pixel value of the c-th channel in feature map F at position (i,j); a is the attention weight generated by 1D convolution; σ represents the activation function, conv is a 1D convolution operation, and its kernel size is determined by the parameter k; z is the feature vector after convolution and activation function processing; F' c (i,j) is the enhanced feature map, a c It is the attention weight corresponding to channel c.

[0046] The constructed ECA-C2f module first passes the input tensor x through the first convolutional layer Conv1 and splits it into two tensors along the channel dimension. Then, it performs n ECA loops on the split second tensor, using one module from the module list in each loop. Next, it concatenates all the tensors after the ECA loop along the channel dimension. Finally, it performs the final feature mapping through the second convolutional layer Conv2.

[0047] Step 1.2: The GA-SPPF module is used in the 9th layer of the Yolov8 backbone network to enhance the traditional spatial pyramid pooling structure SPPF.

[0048] A hybrid pooling strategy is adopted, combining average pooling and max pooling techniques to enhance the SPPF module in the original network, forming a novel GA-SPPF module, such as... Figure 4 As shown, this enables the network to more accurately identify key information when dealing with diverse fabric defects, thus achieving a significant improvement in defect detection performance while maintaining high computational efficiency.

[0049] The GA-SPPF module introduces GhostConv to reduce computation and parameter count while maintaining model performance by generating "phantom features." Traditional convolution operations directly generate all feature maps, while GhostConv generates feature maps in two steps: first, it generates a small number of main feature maps using regular convolutions; then, it generates additional "phantom features" from these main feature maps using depthwise convolutions, thus approximating the complete feature map.

[0050] The GA-SPPF module first passes the input tensor x through a GhostConv (the first convolutional layer cv1), then performs max pooling on x using two parallel MaxPool layers to obtain y1 and y2. The parallel results are then processed by AvgPool to obtain z. Finally, x, y1, y2, and z are concatenated along the channel dimension, and the concatenated result is passed through a second convolutional layer GhostConv (the second convolutional layer cv2) for feature mapping, ultimately returning the output result. The mathematical expression is as follows:

[0051] y1 = Max(x);

[0052] y2 = Max(x);

[0053] Y=cat(GhostConv(x),Avg(y1+y2));

[0054] In the above formula, Y represents the final output result, GhostConv represents the second convolutional layer cv2, cat represents the concatenation operation in the channel dimension, Max and Avg represent the max pooling and average pooling operations respectively, y1 is the result of the first max pooling, and y2 is the result of the second max pooling.

[0055] The improved backbone network consists of the first convolutional module, the second convolutional module, the first ECA-C2f module, the third convolutional module, the second ECA-C2f module, the fourth convolutional module, the third ECA-C2f module, the fifth convolutional module, the fourth ECA-C2f module, and the GA-SPPF module.

[0056] Step 1.3: Add a Focus on the Dispersion Module (FDM) to the original neck network to obtain multi-scale feature maps; the network structure diagram of the FDM module is as follows. Figure 5 As shown.

[0057] The focusing and dispersing module first uses three sets of GhostConv convolutional and upsampling modules to receive three feature maps P3, P4, and P5 generated from layers 4, 6, and 9 of the backbone network. These three multi-scale feature maps are concatenated together, and a set of parallel convolutional modules is used to capture rich cross-scale information. Then, a dispersing mechanism distributes the obtained multi-scale information across various detection scales. This is beneficial for detecting small fabric defects.

[0058] The specific workflow is as follows:

[0059] First, define a module containing three sets of GhostConv and upsampling modules, setting the scale_factor of the upsampling layers to 2. Second, define a list of modules containing multiple depthwise separable convolutional layers, each with 3 input and output channels (hidc*3). Separate the input x into three feature maps of different scales (x1, x2, x3). Process these three feature maps using GhostConv and the upsampling modules, then merge them along the channel dimension (dim=1) into a new feature map x. Pass the merged feature map x through parallel convolutional layers sequentially, summing the results to obtain the feature. Finally, sum each feature to obtain the final output. The module formula is as follows:

[0060] X=cat([x1, x2, x3], dim=1);

[0061]

[0062] R = X + F;

[0063] Where x1, x2, and x3 are the results obtained by upsampling and convolution of feature maps of different dimensions, X is the result of concatenating the processed feature maps along the channel dimension; each convolutional layer is applied to x, and the sum of these results is calculated to obtain an aggregated feature map F; R is the result obtained by adding the feature map to the original concatenated feature map X.

[0064] Step 2: Obtain the publicly available sample dataset, convert the JSON file to YOLO training format, and divide it into training and validation sets; the specific steps are as follows:

[0065] The dataset required for the experiment was constructed using publicly available datasets. The 2019 Guangdong Industrial Intelligent Manufacturing Innovation Competition Tianchi Fabric Defect Detection Dataset was obtained, and 7312 images containing 21 categories were selected. The categories of fabric defects include the following types: no defects, holes, water stains, oil stains, dirt, three threads, knots, pattern skips, 100 feet, fuzz, coarse warp, loose warp, broken warp, hanging warp, coarse fiber, weft shrinkage, sizing spots, warp knots, star skips, pattern skips, broken spandex, sparse and dense sections, wavy sections, color difference sections, abrasion marks, creases, dead wrinkles, and poor weft yarn.

[0066] Convert the JSON file to a YOLO training TXT file, and divide the dataset into training, validation, and test sets in an 8:1:1 ratio.

[0067] Step 3: Input the training set obtained in Step 1 into the fabric surface defect detection model obtained by optimizing the Yolov8 model for training, and obtain the optimal fabric surface defect detection model.

[0068] The input image size is set to 640*640, the batch size is 8, the number of training iterations is 300, the initial learning rate is 0.01, the learning momentum is 0.937, the weight decay coefficient is 0.0005, the optimizer uses SGD, and after training is completed, the model will save the best weight file, best.pt.

[0069] To evaluate the performance of the improved model, the metrics for assessing its superiority include: mAP (average AP for each class), Precision, and Recall. A PR curve is plotted with recall on the x-axis and precision on the y-axis. The area under the PR curve is defined as AP. The formulas for calculating each metric are as follows:

[0070]

[0071]

[0072] Where TP represents the number of samples correctly predicted as positive, FN is the number of samples that the model failed to correctly identify in the actual positive samples, n represents the total number of classes, and P(R)D(R) represents the average precision of the R-th class.

[0073] Step 4: Utilize the trained fabric surface defect detection model to detect defects in the image to be tested, ultimately enhancing the model's detection accuracy, generalization ability, and robustness. When detecting defects in the image, the best.pt file obtained in Step 3 is used as the weight file for detecting fabric surface defects.

[0074] To verify the feasibility and advantages of this solution, the inventors conducted a comparative experiment on the improved Yolov8 model and the original Yolov8 model. The comparison of the detection results of this embodiment with Yolov8 is shown in Table 1:

[0075] Table 1 Comparison of experimental results

[0076] Yolov8 0.458 0.238 0.858 Our 0.608 0.349 0.866

[0077] The data in the table above shows that the results of the network optimization in this invention are significantly improved on Yolov8. At mAP@0.5%, the model of this invention is 15% higher than the original model. At mAP@0.5:0.95%, the model of this invention is 11.1% higher than the original model. In terms of precision, the model of this invention is 0.8% higher than the original model.

[0078] The comparison of the detection results (mAP@0.5%) of the Yolov8 network before and after the improvement for 20 defect categories is shown in Table 2:

[0079] Table 2 Comparison of Categories

[0080]

[0081]

[0082] The data in the table above shows that improvements were achieved in various ways across all twenty categories of fabric defects. Specifically, a 20.1% improvement was observed for stains similar in color to the fabric, and a 20.7% improvement was achieved for defects with extreme aspect ratios, such as weft yarn defects. This invention effectively improves the performance of the fabric surface defect detection model.

[0083] Table 3 shows the model parameters. The number of parameters in the improved model is lower than that in the original model.

[0084] Yolov8 3014943 Our 2950929

[0085] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered by the present invention.

Claims

1. A method for detecting surface defects in fabrics by fusing multi-scale feature maps, characterized in that: The following steps are included: Step 1: Optimize the Yolov8 object detection algorithm, build and improve the Yolov8 model to obtain a fabric surface defect detection model; specific steps include: Step 1.1: Integrate the ECA attention mechanism into the C2f modules of layers 2, 4, 6, and 8 of the Yolov8 backbone network to generate the ECA-C2f modules; the specific operation is as follows: The ECA-C2f module first passes the input tensor x through the first convolutional layer Conv1 and splits it into two tensors along the channel dimension. It then performs n ECA loops on the split second tensor, using one module from the module list in each loop. Next, it concatenates all the tensors after the ECA loops along the channel dimension. Finally, it performs the final feature mapping through the second convolutional layer Conv2. Step 1.2: The GA-SPPF module is used in the 9th layer of the Yolov8 backbone network to enhance the traditional spatial pyramid pooling structure SPPF; the specific operation is as follows: The GA-SPPF module first passes the input tensor x through a GhostConv, then through two parallel MaxPools to obtain y1 and y2, and then performs AvgPool on the parallel results to obtain z; finally, x, y1, y2, and z are concatenated along the channel dimension, and the concatenated result is passed through a second convolutional layer GhostConv for feature mapping, and finally the output result is returned. Step 1.3: Add a Focus on Dispersion (FDM) module to the original neck network to obtain multi-scale feature maps; the specific content is as follows: Using GhostConv convolution, the feature maps generated by the third, fourth, and fifth convolutions of the backbone network are aggregated and connected together, which allows each scale feature to have detailed contextual information, which is beneficial for defect detection. Then, parallel convolution is used to add the resulting features to capture rich cross-scale information. Step 2: Obtain the publicly available sample dataset, convert the JSON file to YOLO training format, and divide the dataset into training, validation, and test sets according to the specified proportions; Step 3: Obtain the optimal model parameters through debugging. The improved Yolov8 model using the ECA-C2f, GA-SPPF, and FDM modules is defined as EGA-FDM-Yolov8. The specific details are as follows: The backbone network of the fabric surface defect detection model consists of the first convolutional module, the second convolutional module, the first ECA-C2f module, the third convolutional module, the second ECA-C2f module, the fourth convolutional module, the third ECA-C2f module, the fifth convolutional module, the fourth ECA-C2f module, and the GA-SPPF module. The training set obtained in step two is input into the fabric surface defect detection model obtained by the improved EGA-FDM-Yolov8 device for training. On the validation set, the model with the highest average accuracy is saved and its weight file is named best.pt. Step 4: Use the trained fabric surface defect detection model to detect the image to be detected, ultimately enhancing the model's detection accuracy, generalization ability, and robustness.

2. The fabric surface defect detection method based on multi-scale feature maps according to claim 1, characterized in that: The training set, validation set, and test set mentioned in step two are divided in a ratio of 8:1:

1.

3. The fabric surface defect detection method based on multi-scale feature maps according to claim 1 or 2, characterized in that: The publicly available sample dataset uses the open-source Tianchi competition fabric defect detection dataset, totaling 7312 images. It includes 21 categories of fabric defect datasets, including the following types: no defects, holes, water stains, oil stains, dirt, three threads, knots, pattern skips, freckles, lint, thick warp, loose warp, broken warp, hanging warp, coarse fiber, weft shrinkage, sizing spots, warp knots, star skips, pattern skips, broken spandex, sparse and dense sections, wavy sections, color difference sections, abrasion marks, creases, dead wrinkles, and defective weft yarns.

4. The fabric surface defect detection method based on multi-scale feature maps according to claim 1, characterized in that: In step three, the input image size for the training set is set to 640*640, the batch size is 8, the number of training iterations is 300, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, the optimizer uses SGD, and after training is completed, the model will save the best weight file, best.pt.

5. The fabric surface defect detection method based on multi-scale feature maps according to claim 1, characterized in that: In step four, when detecting the image to be detected, the best.pt obtained in step three is used as the weight file for detecting defects on the fabric surface, and the fabric image is then detected.

Images