Fabric Defect Detection Method Based on EFC Feature Extraction and Hybrid Loss Function
By improving the backbone and neck networks of the YOLOv8 model and combining EFC feature extraction and hybrid loss functions, the problems of low accuracy and missed detection of small-scale defects in fabric defect detection are solved, and efficient fabric defect detection is achieved.
Patent Information
- Application Number
- CN202410653179.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-24
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-05-24
AI Technical Summary
Existing methods for detecting fabric defects are not very accurate in detecting small-scale defects and have the problem of missing detections.
A fabric defect detection method based on EFC feature extraction and hybrid loss function is adopted. The backbone network and neck network of the YOLOv8 model are improved. The EMA-FasterNet Block module and the multi-scale feature fusion network CMFPN with cross-layer connection are used, combined with the hybrid loss function HCN Loss based on similarity metric to improve the detection capability of small-scale defects.
It improved the average accuracy of fabric defect detection to 88.9%, significantly improved the detection effect of small-scale defects, and reduced the number of missed detections.
Smart Images

Figure CN118570155B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of fabric defect detection technology, specifically relating to a fabric defect detection method based on EFC feature extraction and hybrid loss function. Background Technology
[0002] The textile industry is a large-scale traditional industry in my country. With economic development, people have increasingly higher requirements for fabric quality. During the weaving process, textiles are prone to numerous tiny defects due to various reasons. These tiny defects are extremely small in scale compared to the overall fabric image, and some are even invisible to the naked eye. This poses a challenge to fabric defect detection methods. Existing detection methods still need to improve their accuracy in detecting small-scale defects, and the lack of targeted focus on small-scale defects leads to missed detections. Summary of the Invention
[0003] The purpose of this invention is to provide a fabric defect detection method based on EFC feature extraction and mixed loss function, which solves the problems of low accuracy and missed detection of small-scale defects in fabrics in the prior art.
[0004] The technical solution adopted in this invention is a fabric defect detection method based on EFC feature extraction and hybrid loss function, comprising the following steps:
[0005] Step 1: Collect images of fabrics with defects and preprocess them. Randomly divide the dataset into training and evaluation sets according to the proportion. Use LabelImg to annotate the defective areas on the fabric images with defects in the training and evaluation sets, generate annotation files, and save them into the corresponding training and evaluation sets.
[0006] Step 2: Improve the YOLOv8 model and construct a fabric defect detection network;
[0007] Step 3: Train the constructed fabric defect detection network using the training set, and evaluate the performance of the trained fabric defect detection network using the evaluation set to obtain the final fabric defect detection model; input the image to be tested into the fabric defect detection model for detection, and obtain the detection results, which include the defect type and confidence level.
[0008] The invention is further characterized by:
[0009] In step 1, the defective fabric images are preprocessed, including scaling, rotating, stitching, and random color transformation to expand the dataset. In step 1, LabelImg is used to annotate the defective areas on the defective fabric images. The annotation includes the location information of the fabric defect, the image path, and the defect category. The defect categories are divided into five types: stains, holes, broken yarns, knots, and stray threads.
[0010] Step 2 involves improving the YOLOv8 model, which includes a backbone network, a neck network, and a head network, to construct a fabric defect detection network. This process includes the following steps:
[0011] Step 21: In the C2f feature extraction module of the YOLOv8 model backbone network, modify the Bottleneck module to the EMA-FasterNet Block module to construct a backbone network based on the EFC feature extraction module; the EMA-FasterNetBlock module includes convolutional layers, multi-scale attention EMA modules, normalization layers and ReLU activation function;
[0012] Step 22: In the neck network of the YOLOv8 model, modify the Feature Pyramid Fusion Network (PAN) to a multi-scale feature fusion network (CMFPN) with cross-layer connectivity. The multi-scale feature fusion network (CMFPN) with cross-layer connectivity includes a downsampling layer, a fusion layer, and a C2f feature fusion module.
[0013] Step 23: In the head network of the YOLOv8 model, modify the CIoU loss function of the bounding box loss to the HCN Loss, a hybrid loss function based on similarity metrics. The HCN Loss includes normalized Gaussian Wasserstein distance and CIoU loss function.
[0014] In step 21, the EMA-FasterNet Block module includes a partially convolutional PConv layer, connected to an EMA module, followed by two PWConv layers, with residual connections to reuse the initial features of the input, and a BN layer and a ReLU activation layer connected between the two PWConv layers.
[0015] The EFC feature extraction module consists of two convolutional Convs and several EMA-FasterNet Blocks, connected by Split and Concat.
[0016] In step 23, the multi-scale feature fusion network CMPPN, which connects across layers, is obtained by removing nodes that do not have feature fusion based on the combination of FPN and PAN.
[0017] The construction process of the HCN Loss based on similarity metric in step 23 is as follows:
[0018] Step 231: Model the bounding box as a two-dimensional Gaussian distribution;
[0019] Step 232: Calculate the distribution distance using Wasserstein distance for the predicted bounding box P = (cxp ,c yp ,w p ,h p ) and the true bounding box G = (c xg ,c yg ,w g ,h g Gaussian distribution model N P and N G The normalized Wasserstein distance between the two is calculated using the following formula:
[0020]
[0021] In the formula, It is the Wasserstein distance between the ground truth bounding box and the predicted bounding box, c xp ,c yp ,w p ,h p These represent the center coordinates, width, and height of the predicted bounding box, respectively; c xg ,c yg ,w g ,h g These represent the center coordinates, width, and height of the actual bounding box, respectively;
[0022] Step 233: Normalize formula (4) to the form of IoU ratio, and... Converting the value to a range of 0 to 1, we obtain the normalized Wasserstein distance (NWD), as shown in the following formula:
[0023]
[0024] In the formula, C0 is a constant;
[0025] Loss function based on NWD as a similarity measure NWD The calculation formula is as follows:
[0026]
[0027] Step 234: Construct a hybrid loss function HCN Loss based on similarity metrics, as shown in the following formula:
[0028] Loss HCN = (1-K)Loss CIoU +K(1-NWD)=(1-K)Loss CIoU +KLoss NWD (7) Where, Loss CIoU Let CIoU represent the loss function, and K represent the weight adjustment coefficients for the CIoU loss function and the NWD loss function.
[0029] Step 2: The fabric defect detection network includes an input end, a backbone network, a neck network, and a head network. The backbone network contains a convolutional Conv, several EFC feature extraction modules, and an SPPF module connected in sequence. Each pair of EFC feature extraction modules is connected to a Merging module, which consists of a convolutional Conv, a BN layer, and a SiLU activation function. The neck network consists of two upsampling modules and several C2f modules. It uses a multi-scale feature fusion network (CMFPN) that connects across layers to fuse 16x downsampling into 40×40 features with the network layers.
[0030] The total loss function used in training the fabric defect detection network is:
[0031] Loss = w cls L cls +wb ox L box +w obj L obj (1)
[0032] w cls +w box +w obj =1 (2) In the formula, L cls For the classification loss function, L box Let L be the bounding box loss function. obj Let w be the confidence loss function. cls ,w box ,w obj These are the weight coefficients corresponding to the classification loss function, bounding box loss function, and confidence loss function, respectively. The classification loss function adopts the binary cross-entropy loss function (BCE), the bounding box loss function adopts the hybrid loss function HCN Loss based on similarity measurement, and the confidence loss function adopts the distributed focus loss function (DFL).
[0033] Step 3 specifically includes the following steps:
[0034] Step 31: Use the training set as the training path to train the constructed fabric defect detection network; the training input is a fabric image with defective pixels of 640x640, and the number of fabric images with defective defects is 16 each time. Iterate for 200 rounds until the training ends, and output the model file, which contains the network structure and parameters of the fabric defect detection network.
[0035] Step 32: Use the model file obtained in Step 31 and the evaluation set obtained in Step 1 to evaluate the performance of the trained fabric defect detection network and obtain the final fabric defect detection model.
[0036] Step 33: Input the image to be tested into the fabric defect detection model and output the detection results with defect type and confidence level.
[0037] Step 32 uses the model file and evaluation set to evaluate the performance of the trained fabric defect detection network. The evaluation metrics include: model detection accuracy P, model detection performance R under positive sample conditions, detection accuracy AP for each type of defect, and overall detection accuracy mAP for each type of defect. The formulas for calculating the above metrics are as follows:
[0038] P = TP / (TP + FP) (10)
[0039] R = TP / (TP + FN) (11)
[0040] In the formula, TP represents the number of samples correctly predicted as positive, FP represents the number of negative samples incorrectly predicted as positive, and FN represents the number of positive samples incorrectly predicted as negative.
[0041]
[0042] In the formula, the P(R) function is a curve enclosed by the detection accuracy P of the index model as the vertical axis and the detection effect R of the index model under the positive sample condition as the horizontal axis. The AP value is obtained by summing the curves using an integral function.
[0043]
[0044] In the formula, K is a constant related to the dataset, i.e., the number of defect types, and mAP value is the average AP value of each defect obtained by summing and averaging through an integral function.
[0045] The beneficial effects of this invention are:
[0046] 1. This invention is a fabric defect detection method based on EFC feature extraction and mixed loss function. The C2f module of the backbone network is improved by using the EMA-FasterRCNN Block module, which further improves the computational efficiency of the network and its ability to extract small-scale defect features.
[0047] 2. This invention presents a fabric defect detection method based on EFC feature extraction and a hybrid loss function. It constructs a multi-scale feature fusion network across links, alleviating the problem of deep networks easily losing small-scale defect features. Furthermore, it replaces the original CIoU loss function with a hybrid loss function based on similarity metrics, solving the problem of sensitivity to the positional offset of small-scale defects in label assignment. Ultimately, this results in an average detection accuracy of 88.9%. Attached Figure Description
[0048] Figure 1This is a flowchart of the fabric defect detection method based on EFC feature extraction and hybrid loss function of the present invention;
[0049] Figure 2 This is a schematic diagram of the fabric defect detection model in the fabric defect detection method based on EFC feature extraction and hybrid loss function in this invention;
[0050] Figure 3 This is a schematic diagram of the backbone network structure of the EFC module in the fabric defect detection method based on EFC feature extraction and hybrid loss function in this invention;
[0051] Figure 4 This is a schematic diagram of the EFC feature extraction module and the EMA-FasterNet Block module in the fabric defect detection method based on EFC feature extraction and hybrid loss function in this invention;
[0052] Figure 5 This is a schematic diagram of the cross-link multi-scale feature fusion network (CMFPN) structure in the fabric defect detection method based on EFC feature extraction and hybrid loss function in this invention.
[0053] Figure 6 This is a comparison of the fabric defect detection method based on EFC feature extraction and hybrid loss function in this invention with the detection results of the YOLOv8 model before the improvement; Detailed Implementation
[0054] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0055] Fabric defect detection method based on EFC feature extraction and hybrid loss function, such as Figure 1 As shown, it includes the following steps:
[0056] Step 1: Collect images of fabrics with defects and preprocess them. Randomly divide the dataset into training and evaluation sets according to the proportion. Use LabelImg to annotate the defective areas on the fabric images with defects in the training and evaluation sets, generate annotation files, and store them in the corresponding training and evaluation sets.
[0057] Images of defective fabrics were collected, and preprocessed using methods such as scaling, rotation, stitching, and random coloring to expand the number of defective fabric images from 800 to 1123, all uniformly named in the format ####.png. The 1123 preprocessed images were randomly divided into training and evaluation sets at a 9:1 ratio. Defective regions were then labeled using LabelImg, automatically generating labeled files which were stored in the corresponding training and evaluation sets. The labels included the location information of the fabric defects, image paths, and defect categories. Defect categories were categorized into five types based on their characteristics: stains, holes, broken yarns, knots, and loose threads.
[0058] Step 2: Improve the YOLOv8 model and construct a fabric defect detection network;
[0059] The YOLOv8 model consists of a backbone network, a neck network, and a head network. An improved YOLOv8 model is constructed to perform fabric defect detection. The overall network is as follows: Figure 2 As shown, the network includes an input, a backbone, a neck, and a head. The backbone is based on the EFC feature extraction module. The neck consists of two upsampling modules and several C2f modules. The multi-scale feature fusion network CMPPN, which connects across layers, performs feature fusion by downsampling by 16 times to form 40×40 features and then connects them across layers. The head consists of a Merging module and convolutional layers. The connection loss function predicts the location and category of defects in the image to be detected.
[0060] Specifically, the following steps are included:
[0061] Step 21: In the C2f feature extraction module of the YOLOv8 model backbone network, modify the Bottleneck module to the EMA-FasterNet Block module to construct a backbone network based on the EFC feature extraction module; the EMA-FasterNetBlock module includes convolutional layers, multi-scale attention EMA modules, normalization layers and ReLU activation functions.
[0062] The backbone network based on the EFC feature extraction module is as follows: Figure 3 As shown, it includes a convolutional Conv, several EFC feature extraction modules and an SPPF module connected in sequence. A Merging module is connected between every two EFC feature extraction modules. The Merging module consists of a convolutional Conv, a BN layer and a SiLU activation function.
[0063] The EFC feature extraction module is built upon the original CSPDarknet53. The construction of the EFC feature extraction module involves two steps: first, combining the FasterNet Block module and the EMA module in the FasterNet network to design an attention-based EMA-FasterNet Block inverse residual module; second, replacing the Bottleneck module in the original C2f feature extraction module with the EMA-FasterNet Block module to construct the EFC feature extraction module. For example... Figure 4 As shown, the EFC feature extraction module consists of two convolutional Conv layers and several EMA-FasterNet Block modules connected by Split and Concat. This allows the network to be lightweight while obtaining richer gradient flow information, thereby improving the backbone network's comprehensive and accurate extraction of defect features. The EMA-FasterNet Block module is a novel inverse residual structure constructed by embedding an efficient multi-scale attention module EMA that learns across spatial spaces into the original FasterNet Block module. It includes a partial convolution (PConv) layer, followed by an EMA attention module, then two PWConv (or Conv1×1) layers, and then residual connections are placed to reuse the initial input features, together forming the inverse residual block. This effectively avoids the loss of information features and enhances the extraction and discrimination of defects. At the same time, after each intermediate convolution, a BN layer is used for batch normalization and a ReLU activation layer to increase the nonlinearity of the network and avoid limiting the diversity of features.
[0064] Step 22: In the neck network of the YOLOv8 model, the Feature Pyramid Fusion Network (PAN) is modified into a cross-layer connected Multi-Scale Feature Fusion Network (CMFPN). The cross-layer connected Multi-Scale Feature Fusion Network (CMFPN) includes a downsampling layer, a fusion layer, and a C2f feature fusion module.
[0065] Feature fusion networks fuse image features at different scales extracted through the backbone network to obtain high-level semantic features for defect classification. The cross-layer connected multi-scale feature fusion network CMFPN combines FPN (Feature Pyramid Networks) and PAN (Path Aggregation Network), removing nodes without feature fusion, simplifying the network structure, and using cross-layer connections to achieve multi-scale feature fusion. Features are concatenated in a concat form, preserving more layers of feature information. Figure 5As shown, specifically, a 40×40 feature map formed by 16x downsampling is connected across layers with subsequent feature maps. This allows feature maps to interact nonlocally across space and scale while maintaining parameter costs, fusing nonlocal features with more defect information, resulting in richer location information and more accurate defect localization. Shallow networks in the feature extraction network extract feature maps with larger scales and more detail, while deep networks extract feature maps with smaller scales and more abstract information. Therefore, CMFPN combines the two to obtain feature maps that are more effective at detecting smaller fabric defects.
[0066] Step 23: In the head network of the YOLOv8 model, modify the CIoU loss function of the bounding box loss to a hybrid loss function HCN Loss based on similarity metrics. The hybrid loss function HCN Loss based on similarity metrics includes normalized Gaussian Wasserstein distance and CIoU loss function.
[0067] The main part of the YOLOv8 model's head network consists of three detectors. This involves using grid-based anchors to detect objects on feature maps of three scales: 80x80, 40x40, and 20x20. The feature maps are divided into grids according to their size, and three anchors with different aspect ratios are pre-set for each grid on each feature map to predict and regress the target. The prediction is performed on each grid on the feature map, and the predicted information is compared with the real information. The closer the predicted information is to the real information, the smaller the loss function value.
[0068] The total loss function used in training the fabric defect detection network is:
[0069] Loss = w cls L cls +w box L box +w obj L obj (1)
[0070] w cls +w box +w obj =1 (2)
[0071] In the formula, L cls For the classification loss function, L box Let L be the bounding box loss function. obj Let w be the confidence loss function. cls ,w box ,w objThese are the weight coefficients corresponding to the classification loss function, bounding box loss function, and confidence loss function, respectively. The classification loss function adopts the binary cross-entropy loss function (BCE), the bounding box loss function adopts the hybrid loss function HCN Loss based on similarity measurement, and the confidence loss function adopts the distributed focus loss function (DFL).
[0072] The classification loss function helps the model correctly classify samples. We use the binary cross-entropy loss function (BCE), as shown in the following expression:
[0073] L cls = -ylogp-(1-y)log(1-p) (3)
[0074] Where y is the true label and p is the foreground probability predicted by the model.
[0075] The bounding box loss function is used to evaluate the model's localization and detection performance. A hybrid loss function based on similarity metrics, HCN Loss, is employed. The construction process of HCN Loss is as follows:
[0076] Step 231: Model the bounding box as a two-dimensional Gaussian distribution;
[0077] Step 232: Calculate the distribution distance using Wasserstein distance for the predicted bounding box P = (c xp ,c yp ,w p ,h p ) and the true bounding box G = (c xg ,c yg ,w g ,h g Gaussian distribution model N P and N G The normalized Wasserstein distance between the two is calculated using the following formula:
[0078]
[0079] In the formula, It is the Wasserstein distance between the ground truth bounding box and the predicted bounding box, c xp ,c yp ,w p ,h p These represent the center coordinates, width, and height of the predicted bounding box, respectively; c xg ,c yg ,w g ,h g These represent the center coordinates, width, and height of the actual bounding box, respectively;
[0080] Step 233: Normalize formula (4) to the form of IoU ratio, and... Converting the value to a range of 0 to 1, we obtain the normalized Wasserstein distance (NWD), as shown in the following formula:
[0081]
[0082] In the formula, C0 is a constant;
[0083] Loss function based on NWD as a similarity measure NWD The calculation formula is as follows:
[0084]
[0085] Step 234: Construct a hybrid loss function HCN Loss based on similarity metrics, as shown in the following formula:
[0086] Loss HCN = (1-K)Loss CIoU +K(1-NWD)=(1-K)Loss CIoU +KLoss NWD (7)
[0087] Among them, Loss CIoU Let CIoU represent the loss function, and K represent the weight adjustment coefficients for the CIoU loss function and the NWD loss function.
[0088] The CIoU loss function expression is as follows:
[0089]
[0090] In the formula, ρ 2 (b,b gt ) represents the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, C1 represents the diagonal distance of the smallest closure region that can simultaneously contain the predicted box and the ground truth box, v measures the aspect ratio distance between the two boxes, and α is the weight coefficient.
[0091] The confidence loss function measures the difference between the model's prediction of the confidence level of the detected object and the actual situation. The Distributed Focus Loss (DFL) function is used, and its expression is as follows:
[0092] L obj =DFL(S) i ,S i+1 )=-x((x i+1 -x)log(S i )+(xx i )log(S i+1 (9)
[0093] In the formula, x represents the predicted value, x i x represents the rounded predicted value. i+1 S represents the nearest predicted value. i This indicates that the Softmax function is used to expand the probability around the predicted value x.
[0094] Step 3: Train the constructed fabric defect detection network using the training set, and evaluate the performance of the trained fabric defect detection network using the evaluation set to obtain the final fabric defect detection model; input the image to be tested into the fabric defect detection model for detection, and obtain the detection results, which include defect type and confidence level.
[0095] Step 31: Use the training set as the training path to train the constructed fabric defect detection network; the training input is a fabric image with defective pixels of 640x640, and the number of fabric images with defective defects is 16 each time. Iterate for 200 rounds until the training ends, and output the model file, which contains the network structure and parameters of the fabric defect detection network.
[0096] Step 32: Use the model file obtained in Step 31 and the evaluation set obtained in Step 1 to evaluate the performance of the trained fabric defect detection network and obtain the final fabric defect detection model.
[0097] Evaluation metrics include: model detection accuracy P, model detection performance R under positive sample conditions, detection accuracy AP for various defects, and overall detection accuracy mAP for various defects; the calculation formulas for the above metrics are as follows:
[0098] TP / (TP+FP)(10)
[0099] R = TP / (TP + FN) (11)
[0100] In the formula, TP represents the number of samples correctly predicted as positive, FP represents the number of negative samples incorrectly predicted as positive, and FN represents the number of positive samples incorrectly predicted as negative.
[0101]
[0102] In the formula, the P(R) function is a curve enclosed by the detection accuracy P of the index model as the vertical axis and the detection effect R of the index model under the positive sample condition as the horizontal axis. The AP value is obtained by summing the curves using an integral function.
[0103]
[0104] In the formula, K is a constant related to the dataset, i.e., the number of defect types, and mAP value is the average AP value of each defect obtained by summing and averaging through an integral function.
[0105] Step 33: Input the image to be tested into the fabric defect detection model and output the detection results with defect type and confidence level.
[0106] The input image to be detected is processed by the backbone extraction network for feature extraction. The multi-scale feature fusion network CMPPN, which connects across layers, fuses the multi-scale features and inputs them into three detectors. Each loss function predicts the location and category of defects in the image to be detected. Finally, the output is a bounding box with classification category and confidence level, which outlines the region of defects in the image to be detected, and the detection result is output.
[0107] Qualitative evaluation results:
[0108] To intuitively evaluate the improvement in small-scale defect detection performance between the method of this invention and the YOLOv8 model method, four sets of fabric images containing multiple micro-defects and different backgrounds were selected for testing, such as... Figure 6 The image shows a comparison of the detection results of the YOLOv8 model method and the method of this invention. The first group shows the detection results of fabric defects against a solid-color fabric background, the second group shows the detection results of fabric defects against a wheat-ear texture background, the third group shows the detection results of fabric defects against a background of regular diagonal stripes, and the fourth group shows the detection results of fabric defects against a background of horizontal and vertical stripes.
[0109] contrast Figure 6 (a) Figure 6 (b) It can be seen that the YOLOv8 model method misses the defect point in the lower left corner, while the method of this invention can extract more detailed defect features, thus enabling the detection of extremely subtle micro-defects, and the confidence level of all defects is higher than that of the YOLOv8 model method. (Comparison) Figure 6 (c), 6(d) and Figure 6 As can be seen in (e) and 6(f), both methods can detect defects of a larger scale, but the detection box of the method of this invention is more accurate, and the confidence level is significantly higher than that of the YOLOv8 model method. For tiny defect points in the image that are highly similar to the fabric background color and are difficult to distinguish with the naked eye, the detection effect of the YOLOv8 model method is poor, with significant missed detections. However, the method of this invention uses a hybrid loss function to improve the detection capability of tiny defects. Therefore, it can detect all the tiny defects in the image, improving the situation of missing small defects. Comparison Figure 6As can be seen from (g) and (h), when detecting dense small-scale defects against a background of equally complex textures, the YOLOv8 model method misses two defects and has a low accuracy, while the method of this invention not only detects all defects but also improves the confidence level of each defect. Through direct visual comparison of the above defect detection results, it is evident that the method of this invention significantly outperforms the YOLOv8 model method in detecting small-scale defects in complex scenes.
[0110] Quantitative evaluation results:
[0111] To objectively evaluate the improvement in detection performance between the method of this invention and the YOLOv8 model method, a comparative experiment was conducted on the detection performance of five different types of fabric defects using parameter indices. The experimental results are shown in Table 1.
[0112] Table 1 compares the P-value and mAP@0.5 of the detection results for various defect categories using the method of this invention and the YOLOv8 model method.
[0113]
[0114] As can be seen from the data comparison in Table 1, the method of the present invention improves the P and mAP@0.5 indices for detecting multiple types of defects to varying degrees compared with the YOLOv8 model method. Specifically, the FDDM-EHL method improves the detection accuracy of stains, knots, broken yarns, stray threads, and holes by 2%, 2.9%, 6.3%, 0.2%, and 0.4%, respectively, with an overall average improvement of 3.5%; and improves the detection precision of stains, knots, broken yarns, stray threads, and holes by 0.2%, 6.7%, 7.5%, 0.2%, and 4.1%, respectively, with an overall average improvement of 3.7%. These results indicate that the method of the present invention improves the detection accuracy and precision of each type of defect to varying degrees. The improvement is most significant for yarn defects, while the accuracy and precision for knot defects are the highest, and the detection performance is significantly better than the YOLOv8 model method.
[0115] Example 1:
[0116] A fabric defect detection method based on EFC feature extraction and hybrid loss function includes the following steps:
[0117] Step 1: Collect 800 images of defective fabrics and perform preprocessing such as scaling, rotation, stitching, and random coloring to expand the images to 1123 images and uniformly name them as ####.png. Randomly divide the dataset into training and evaluation sets in a 9:1 ratio. Use LabelImg to annotate the defective regions on the defective fabric images in the training and evaluation sets. The annotation content includes the location information of the fabric defect, the image path, and the defect category. Generate annotation files and save them to the corresponding training and evaluation sets.
[0118] Step 2: Improve the YOLOv8 model and construct a fabric defect detection network;
[0119] The YOLOv8 model includes a backbone network, a neck network, and a head network. The YOLOv8 model is improved to construct a fabric defect detection network, specifically including the following steps:
[0120] Step 21: In the C2f feature extraction module of the YOLOv8 model backbone network, modify the Bottleneck module to the EMA-FasterNet Block module to construct a backbone network based on the EFC feature extraction module; the EMA-FasterNetBlock module includes convolutional layers, multi-scale attention EMA modules, normalization layers and ReLU activation function;
[0121] Step 22: In the neck network of the YOLOv8 model, modify the Feature Pyramid Fusion Network (PAN) to a multi-scale feature fusion network (CMFPN) with cross-layer connectivity. The multi-scale feature fusion network (CMFPN) with cross-layer connectivity includes a downsampling layer, a fusion layer, and a C2f feature fusion module.
[0122] Step 23: In the head network of the YOLOv8 model, modify the CIoU loss function of the bounding box loss to the HCN Loss, a hybrid loss function based on similarity metrics. The HCN Loss includes normalized Gaussian Wasserstein distance and CIoU loss function.
[0123] Step 3: Train the constructed fabric defect detection network using the training set, and evaluate the performance of the trained fabric defect detection network using the evaluation set to obtain the final fabric defect detection model; input the image to be tested into the fabric defect detection model for detection, and obtain the detection results, which include defect type and confidence level.
[0124] Step 31: Use the training set as the training path to train the constructed fabric defect detection network; the training input is a 640x640 pixel fabric image containing defects, and the number of fabric images containing defects in each input is 16. Iterate for 200 rounds until the training ends, and output the model file, which contains the network structure and parameters of the fabric defect detection network.
[0125] Step 32: Use the model file obtained in Step 31 and the evaluation set obtained in Step 1 to evaluate the performance of the trained fabric defect detection network and obtain the final fabric defect detection model.
[0126] Step 33: Input the image to be tested into the fabric defect detection model and output the detection results with defect type and confidence level.
[0127] Example 2:
[0128] A fabric defect detection method based on EFC feature extraction and hybrid loss function includes the following steps:
[0129] Step 1: Collect 800 images of defective fabrics and perform preprocessing such as scaling, rotation, stitching, and random coloring to expand the images to 1123 images and uniformly name them as ####.png. Randomly divide the dataset into training and evaluation sets in a 9:1 ratio. Use LabelImg to annotate the defective regions on the defective fabric images in the training and evaluation sets. The annotation content includes the location information of the fabric defect, the image path, and the defect category. Generate annotation files and save them to the corresponding training and evaluation sets.
[0130] Step 2: Improve the YOLOv8 model and construct a fabric defect detection network;
[0131] The YOLOv8 model includes a backbone network, a neck network, and a head network. The YOLOv8 model is improved to construct a fabric defect detection network, specifically including the following steps:
[0132] Step 21: In the C2f feature extraction module of the YOLOv8 model backbone network, modify the Bottleneck module to the EMA-FasterNet Block module to construct a backbone network based on the EFC feature extraction module; the EMA-FasterNetBlock module includes convolutional layers, multi-scale attention EMA modules, normalization layers and ReLU activation function;
[0133] The EMA-FasterNet Block module consists of a partially convolutional PConv layer, connected to an EMA module, followed by two PWConv layers, with residual connections to reuse the initial features of the input, and a BN layer and a ReLU activation layer connected between the two PWConv layers.
[0134] The EFC feature extraction module consists of two convolutional modules (Conv) and several EMA-FasterNet Block modules, connected by Split and Concat.
[0135] Step 22: In the neck network of the YOLOv8 model, modify the Feature Pyramid Fusion Network (PAN) to a multi-scale feature fusion network (CMFPN) with cross-layer connectivity. The multi-scale feature fusion network (CMFPN) with cross-layer connectivity includes a downsampling layer, a fusion layer, and a C2f feature fusion module.
[0136] The multi-scale feature fusion network CMPPN with cross-layer connectivity is obtained by removing nodes that do not have feature fusion based on the combination of FPN and PAN.
[0137] Step 23: In the head network of the YOLOv8 model, modify the CIoU loss function of the bounding box loss to the HCN Loss, a hybrid loss function based on similarity metrics. The HCN Loss includes normalized Gaussian Wasserstein distance and CIoU loss function.
[0138] The construction process of the HCN Loss, a hybrid loss function based on similarity metrics, is as follows:
[0139] Step 231: Model the bounding box as a two-dimensional Gaussian distribution;
[0140] Step 232: Calculate the distribution distance using Wasserstein distance for the predicted bounding box P = (c xp ,c yp ,w p ,h p ) and the true bounding box G = (c xg ,c yg ,w g ,h g Gaussian distribution model N P and N G The normalized Wasserstein distance between the two is calculated using the following formula:
[0141]
[0142] In the formula, It is the Wasserstein distance between the ground truth bounding box and the predicted bounding box, c xp ,c yp ,w p ,h p These represent the center coordinates, width, and height of the predicted bounding box, respectively; c xg ,c yg ,w g ,h g These represent the center coordinates, width, and height of the actual bounding box, respectively;
[0143] Step 233: Normalize formula (4) to the form of IoU ratio, and... Converting the value to a range of 0 to 1, we obtain the normalized Wasserstein distance (NWD), as shown in the following formula:
[0144]
[0145] In the formula, C0 is a constant;
[0146] Loss function based on NWD as a similarity measure NWD The calculation formula is as follows:
[0147]
[0148] Step 234: Construct a hybrid loss function HCN Loss based on similarity metrics, as shown in the following formula:
[0149] Loss HCN = (1-K)Loss CIoU +K(1-NWD)=(1-K)Loss CIoU +KLoss NWD (7)
[0150] Among them, Loss CIoU Let CIoU represent the loss function, and K represent the weight adjustment coefficients for the CIoU loss function and the NWD loss function.
[0151] Step 3: Train the constructed fabric defect detection network using the training set, and evaluate the performance of the trained fabric defect detection network using the evaluation set to obtain the final fabric defect detection model; input the image to be tested into the fabric defect detection model for detection, and obtain the detection results, which include defect type and confidence level.
[0152] Step 31: Use the training set as the training path to train the constructed fabric defect detection network; the training input is a 640x640 pixel fabric image containing defects, and the number of fabric images containing defects in each input is 16. Iterate for 200 rounds until the training ends, and output the model file, which contains the network structure and parameters of the fabric defect detection network.
[0153] Step 32: Use the model file obtained in Step 31 and the evaluation set obtained in Step 1 to evaluate the performance of the trained fabric defect detection network and obtain the final fabric defect detection model.
[0154] Step 33: Input the image to be tested into the fabric defect detection model and output the detection results with defect type and confidence level.
[0155] Example 3:
[0156] A fabric defect detection method based on EFC feature extraction and hybrid loss function includes the following steps:
[0157] Step 1: Collect 800 images of defective fabrics and perform preprocessing such as scaling, rotation, stitching, and random coloring to expand the images to 1123 images and uniformly name them as ####.png. Randomly divide the dataset into training and evaluation sets in a 9:1 ratio. Use LabelImg to annotate the defective regions on the defective fabric images in the training and evaluation sets. The annotation content includes the location information of the fabric defect, the image path, and the defect category. Generate annotation files and save them to the corresponding training and evaluation sets.
[0158] Step 2: Improve the YOLOv8 model and construct a fabric defect detection network;
[0159] The YOLOv8 model includes a backbone network, a neck network, and a head network. The YOLOv8 model is improved to construct a fabric defect detection network, specifically including the following steps:
[0160] Step 21: In the C2f feature extraction module of the YOLOv8 model backbone network, modify the Bottleneck module to the EMA-FasterNet Block module to construct a backbone network based on the EFC feature extraction module; the EMA-FasterNetBlock module includes convolutional layers, multi-scale attention EMA modules, normalization layers and ReLU activation function;
[0161] The EMA-FasterNet Block module consists of a partially convolutional PConv layer, connected to an EMA module, followed by two PWConv layers, with residual connections to reuse the initial features of the input, and a BN layer and a ReLU activation layer connected between the two PWConv layers.
[0162] The EFC feature extraction module consists of two convolutional modules (Conv) and several EMA-FasterNet Block modules, connected by Split and Concat.
[0163] Step 22: In the neck network of the YOLOv8 model, modify the Feature Pyramid Fusion Network (PAN) to a multi-scale feature fusion network (CMFPN) with cross-layer connectivity. The multi-scale feature fusion network (CMFPN) with cross-layer connectivity includes a downsampling layer, a fusion layer, and a C2f feature fusion module.
[0164] The multi-scale feature fusion network CMPPN with cross-layer connectivity is obtained by removing nodes that do not have feature fusion based on the combination of FPN and PAN.
[0165] Step 23: In the head network of the YOLOv8 model, modify the CIoU loss function of the bounding box loss to the HCN Loss, a hybrid loss function based on similarity metrics. The HCN Loss includes normalized Gaussian Wasserstein distance and CIoU loss function.
[0166] The construction process of the HCN Loss, a hybrid loss function based on similarity metrics, is as follows:
[0167] Step 231: Model the bounding box as a two-dimensional Gaussian distribution;
[0168] Step 232: Calculate the distribution distance using Wasserstein distance for the predicted bounding box P = (c xp ,c yp ,w p ,h p ) and the true bounding box G = (c xg ,c yg ,w g ,h g Gaussian distribution model N P and N G The normalized Wasserstein distance between the two is calculated using the following formula:
[0169]
[0170] In the formula, It is the Wasserstein distance between the ground truth bounding box and the predicted bounding box, c xp ,c yp ,w p ,h p These represent the center coordinates, width, and height of the predicted bounding box, respectively; c xg ,c yg ,w g ,h g These represent the center coordinates, width, and height of the actual bounding box, respectively;
[0171] Step 233: Normalize formula (4) to the form of IoU ratio, and... Converting the value to a range of 0 to 1, we obtain the normalized Wasserstein distance (NWD), as shown in the following formula:
[0172]
[0173] In the formula, C0 is a constant;
[0174] Loss function based on NWD as a similarity measure NWD The calculation formula is as follows:
[0175]
[0176] Step 234: Construct a hybrid loss function HCN Loss based on similarity metrics, as shown in the following formula:
[0177] Loss HCN = (1-K)Loss CIoU +K(1-NWD)=(1-K)Loss CIoU +KLoss NWD (7)
[0178] Among them, Loss CIoU Let CIoU represent the loss function, and K represent the weight adjustment coefficients for the CIoU loss function and the NWD loss function.
[0179] Step 3: Train the constructed fabric defect detection network using the training set, and evaluate the performance of the trained fabric defect detection network using the evaluation set to obtain the final fabric defect detection model; input the image to be tested into the fabric defect detection model for detection, and obtain the detection results, which include defect type and confidence level.
[0180] Step 31: Use the training set as the training path to train the constructed fabric defect detection network; the training input is a 640x640 pixel fabric image containing defects, and the number of fabric images containing defects in each input is 16. Iterate for 200 rounds until the training ends, and output the model file, which contains the network structure and parameters of the fabric defect detection network.
[0181] The total loss function used in training the fabric defect detection network is:
[0182] Loss = w cls L cls +w box L box +w obj L obj (1)
[0183] w cls +w box +w obj =1 (2) In the formula, L cls For the classification loss function, L box Let L be the bounding box loss function. obj Let w be the confidence loss function. cls ,w box ,w obj These are the weight coefficients corresponding to the classification loss function, bounding box loss function, and confidence loss function, respectively. The classification loss function adopts the binary cross-entropy loss function (BCE), the bounding box loss function adopts the hybrid loss function HCN Loss based on similarity measurement, and the confidence loss function adopts the distributed focus loss function (DFL).
[0184] Step 32: Use the model file obtained in Step 31 and the evaluation set obtained in Step 1 to evaluate the performance of the trained fabric defect detection network and obtain the final fabric defect detection model.
[0185] Evaluation metrics include: model detection accuracy P, model detection performance R under positive sample conditions, detection accuracy AP for various defects, and overall detection accuracy mAP for various defects; the calculation formulas for the above metrics are as follows:
[0186] P = TP / (TP + FP) (10)
[0187] R = TP / (TP + FN) (11) where TP represents the number of samples correctly predicted as positive, FP represents the number of negative samples incorrectly predicted as positive, and FN represents the number of positive samples incorrectly predicted as negative.
[0188]
[0189] In the formula, the P(R) function is a curve enclosed by the detection accuracy P of the index model as the vertical axis and the detection effect R of the index model under the positive sample condition as the horizontal axis. The AP value is obtained by summing the curves using an integral function.
[0190]
[0191] In the formula, K is a constant related to the dataset, i.e., the number of defect types, and mAP value is the average AP value of each defect obtained by summing and averaging through an integral function.
[0192] Step 33: Input the image to be tested into the fabric defect detection model and output the detection results with defect type and confidence level.
Claims
1. A fabric defect detection method based on EFC feature extraction and hybrid loss function, characterized in that, Includes the following steps: Step 1: Collect images of fabrics with defects and preprocess them. Randomly divide the dataset into training and evaluation sets according to the proportion. Use LabelImg to annotate the defective areas on the fabric images with defects in the training and evaluation sets, generate annotation files, and save them into the corresponding training and evaluation sets. Step 2: Improve the YOLOv8 model and construct a fabric defect detection network; The YOLOv8 model includes a backbone network, a neck network, and a head network. The YOLOv8 model is improved to construct a fabric defect detection network, specifically including the following steps: Step 21: In the C2f feature extraction module of the YOLOv8 model backbone network, the Bottleneck module is modified to the EMA-FasterNet Block module to construct a backbone network based on the EFC feature extraction module; the EMA-FasterNetBlock module includes convolutional layers, multi-scale attention EMA modules, normalization layers and ReLU activation functions; The EMA-FasterNet Block module includes a partially convolutional PConv layer, connected to an EMA module, followed by two PWConv layers, with residual connections to reuse the initial features of the input, and a BN layer and a ReLU activation layer connected between the two PWConv layers. The EFC feature extraction module consists of two convolutional Convs and several EMA-FasterNet Block modules, connected by Split and Concat. Step 22: In the neck network of the YOLOv8 model, the Feature Pyramid Fusion Network (PAN) is modified into a cross-layer connected Multi-Scale Feature Fusion Network (CMFPN). The cross-layer connected Multi-Scale Feature Fusion Network (CMFPN) includes a downsampling layer, a fusion layer, and a C2f feature fusion module. The cross-layer connected multi-scale feature fusion network CMPPN is obtained by removing nodes without feature fusion based on the combination of FPN and PAN; Step 23: In the head network of the YOLOv8 model, modify the CIoU loss function of the bounding box loss to the hybrid loss function HCN Loss based on similarity metric. The hybrid loss function HCN Loss based on similarity metric includes normalized Gaussian Wasserstein distance and CIoU loss function. Step 3: Train the constructed fabric defect detection network using the training set, and evaluate the performance of the trained fabric defect detection network using the evaluation set to obtain the final fabric defect detection model; input the image to be tested into the fabric defect detection model for detection, and obtain the detection results, which include defect type and confidence level.
2. The fabric defect detection method based on EFC feature extraction and hybrid loss function according to claim 1, characterized in that, In step 1, the defective fabric images are preprocessed, including scaling, rotating, stitching, and random color transformation to expand the dataset. In step 1, LabelImg is used to annotate the defective areas on the defective fabric images. The annotation content includes the location information of the fabric defects, the image path, and the defect category. The defect categories are stains, holes, broken yarns, knots, and stray threads.
3. The fabric defect detection method based on EFC feature extraction and hybrid loss function according to claim 1, characterized in that, The construction process of the HCN Loss based on similarity metric described in step 23 is as follows: Step 231: Model the bounding box as a two-dimensional Gaussian distribution; Step 232: Calculate the distribution distance using Wasserstein distance for the predicted bounding box. and the true bounding box Gaussian distribution model and Calculate the normalized Wasserstein distance between the two, using the following formula: (4) In the formula, It is the Wasserstein distance between the ground truth bounding box and the predicted bounding box. These represent the center coordinates, width, and height of the predicted bounding box, respectively. These represent the center coordinates, width, and height of the actual bounding box, respectively; Step 233: Normalize formula (4) to the form of IoU ratio, and... Converting the value to a range of 0 to 1, we obtain the normalized Wasserstein distance (NWD), as shown in the following formula: (5) In the formula, It is a constant; Loss function based on NWD as a similarity measure The calculation formula is as follows: (6) Step 234: Construct a hybrid loss function HCN Loss based on similarity metrics, as shown in the following formula: (7) in, Represents the CIoU loss function. This represents the weight adjustment coefficients for the CIoU loss function and the NWD loss function.
4. The fabric defect detection method based on EFC feature extraction and hybrid loss function according to claim 1, characterized in that, The fabric defect detection network described in step 2 includes an input end, a backbone network, a neck network, and a head network. The backbone network contains a convolutional Conv, several EFC feature extraction modules, and an SPPF module connected in sequence. A Merging module is connected between every two EFC feature extraction modules. The Merging module consists of a convolutional Conv, a BN layer, and a SiLU activation function. The neck network consists of two upsampling modules and several C2f modules. The 16x downsampling to form 40×40 features is fused with the network layers through cross-layer connections using a multi-scale feature fusion network CMPPN.
5. The fabric defect detection method based on EFC feature extraction and hybrid loss function according to claim 1, characterized in that, The total loss function used in the training of the fabric defect detection network is: (1) (2) In the formula, For classification loss function, The bounding box loss function is... For confidence loss function, These are the weight coefficients corresponding to the classification loss function, bounding box loss function, and confidence loss function, respectively. The classification loss function adopts the binary cross-entropy loss function (BCE), the bounding box loss function adopts the hybrid loss function HCLOSs based on similarity measurement, and the confidence loss function adopts the distributed focus loss function (DFL).
6. The fabric defect detection method based on EFC feature extraction and hybrid loss function according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 31: Use the training set as the training path to train the constructed fabric defect detection network; the training input is a fabric image with defective pixels of 640x640, and the number of fabric images with defective pixels is 16 each time. Iterate for 200 rounds until the end of training, and output the model file, which contains the network structure and parameters of the fabric defect detection network. Step 32: Use the model file obtained in Step 31 and the evaluation set obtained in Step 1 to evaluate the performance of the trained fabric defect detection network and obtain the final fabric defect detection model. Step 33: Input the image to be tested into the fabric defect detection model, and output the detection results with defect type and confidence level.
7. The fabric defect detection method based on EFC feature extraction and hybrid loss function according to claim 6, characterized in that, Step 32 uses the model file and evaluation set to evaluate the performance of the trained fabric defect detection network. The evaluation metrics include: model detection accuracy. The detection performance of the model under positive class sample conditions Detection accuracy of various defects and the comprehensive detection accuracy of various defects. The calculation formulas for the above indicators are as follows: (10) (11) In the formula, This represents the number of samples that were correctly predicted as the positive class. This represents the number of negative samples that were incorrectly predicted as positive. This indicates the number of positive samples that were incorrectly predicted as the negative class. (12) In the formula, The function is used to detect the accuracy rate using an index model. The vertical axis represents the detection performance of the index model under positive class sample conditions. The curve bounded by the horizontal axis is obtained by summing using an integral function. value; (13) In the formula, This is a constant related to the dataset, namely the number of defect types. The average value is obtained by summing and averaging the values of each defect using an integral function. value.
Citation Information
Patent Citations
Road target detection method and system based on improved YOLOv8
CN117037119A
Fabric defect detection method and system based on YOLOv8 network
CN117576038A
