A high-efficiency real-time detection method and system for fabric surface defect detection

Abstract

The application discloses a kind of high-efficiency real-time detection method and system for fabric surface flaw detection, method as follows: fabric surface flaw dataset is collected, data set is carried out data enhancement, and data set is divided;X image in training set is randomly selected and input into selected backbone network, feature extraction is carried out, fusion feature map is carried out, and N prediction feature maps are obtained by multi-scale adaptive perception detection head;For N prediction feature maps, boundary box is generated;Training is carried out, and the network of training completion is obtained;Prediction is carried out to validation set, and all candidate boxes are generated;Candidate box is sorted according to confidence score first, then adjusted candidate box is carried out non-maximum suppression NMS to remove redundant prediction box, and final prediction box is obtained;According to proportional relationship, prediction box on feature map scale is mapped to original map scale, and the positioning of fabric image flaw is realized.The application is more accurate for the identification precision of fabric image surface flaw, and detection speed is faster.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial defect detection, and in particular relates to a method and system for detecting fabric surface defects based on an improved YOLOv5s model. Background Art

[0002] The fabric industry plays a vital role in China's national economic development. As people's living standards improve, the quality requirements of light industrial products across all industries continue to rise. Therefore, ensuring fabric quality is crucial for both manufacturers and consumers. Fabric surface defects not only affect product prices but also determine market competitiveness and consumer satisfaction.

[0003] In modern fabric production, professional quality inspectors are often required to examine the fabric surface for defects. However, this manual inspection method is inefficient and highly subjective. Inspectors may become fatigued, leading to false or missed inspections, which can negatively impact product quality and production efficiency. Therefore, automated fabric defect detection technology is essential.

[0004] In recent years, deep convolutional neural networks (CNNs) have achieved remarkable success in tasks such as image classification, detection, and segmentation, garnering extensive research. In the field of object detection, numerous outstanding research achievements have emerged, such as the R-CNN and YOLO series of detection models, which continue to evolve. The YOLO series has attracted considerable attention for its fast detection speed and lightweight nature, and its application has expanded to a wider range of fields.

[0005] Although existing detection technologies have made significant progress, they still face challenges in fabric defect detection, such as complex background interference and irregular shapes that make it difficult to identify fabric defects. Therefore, the present invention proposes an efficient, real-time detection method and system for fabric surface defect detection. Summary of the Invention

[0006] In response to the aforementioned status quo of the existing technology, the present invention proposes an efficient, real-time detection method and system for fabric surface defect detection. This method introduces the EASE-enhanced arbitrary shape feature extraction structure at specific locations in the network to capture the semantic features of irregularly shaped fabric defects and perform feature focusing. Furthermore, a new feature fusion structure, AD-FPN, and a novel multi-scale adaptive sensing detection head, MASH, are designed to address the complex background interference and irregular shapes of fabric defects that are difficult to identify in fabric defect detection tasks. Using this method for fabric image defect detection offers significant advantages in both detection accuracy and efficiency over existing advanced detection technologies, effectively meeting the real-time requirements of practical scenarios.

[0007] The present invention adopts the following technical solutions:

[0008] An efficient real-time detection method for fabric surface defect detection comprises the following steps:

[0009] S1, collect fabric image dataset;

[0010] S2, perform data augmentation on the fabric image dataset;

[0011] S3. Divide the enhanced dataset into a training set, a validation set, and a test set according to a preset ratio, and save the corresponding image name list, label position, and category information;

[0012] S4. Randomly select X images from the training set as network input. Select the CSPDarknet backbone for feature extraction to obtain N effective feature maps of different scales. Insert the proposed EASE module into the last feature extraction module to optimize the feature extraction capability of the backbone. Then, perform feature fusion on the obtained effective feature maps using the lightweight AD-FPN to further fuse them, obtaining N fused feature maps of different scales. Finally, the fused feature maps are processed using the multi-scale adaptive perceptual detection head (MASH) to integrate different local context information, providing dynamic receptive fields for features of different scales to obtain N predicted feature maps.

[0013] S5. For the N predicted feature maps, predict the probability that each pixel in the image belongs to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and then generate a bounding box based on this information; calculate the network loss value, bounding box confidence loss, and intersection over union (IOU) loss based on the obtained predicted box and the GT box of the corresponding image, and use the cross-entropy loss (MSE) to calculate the position loss. The category loss measures the difference between the probability distribution of each category in the predicted box and the object category in the ground truth box. The cross-entropy loss is used to calculate the category loss. The position loss and the category loss are added together to obtain the total loss of the predicted box. The parameters of the network are updated using the gradient descent method; the operation is repeated until all images in the training set are input into the network in batches. The network with updated parameters predicts each image in the validation set, and the AP value of each category in the validation set is output after statistics; the calculation process of step S5 is repeated until the mAP counted in step S5 stabilizes at a certain value, and the trained network is obtained;

[0014] S6. During the prediction process of the validation set using the network trained in step S5, the network outputs the center coordinates of the candidate box, the length and width of the box, and the category probability, generates all candidate boxes, sorts the candidate boxes according to the confidence score, and then performs non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes to obtain the final prediction box;

[0015] S7. Use the trained network to predict all images in the test set and obtain the predicted boxes on the feature map. Based on the proportional relationship between the feature map and the original image, the predicted boxes are mapped to the corresponding original image to locate the defects.

[0016] As a preferred solution, in step S1, the Ali Tianchi fabric dataset is selected. The fabric surface defect dataset contains multiple defect images I of different categories and corresponding label files. The label files contain the real-box marking information of the objects in the images; each image corresponds to a txt format annotation file, in which each defect in the image is annotated with the category and location.

[0017] As a preferred solution, in step S1, the annotation of category and position is used as a real box, called Ground Truth (GT). The object is a fabric surface defect, and the GT box labeling data is (c, x1, y1, x2, y2 y2), where c represents the category of the object contained in the GT box, x1 and y1 represent the x-coordinate and y-coordinate of the upper left corner vertex in the GT box, respectively, and x2 and y2 represent the x-coordinate and y-coordinate of the lower right corner vertex in the GT box.

[0018] As a preferred solution, in step S1, the width and height of the fabric image are 2048×1696.

[0019] As a preferred solution, in step S2, data enhancement (HSV transformation, random rotation, translation, scaling, cropping, etc.) is performed on the fabric image dataset to expand the dataset to obtain an enhanced fabric image dataset.

[0020] As a preferred solution, in step S2, the fabric image dataset is enhanced using Mosaic data enhancement combined with Mixup data enhancement with a probability of 20%. The Mosaic data enhancement is used to randomly scale, randomly crop, and randomly arrange any four pictures in the dataset to obtain new pictures and add them to the dataset to expand the dataset.

[0021] As a preferred solution, in step S3, the ratio of the divided training set, validation set, and test set is 0.75:0.15:0.15, and train.txt, val.txt, and test.txt files are generated to save the corresponding image name lists and the label positions and category information therein.

[0022] As a preferred solution, in step S3, the generated file format is divided into rows for each picture. Taking the train.txt file as an example, each row sequentially stores the path of the picture divided into the training set, the position information of all the annotation boxes in the picture (x1, y1, x2, y2) and the category information.

[0023] As a preferred solution, step S4 is specifically as follows:

[0024] S4.1. Replace the original Bottleneck neck feature enhancement structure of the last C3 with the EASE structure to obtain C3EASE. The output of C3EASE is used as the input of the SPPF multi-scale feature extraction module to obtain the output M5 of the feature backbone extraction network.

[0025] S4.2. The three valid feature maps obtained in S4.1 are named M3, M4, and M5 in ascending order of receptive field. They are input into the improved feature fusion module AD-FPN (M5 improved by C3EASE has better feature extraction performance. M3, M4, and M5 are used as inputs of AD-FPN) for multi-scale fusion. After cross-layer feature fusion guided by aggregation distribution, features from different layers are utilized to output three feature maps K3, K4, and K5.

[0026] S4.3. For the three feature maps K3, K4, and K5 obtained, the MASH module is used to provide dynamic receptive fields for features of different scales.

[0027] As a preferred solution, in step S4, the existing backbone network Darknet is mainly composed of three C3 modules, and the last one includes a C3EASE module, which is inserted into the last C3 module. The three effective feature layers output by the backbone feature extraction network are fused using a lightweight AD-FPN, and then the output of AD-FPN is further enhanced using MASH to integrate the features of each feature layer. Step S4 specifically includes the following steps:

[0028] S4.1. Feature extraction: X images are randomly selected from the training set and scaled. The scales are then input into the selected backbone network CSPDarknet for multi-scale feature extraction. Three effective feature layers with different scales are output, with scales of 20×20, 40×40, and 80×80 respectively. The deepest feature map is passed through the fast spatial pyramid pooling structure (SPPF) to obtain a feature map M5.

[0029] Step S4.1 specifically includes the following steps:

[0030] S4.1.1. Replace the original Bottleneck neck feature enhancement structure of the last C3 with the EASE structure. The EASE structure is obtained by connecting the AKConv arbitrary shape sampling convolution and the Siam self-similarity focusing structure in series.

[0031] S4.1.2. The output of C3EASE is used as the input of the SPPF multi-scale feature extraction module to obtain the output M5 of the feature backbone extraction network. M5 is the feature layer with the largest receptive field.

[0032] S4.2, feature fusion: The three valid feature maps obtained in step S4.1 (referred to as M3, M4, and M5 according to the receptive fields from small to large) are input into the improved feature fusion module AD-FPN for further fusion. The output feature maps correspond to the output feature maps of the backbone network, which are 80×80, 40×40, and 20×20, respectively. The cross-layer feature fusion guided by aggregation distribution makes full use of the features of different layers. For each layer of the output feature maps K3, K4, and K5 of the feature fusion module, their scales are consistent with the input feature maps M3, M4, and M5. Among them, the output of the backbone feature extraction network is used as the input of AD-FPN to generate intermediate-scale feature maps with smaller scale gaps for multi-scale feature fusion enhancement, and then the aggregated features are distributed to other scales for multi-scale feature fusion. AD-FPN extracts multi-scale features from SPPF and simplifies some of the calculation nodes to obtain an approximate bidirectional pyramid structure.

[0033] As a preferred solution, step S4.2 specifically includes the following steps:

[0034] S4.2.1. Sampling the M5 and M4 feature maps by 1.5 and 0.75 times, respectively, adds the results and performs a 3x3 convolution to obtain an intermediate-scale feature map of 30x30. The intermediate feature map of 30x30 is again added to the result of performing non-integer sampling and adaptively supplemented with feature information of different scales through a 3x3 convolution, thereby obtaining an intermediate-scale feature map K6 with richer feature information. Similarly, performing the same operation on the M4 and M3 feature maps yields an intermediate-scale feature map of 60x60, called K7, which is described by the following formula:

[0035] K6=Conv 3×3 (Conv 3×3 (↑ ×1.5 M5+↓ ×0.75 M4)+↓ ×0.75 M4+↑ ×1.5 M5)

[0036] K7=Conv 3×3 (Conv 3×3 (↑ ×1.5 M4+↓ ×0.75 M3)+↓ ×0.75 M3+↑ ×1.5 M4)

[0037] ↑ ×1.5 Refers to 1.5 times upsampling,↓ ×0.75 Refers to 0.75 times downsampling.

[0038] S4.2.2. K6, K7 and M4 are concatenated and fused through 1x1 convolution to obtain the aggregate feature K4. K4 is passed through the MFFM structure. MFFM includes an attention mechanism based on parallel multi-scale learning and a series of convolution operations. Multi-scale fusion is used to guide feature enhancement to obtain an aggregate feature map K4 with more obvious features. The formula is described as follows:

[0039] K4=MF(Conv 1×1 (Concat(K6,M4,K7)))

[0040] Concat refers to the concatenation operation, and MF refers to the MFFM multi-scale fusion operation.

[0041] S4.2.3. Sample the K4 feature map and concatenate and fuse it with the M3 feature map. At the same time, perform feature enhancement through the MFFM module to obtain the low-level output feature map K3. The formula is as follows:

[0042] K3=MF(Conv 1×1 (Concat(M3, K4)))

[0043] S4.2.4. Sample the K4 feature map and concatenate and fuse it with the M5 feature map. At the same time, perform feature enhancement through the MFFM module to obtain the deepest semantic output feature map K5. The formula is described as follows:

[0044] K5=MF(Conv 1×1 (Concat(M5,K4)))

[0045] As a preferred solution, step S4.3 specifically includes the following steps:

[0046] S4.3. Feature Detection Head: The MASH module is used to provide dynamic receptive fields for features at different scales using the three feature maps K3, K4, and K5 obtained in the above steps. Taking the AD-FPN feature output K3 as an example, K3 integrates rich local context through the MASH multi-scale adaptive perception detection head. Depthwise separable convolutions with different dilation ratios are performed on K3. A set of adaptive spatial parameters are used to modulate the different receptive fields captured by multiple parallel branches. Finally, a 1x1 convolution is performed to obtain the final output P3. Similarly, P4 and P5 can be obtained, as described by the following formulas:

[0047]

[0048] DSConv is a depth-wise separable convolution, where α, β, and γ are a set of adaptively adjusted spatial parameters. represents element-wise multiplication, and d represents the dilation ratio.

[0049] After the training is completed, the output of S4.3 adjusts the number of channels to obtain the prediction information on the feature map, and the prediction information is used to enlarge the position of the prediction box on the original image according to the ratio of the feature map to the input image.

[0050] As a preferred solution, in step S4, the EASE module is inserted after the last C3 module to capture the semantics of irregular defect features.

[0051] As a preferred solution, in step S4, non-integer sampling is performed on the three feature maps to obtain intermediate-scale feature layers with scales of 20×20, 30×30, 40×40, 60×60, and 80×80, respectively. The units on the feature maps are usually called "grid cells." These grids are used to represent different regions in the image space, and object detection prediction is performed within each grid cell.

[0052] In step S4, the EASE feature extraction module is embedded in the backbone network of CSPDarknet to improve the feature extraction capability, while the aggregation distribution guidance strategy is used in the path aggregation network to enhance the feature fusion capability of the network architecture.

[0053] As a preferred solution, step S5 specifically involves directly predicting the N predicted feature maps using the YOLO detection process. The number of channels in the feature map output by the network is adjusted to 3*(5+num_class) using a lightweight YOLO head. This number of channels represents the number of channels required for network prediction. The channels include the probability of each pixel belonging to an object, the center point location, the width and height of the object's bounding box, and the probability of the detected object class, where num_class represents the number of predicted classes. Based on the location of the ground-truth box, the corresponding prior bounding box is found. The ground-truth and label information are converted into a vector of length 5+num_class. This prior bounding box is then compared with each prior bounding box vector in the predicted feature map to calculate the loss and update the network parameters. An epoch is the process of feeding all images in the training set into the network for forward propagation and reverse optimization of the network parameters. After each epoch, the network with updated parameters is used to predict each image in the validation set. The AP metric for each class in the validation set is calculated based on the predictions and ground-truth for each image. Network convergence is determined by the AP value remaining constant or showing a downward trend over multiple rounds. Otherwise, training continues with the next epoch. For each image, the network outputs the corresponding N predicted feature maps, and then the algorithm directly predicts the probability of belonging to an object in the image, the probability of which category the object to be detected belongs to, and the bounding box information of the object.

[0054] As a preferred solution, step S5 uses the YOLO series detection processing method to directly predict the probability of each pixel in the image belonging to an object.

[0055] As a preferred solution, step S5 obtains prediction information, trains the network, and integrates the output features into the channel information required for prediction.

[0056] As a preferred solution, step S6 generates all prediction boxes for each image to be predicted based on the trained network weights, and then uses non-maximum suppression (NMS) to obtain the prediction boxes on the feature map.

[0057] As a preferred solution, in step S7, specifically, the predicted box on the feature map scale is finally mapped to the original image scale according to the proportional relationship, thereby realizing the positioning of defects on the fabric image.

[0058] The present invention also discloses an efficient real-time detection system for fabric surface defect detection, which is used to execute the above method and includes the following modules:

[0059] Data acquisition module: collects fabric surface defect data sets;

[0060] Data enhancement module: performs data enhancement on the fabric dataset;

[0061] Dataset partitioning module: divides the data set after data enhancement into training set, validation set, and test set, and saves the corresponding image name list, label position, and category information;

[0062] Prediction feature map acquisition module: X images from the training set are randomly selected and input into the selected backbone network CSPDarknet for feature extraction, obtaining N effective feature maps of different scales. The feature extraction capability of the backbone is optimized through the EASE module. Then, feature fusion is performed through the lightweight AD-FPN to obtain N fused feature maps of different scales. Finally, the fused feature maps are passed through the multi-scale adaptive sensing detection head MASH to obtain N prediction feature maps.

[0063] Training module: For N predicted feature maps, the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category are predicted to generate a bounding box. During the training process, the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss are calculated based on the network output and the true GT box. The loss is calculated repeatedly until the statistical mAP is determined, and the trained network is obtained.

[0064] Prediction box acquisition module: Use the trained network to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence score, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box;

[0065] Defect localization module: maps the predicted box on the feature map scale to the original image scale based on the proportional relationship to locate fabric image defects.

[0066] The beneficial effects of the present invention are:

[0067] (1) The present invention improves the YOLOv5s algorithm, enhances the features of the redundant residual blocks in the deep layer of the backbone network, and proposes a C3EASE feature extraction module. By introducing C3EASE before SPPF performs multi-scale extraction, it can screen the semantic features of irregular defects that are more beneficial to the detection task, thereby accelerating the network and improving the detection accuracy of the network.

[0068] (2) Considering the redundant connections in the original feature fusion module PANet, this paper removes some redundant connection nodes and designs AD-FPN to improve the efficiency of network feature transmission. Unlike PANet, AD-FPN is a simplified bidirectional pyramid structure. It improves the bottom-up feature flow path based on PANet. Through the strategy of aggregate distribution guidance, it avoids feature misalignment caused by multiple feature fusions, improves the inference speed of the network architecture, and efficiently guides multi-scale feature fusion.

[0069] (3) For the prediction head of the network, the present invention uses a lightweight multi-scale adaptive perceptual detection head. This head modulates multiple parallel local contexts through a set of adaptive spatial parameters to select a receptive field that is more favorable for the final prediction at each scale. This improves the accuracy of the model in detecting minor defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 This is a flow chart of an efficient real-time detection method for fabric surface defect detection according to a preferred embodiment of the present invention;

[0072] Figure 2 The following are the EASE (Enhanced Arbitrary Shape Feature Extraction) structure diagram, MFFM (Multi-Scale Feature Fusion Module) structure diagram, and MASH (Multi-Scale Adaptive Awareness Detection Head) structure diagram proposed in the preferred embodiment of the present invention;

[0073] Figure 3 This is a schematic diagram of the network architecture of a preferred embodiment of the present invention;

[0074] Figure 4 is a loss graph of the training process of the preferred embodiment of the present invention;

[0075] Figure 5 This is a block diagram of an efficient real-time detection system for fabric surface defect detection according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0076] The following describes the embodiments of the present invention through specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0077] This embodiment provides an efficient real-time detection method for fabric surface defect detection. Figure 1 The figure shows the flow chart of the method. Figure 2 , which is the C3EASE structure diagram, refer to Figure 3, which is a network architecture diagram, processes the image according to the flowchart of the method of the present invention to illustrate in detail the effect of the method of the present invention on improving detection accuracy.

[0078] An efficient real-time detection method for fabric surface defect detection comprises the steps of:

[0079] S1. Collect a fabric image dataset. Each image corresponds to an XML annotation file, in which each defect in the image is annotated with its category and location.

[0080] S2. performing data augmentation on the fabric image dataset to expand the dataset to obtain an enhanced fabric image dataset;

[0081] S3. Divide the enhanced dataset into a training set, a validation set, and a test set according to a preset ratio;

[0082] S4. Randomly select X images from the training set as the input of the network. After the backbone network feature extraction, N effective feature maps of different scales are obtained. Then, the feature aggregation module further fuses the obtained effective feature maps to obtain N fused feature maps of different scales. Finally, the channels of the fused feature maps are adjusted to obtain N predicted feature maps.

[0083] S5: For each of the N predicted feature maps, directly predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and then generate a bounding box based on this information; calculate the network loss value based on the predicted box obtained in S5 and the GT box of the corresponding image, and use the gradient descent method to update the network parameters; repeat until all images in the training set are input into the network once. Use the network with updated parameters to predict each image in the validation set, and output the AP value of each category in the validation set after statistics; repeat until the mAP counted in step S5 stabilizes at a certain value, and the trained network is obtained;

[0084] S6. Use the trained network to predict all images in the test set and obtain the predicted boxes on the feature map.

[0085] S7. Based on the proportional relationship between the feature map and the original image (the ratio of the feature map size to the original image), the prediction box on the feature layer is mapped to the corresponding original image to locate the defect.

[0086] The following describes each step in more detail.

[0087] In step S1, the publicly available Alibaba Tianchi fabric dataset is selected. Specifically, the resulting dataset includes multiple fabric images I and corresponding label files. The input image size is 640×640 pixels. The label file is an XML file that records the object location and category information in the image.

[0088] Specifically, step S2 is to expand the fabric dataset using Mosaic data enhancement. The Mosaic data enhancement is to randomly select four images from four training sets, randomly crop the four images, and then splice them into one image as new training data.

[0089] In step S3, the division ratio of the training set, validation set, and test set is set to 0.75:0.15:0.15, and train.txt, val.txt, and test.txt files are generated to save the corresponding image lists.

[0090] Step S4 specifically includes the following steps:

[0091] S4.1. Randomly select X images from the training set and input them into the selected backbone network CSPDarknet for multi-scale feature extraction. The network outputs three effective feature layers of different scales, with scales of 20×20, 40×40, and 80×80, respectively. The proposed Darknet is composed of 2 Conv modules, 3 C3 modules, and a C3EASE module. The deepest feature map is processed by the fast spatial pyramid pooling structure (SPPF) to obtain a feature map M5.

[0092] S4.2. The three effective feature maps of different scales extracted by the backbone network are further fused through the designed AD-FPN. The output feature maps correspond to the scales of the backbone network output feature maps (80×80, 40×40, and 20×20, respectively). The designed AD-FPN extracts multi-scale features from SPPF and simplifies some of the calculation nodes to obtain an approximate bidirectional pyramid structure.

[0093] S4.3. The three effective feature maps of different scales output after the feature fusion architecture is enhanced are output by integrating the effective adaptive receptive field through the designed MASH structure.

[0094] In step S5, the feature output of the lightweight detection head adjusts the number of channels of the fusion feature map to 3*(5+num_class), and outputs N predicted feature maps, where num_classe represents the number of categories. The overall network loss is calculated based on the network output feature map, the prediction box, and the corresponding GT box, specifically: the intersection over Union (IoU) loss is calculated based on the prediction box and the corresponding GT box, and the classification confidence loss and the border confidence loss are calculated based on the classification confidence and border confidence of each prediction box contained in the network output feature map, and the intersection loss, classification confidence loss, and border confidence loss are weighted and summed in a preset ratio to obtain the overall network loss, and back propagation is performed to optimize the network parameters. Specifically, network model training is performed, which specifically includes the following steps:

[0095] A: Configure the network environment, Python version 3.9 and deep learning framework PyTorch 2.1, and use CUDA for acceleration;

[0096] B: Set the initial learning rate to 0.001, the learning rate adjustment strategy to cosine annealing decay, and the optimizer to SGD;

[0097] C: Set the number of images input to the network per batch to 16;

[0098] D: The network does not use pre-trained weights. The overall network loss is calculated after each training cycle. The training cycle continues until the mAP on the validation set stabilizes at a certain value, at which point the model network training is stopped.

[0099] In step S6, the final prediction box is obtained from all the adjusted test set prediction boxes, which specifically includes the following steps:

[0100] S6.1. Sort all adjusted prediction boxes by confidence score.

[0101] S6.2. Use non-maximum suppression (NMS) to remove redundant frames in all adjusted prediction frames to obtain the final prediction frame.

[0102] In order to verify the performance of the proposed method, the improved network is used to predict the images in the test set, and the prediction results and GT are used to calculate the mean average precision (mAP) and the accuracy (Precision) and recall (Recall) corresponding to each category. The experimental results are shown in Figure 2. Figure 4 As shown, the present invention can detect various types of fabric defects and achieve a high detection accuracy.

[0103] like Figure 5 As shown, this embodiment discloses an efficient real-time detection system for fabric surface defect detection, which is used in the above method embodiment and includes the following modules:

[0104] Data acquisition module: collects fabric surface defect data sets;

[0105] Data enhancement module: performs data enhancement on the fabric dataset;

[0106] Dataset partitioning module: divides the data set after data enhancement into training set, validation set, and test set, and saves the corresponding image name list, label position, and category information;

[0107] Prediction feature map acquisition module: X images from the training set are randomly selected and input into the selected backbone network CSPDarknet for feature extraction, obtaining N effective feature maps of different scales. The feature extraction capability of the backbone is optimized through the EASE module. Then, feature fusion is performed through the lightweight AD-FPN to obtain N fused feature maps of different scales. Finally, the fused feature maps are passed through the multi-scale adaptive sensing detection head MASH to obtain N prediction feature maps.

[0108] Training module: For N predicted feature maps, the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category are predicted to generate a bounding box. During the training process, the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss are calculated based on the network output and the true GT box. The loss is calculated repeatedly until the statistical mAP is determined, and the trained network is obtained.

[0109] Prediction box acquisition module: Use the trained network to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence score, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box;

[0110] Defect localization module: maps the predicted box on the feature map scale to the original image scale based on the proportional relationship to locate fabric image defects.

[0111] For other contents of this embodiment, please refer to the above method embodiment.

[0112] The beneficial effects of the present invention are:

[0113] (1) The present invention is based on the improvement of the YOLOv5s algorithm, which simplifies the redundant residual blocks in the deep layer of the backbone network, thereby accelerating the network and improving the detection accuracy of the network.

[0114] (2) Considering the redundant connections in the original feature fusion module PANet, AD-FPN improves the efficiency of network feature fusion and accelerates network inference. Unlike PANet, AD-FPN is an optimized approximate bidirectional pyramid structure. It aggregates and distributes the guided feature fusion strategy based on PANet. In addition, C3EASE is introduced before SPPF performs multi-scale extraction to filter irregular defect semantic features that are more beneficial to the detection task.

[0115] (3) For the prediction head of the network, a lightweight MASH structure is used to integrate local receptive fields of different sizes. A set of adaptive spatial parameters is used to select the receptive field that is more spatially favorable for the final prediction. This approach improves the accuracy of the model in detecting small defects.

[0116] In summary, the present invention is based on the improvement of the YOLOv5s network, and proposes the EASE structure to extract semantic features of arbitrary shapes and the multi-scale adaptive perception detection head MASH structure to make full use of the network fusion output to improve accuracy. At the same time, in order to improve the reasoning speed of the model, a new feature fusion network architecture AD-FPN is proposed. While improving the reasoning speed, the defect recognition accuracy of the fabric image surface is higher and the detection speed is faster.

[0117] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection of the present invention.

Claims

1. An efficient real-time detection method for fabric surface defect detection, characterized in that: The steps include: S1, collect fabric surface defect dataset; S2, perform data enhancement on the fabric dataset; S3. Divide the data set after data enhancement into training set, validation set, and test set, and save the corresponding image name list, label position and category information; S4. Randomly select X images from the training set and input them into the selected backbone network CSPDarknet for feature extraction to obtain N effective feature maps of different scales. The feature extraction capability of the backbone is optimized by the EASE structure. Then, feature fusion is performed through the lightweight AD-FPN to obtain N fused feature maps of different scales. Finally, the fused feature maps are passed through the multi-scale adaptive perception detection head MASH to obtain N predicted feature maps. The AD-FPN is a bidirectional pyramid structure obtained by extracting multi-scale features from SPPF and simplifying the calculation nodes. In the feature extraction process of this step: S4.1.

1. Replace the original Bottleneck neck feature enhancement structure of the last C3 module in the backbone network CSPDarknet with the EASE structure. The EASE structure is obtained by concatenating the AKConv arbitrary shape sampling convolution and the Siam self-similarity focusing structure; S4.1.

2. The output of the EASE structure in the C3 module is used as the input of the SPPF multi-scale feature extraction process to obtain the output feature map M5 of the feature backbone extraction network; S5. For the N predicted feature maps, predict the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and generate a bounding box; During the training process, the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss are calculated based on the network output and the real GT box. The loss is calculated repeatedly until the mAP is determined and the trained network is obtained. S6. Use the network trained in step S5 to predict the validation set and generate all candidate boxes. The candidate boxes are first sorted according to the confidence score, and then non-maximum suppression (NMS) is performed on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box. S7. Map the prediction box on the feature map scale to the original image scale according to the proportional relationship to locate the fabric image defects.

2. The efficient real-time detection method for fabric surface defects according to claim 1, characterized in that: In step S1, the public Alibaba Tianchi fabric dataset is selected. The fabric surface defect dataset contains multiple defect images I of different categories and corresponding label files. The label file contains the real-world frame labeling information of the objects in the image; the real-world frame is called GT, the object object is the fabric surface defect, and the GT frame labeling data is (c, x1, y1, x2, y2), where c represents the category of the object contained in the GT frame, x1 and y1 represent the x-coordinate and y-coordinate of the upper left corner vertex in the GT frame, respectively, and x2 and y2 represent the x-coordinate and y-coordinate of the lower right corner vertex in the GT frame, respectively.

3. The efficient real-time detection method for fabric surface defects according to claim 1 or 2, characterized in that: In step S1, the width and height of the fabric image are 2048 × 1696.

4. The efficient real-time detection method for fabric surface defects according to claim 1, characterized in that: In step S2, Mosaic data enhancement is applied to the fabric dataset. Mosaic data enhancement performs random scaling, random cropping, and random arrangement on any four images in the dataset to obtain a new image and add it to the dataset to expand the dataset.

5. The efficient real-time detection method for fabric surface defects according to claim 1, characterized in that: In step S3, the data set after data augmentation is divided into training set, validation set, and test set in a ratio of 0.75:0.15:0.15, and train.txt, val.txt, and test.txt files are generated to save the corresponding image name lists and the label positions and category information therein.

6. The efficient real-time detection method for fabric surface defects according to claim 1 or 5, characterized in that: In step S3, the generated file format is divided into rows for each image. Taking the train.txt file as an example, each row stores the path of the image divided into the training set, the position information of all the annotation boxes in the image (x1, y1, x2, y2) and the category information.

7. The efficient real-time detection method for fabric surface defects according to claim 1, characterized in that: Step S4 The steps include: S4.

1. Randomly select X images from the training set and input them into the selected backbone network CSPDarknet for multi-scale feature extraction. The output is three effective feature layers of different scales, with scales of 20×20, 40×40, and 80×80 respectively. The deepest feature map is subjected to a fast spatial pyramid pooling structure to obtain the feature map M5. S4.

2. The three effective feature maps of different scales extracted by the backbone network are fused through AD-FPN. The output feature maps have the same scale as the backbone network output feature maps, which are 80×80, 40×40, and 20×20 respectively. S4.

3. The three feature maps K3, K4, and K5 obtained use the MASH module to provide dynamic receptive fields for features of different scales, use depthwise separable convolutions with different expansion ratios to obtain different local contexts, and adjust the spatial positions of different local contexts through a set of adaptive spatial parameters. The features after multi-branch modulation are added and integrated through 1x1 convolution to obtain the predicted outputs P3, P4, and P5.

8. The efficient real-time detection method for fabric surface defects according to claim 7, characterized in that: Step S4 .2 Specifically include: S4.2.

1. Sampling the M5 feature map and the M4 feature map by 1.5 times and 0.75 times respectively, adding the results and performing a 3x3 convolution to obtain an intermediate-scale feature map 30x30. The intermediate feature map 30x30 is added to the result of performing non-integer sampling again and adaptively supplemented with feature information of different scales through a 3x3 convolution, thereby obtaining an intermediate-scale feature map K6 with richer feature information. Similarly, performing the same operation on the M4 feature map and the M3 feature map to obtain an intermediate-scale feature map 60x60, called K7, with the following formula: Refers to 1.5 times upsampling, Refers to 0.75 times downsampling; S4.2.

2. Concatenate K6, K7, and M4, and obtain the aggregated feature K4 through 1x1 convolution fusion. K4 is processed by MFFM. The formula is as follows: Concat is a concatenation operation, and MF is a multi-scale fusion operation of MFFM; S4.2.

3. Sample the K4 feature map and concatenate and fuse it with the M3 feature map. At the same time, perform feature enhancement through MFFM to obtain the low-level output feature map K3. The formula is as follows: S4.2.

4. Sample the K4 feature map and concatenate and fuse it with the M5 feature map. At the same time, perform feature enhancement through MFFM to obtain the deepest semantic output feature map K5. The formula is as follows: 。 9. The efficient real-time detection method for fabric surface defects according to claim 7, characterized in that: In step 4.3, the formula is as follows: DSConv is a depth-wise separable convolution. is a set of adaptively adjusted spatial parameters, represents element-wise multiplication, and d represents the dilation ratio.

10. An efficient real-time detection system for fabric surface defect detection, used to execute the method according to any one of claims 1 to 9, characterized in that Includes the following modules: Data acquisition module: collects fabric surface defect data sets; Data enhancement module: performs data enhancement on the fabric dataset; Dataset partitioning module: divides the data set after data enhancement into training set, validation set, and test set, and saves the corresponding image name list, label position, and category information; Prediction feature map acquisition module: X images from the training set are randomly selected and input into the selected backbone network CSPDarknet for feature extraction, obtaining N effective feature maps of different scales. The feature extraction capability of the backbone is optimized through the EASE structure. Then, feature fusion is performed through the lightweight AD-FPN to obtain N fused feature maps of different scales. Finally, the fused feature maps are passed through the multi-scale adaptive sensing detection head MASH to obtain N prediction feature maps. Training module: For N predicted feature maps, predict the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and generate a bounding box; During the training process, the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss are calculated based on the network output and the real GT box. The loss is calculated repeatedly until the mAP is determined and the trained network is obtained. Prediction box acquisition module: Use the trained network to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence score, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box; Defect localization module: maps the predicted box on the feature map scale to the original image scale based on the proportional relationship to locate fabric image defects.

Images