Efficient real-time detection method and system for fabric surface flaw detection

Abstract

The invention discloses an efficient real-time detection method and system for fabric surface flaw detection, and the method comprises the following steps: collecting a fabric surface flaw data set, carrying out the data enhancement of the data set, and dividing the data set; randomly selecting X images in the training set, inputting the X images into the selected backbone network, performing feature extraction, fusing feature maps, and obtaining N prediction feature maps by a multi-scale adaptive sensing detection head; for the N prediction feature maps, generating a bounding box; performing training to obtain a trained network; predicting the verification set to generate all candidate frames, sorting the candidate frames according to confidence scores, and then performing non-maximum suppression (NMS) on the adjusted candidate frames to remove redundant prediction frames to obtain a final prediction frame; and mapping the prediction frame on the scale of the feature map to the scale of the original map according to the proportional relation to realize the positioning of the fabric image defect. According to the method, the defect identification precision of the surface of the fabric image is higher, and the detection speed is higher.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial defect detection, and in particular relates to a method and system for detecting fabric surface defects based on an improved YOLOv5s model. Background Art

[0002] The fabric industry plays an important role in the development of China's national economy. As people's living standards improve, all walks of life have higher requirements for the quality of light industrial products. Therefore, ensuring fabric quality is crucial for both manufacturers and consumers. Fabric surface defects not only affect product prices, but also determine product competitiveness in the market and consumer satisfaction.

[0003] In modern fabric production, professional quality inspectors are usually required to check whether there are defects on the fabric surface. However, this manual inspection method has the problems of low efficiency and high subjectivity. Inspectors may misdetect or miss inspections due to fatigue, which will affect product quality and production efficiency. Therefore, automated fabric defect detection technology becomes necessary.

[0004] In recent years, deep convolutional neural networks (CNNs) have made remarkable achievements in tasks such as image classification, detection, and segmentation, and technicians in this field have conducted a lot of research. In the field of object detection, many excellent scientific research results have emerged, such as the R-CNN and YOLO series of detection models, which are still evolving. The YOLO series has attracted much attention due to its fast detection speed and lightweight characteristics, and its application fields are also more extensive.

[0005] Although the existing detection technology has made great progress, the existing technology still has problems such as complex background interference and irregular shapes of fabric defects that are difficult to identify in the task of fabric defect detection. Based on this, the present invention proposes an efficient real-time detection method and system for fabric surface defect detection. Summary of the invention

[0006] In view of the above-mentioned status of the prior art, the present invention proposes an efficient real-time detection method and system for fabric surface defect detection. The present invention introduces the EASE enhanced arbitrary shape feature extraction structure at a specific position of the network to capture the semantic features of irregularly shaped fabric defects and perform feature focusing, designs a new feature fusion structure AD-FPN and a novel multi-scale adaptive sensing detection head MASH, and solves the problems of complex background interference and irregular fabric defect shapes that are difficult to identify in fabric defect detection tasks. The use of the present invention for defect detection of fabric images has great advantages in detection accuracy and efficiency indicators compared to existing advanced detection technologies, and well meets the real-time requirements in actual scenarios.

[0007] The present invention adopts the following technical scheme:

[0008] An efficient real-time detection method for fabric surface defect detection comprises the following steps:

[0009] S1, collect fabric image dataset;

[0010] S2, perform data enhancement on the fabric image dataset;

[0011] S3, dividing the enhanced data set according to a preset ratio to obtain a training set, a validation set, and a test set, and saving the corresponding picture name list, label position, and category information;

[0012] S4. Randomly select X images in the training set as the input of the network, select the CSPDarknet backbone for feature extraction, and obtain N effective feature maps of different scales. Insert the proposed EASE module into the last feature extraction module to optimize the feature extraction capability of the backbone. Then the lightweight AD-FPN performs feature fusion to further fuse the obtained effective feature maps to obtain N fused feature maps of different scales. Finally, the fused feature maps are integrated with different local context information through the multi-scale adaptive perception detection head MASH to provide dynamic receptive fields for features of different scales to obtain N predicted feature maps.

[0013] S5. For the N predicted feature maps, predict the probability that each pixel in the image belongs to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and then generate a bounding box based on this information; calculate the network loss value, bounding box confidence loss, and intersection over Union (IOU) loss based on the obtained predicted box and the GT box of the corresponding image, and use the gradient descent method to update the network parameters; repeat the operation until all images in the training set are input into the network in batches. Predict each image in the validation set based on the network with updated parameters, and output the AP value of each category in the validation set after statistics; repeat the calculation process of step S5 until the mAP counted in step S5 stabilizes at a certain value, and a trained network is obtained;

[0014] S6, when predicting the validation set using the network trained in step S5, the network outputs the center point coordinates of the candidate box, the length and width of the box, and the category probability, generates all candidate boxes, sorts the candidate boxes according to the confidence scores, and then performs non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes to obtain the final prediction box;

[0015] S7. Use the trained network to predict all images in the test set and obtain the prediction box on the feature map. According to the proportional relationship between the feature map and the original image, the prediction box is mapped to the corresponding original image to locate the defect.

[0016] As a preferred solution, in step S1, the Ali Tianchi fabric dataset is selected. The fabric surface defect dataset contains multiple defect images I of different categories and corresponding label files. The label files contain the real frame marking information of the objects in the images; each image corresponds to a txt format annotation file, in which each defect in the image is annotated with the category and position.

[0017] As a preferred solution, in step S1, the annotation of category and position is used as a real box, called Ground Truth (GT), the object is a fabric surface defect, and the GT box labeling data is (c, x1, y1, x2, y2 y2), where c represents the category of the object contained in the GT box, x1 and y1 represent the x-coordinate and y-coordinate of the upper left corner vertex in the GT box, respectively, and x2 and y2 represent the x-coordinate and y-coordinate of the lower right corner vertex in the GT box.

[0018] As a preferred solution, in step S1, the width and height of the fabric image are 2048×1696.

[0019] As a preferred solution, in step S2, data enhancement (HSV transformation, random rotation, translation, scaling, cropping, etc.) is performed on the fabric image dataset to expand the dataset to obtain an enhanced fabric image dataset.

[0020] As a preferred solution, in step S2, the fabric image dataset is enhanced using Mosaic data enhancement combined with Mixup data enhancement with a probability of 20%. The Mosaic data enhancement is used to randomly scale, randomly crop, and randomly arrange any 4 images in the dataset to obtain new images and add them to the dataset to expand the dataset.

[0021] As a preferred solution, in step S3, the ratio of the divided training set, validation set, and test set is 0.75:0.15:0.15, and train.txt, val.txt, and test.txt files are generated to save the corresponding picture name lists and the label positions and category information therein.

[0022] As a preferred solution, in step S3, a file format is generated to divide each image by row. Taking the train.txt file as an example, each row sequentially stores the path of the image divided into the training set, the position information of all the annotation boxes in the image (x1, y1, x2, y2) and the category information.

[0023] As a preferred solution, step S4 is specifically as follows:

[0024] S4.1, replace the original Bottleneck neck feature enhancement structure of the last C3 with the EASE structure to obtain C3EASE, and use the output of C3EASE as the input of the SPPF multi-scale feature extraction module to obtain the output M5 of the feature backbone extraction network;

[0025] S4.2, the three valid feature maps obtained in S4.1 are called M3, M4, and M5 respectively according to the receptive field from small to large, and input into the improved feature fusion module AD-FPN (M5 improved by C3EASE has better feature extraction effect, and M3, M4, and M5 are used as the input of AD-FPN) for multi-scale fusion. After cross-layer feature fusion guided by aggregation distribution, the features of different layers are used to output three feature maps K3, K4, and K5;

[0026] S4.3. For the three feature maps K3, K4, and K5 obtained, the MASH module is used to provide dynamic receptive fields for features of different scales.

[0027] As a preferred solution, in step S4, the existing backbone network Darknet is mainly composed of three C3 modules, and finally includes a C3EASE module, which is to insert the EASE module into the last C3 module. The three effective feature layers output from the backbone feature extraction network are fused using lightweight AD-FPN, and then MASH is used to integrate and enhance the features of each feature layer for the output of AD-FPN. Step S4 specifically includes the following steps:

[0028] S4.1. Feature extraction: Randomly select X images from the training set and scale them, then input them into the selected backbone network CSPDarknet for multi-scale feature extraction, and output three effective feature layers of different scales, with scales of 20×20, 40×40, and 80×80 respectively. After the deepest feature map passes through the fast spatial pyramid pooling structure (SPPF), a feature map M5 is obtained.

[0029] Step S4.1 specifically includes the following steps:

[0030] S4.1.1. Replace the original Bottleneck neck feature enhancement structure of the last C3 with the EASE structure. The EASE structure is obtained by connecting the AKConv arbitrary shape sampling convolution and the Siam self-similarity focusing structure in series.

[0031] S4.1.2. The output of C3EASE is used as the input of the SPPF multi-scale feature extraction module to obtain the output M5 of the feature backbone extraction network. M5 is the feature layer with the largest receptive field.

[0032] S4.2, feature fusion: The three valid feature maps obtained in step S4.1 (referred to as M3, M4, and M5 according to the receptive field from small to large) are input into the improved feature fusion module AD-FPN for further fusion. The scales of the output feature maps correspond to those of the backbone network output feature maps, which are 80×80, 40×40, and 20×20, respectively. The cross-layer feature fusion guided by aggregation distribution makes full use of the features of different layers. For each layer of the output feature maps K3, K4, and K5 of the feature fusion module, their scales are consistent with the input feature maps M3, M4, and M5; among them, the output of the backbone feature extraction network is used as the input of AD-FPN to generate intermediate-scale feature maps with smaller scale gaps for multi-scale feature fusion enhancement, and then the aggregated features are distributed to other scales for multi-scale feature fusion. AD-FPN extracts multi-scale features from SPPF and simplifies some of the calculation nodes to obtain an approximate bidirectional pyramid structure;

[0033] As a preferred solution, step S4.2 specifically includes the following steps:

[0034] S4.2.1, the M5 feature map and the M4 feature map are sampled by 1.5 times and 0.75 times respectively, the results are added and a 3x3 convolution is performed to obtain an intermediate scale feature map 30x30, the intermediate feature map 30x30 is added again with the result of performing non-integer sampling and the feature information of different scales is adaptively supplemented by a 3x3 convolution, thereby obtaining an intermediate scale feature map K6 with richer feature information. Similarly, the M4 feature map and the M3 feature map are subjected to the same operation to obtain an intermediate scale feature map 60x60, called K7, and the formula is described as follows:

[0035] K6=Conv 3×3 (Conv 3×3 (↑ ×1.5 M5+↓ ×0.75 M4)+↓ ×0.75 M4+↑ ×1.5 M5)

[0036] K7=Conv 3×3 (Conv 3×3 (↑ ×1.5 M4+↓ ×0.75 M3)+↓ ×0.75 M3+↑ ×1.5 M4)

[0037] ↑ ×1.5 Refers to 1.5 times upsampling,↓ ×0.75 Refers to 0.7 times downsampling.

[0038] S4.2.2. K6, K7 and M4 are concatenated and fused through 1x1 convolution to obtain the aggregate feature K4. K4 passes through the MFFM structure. MFFM includes an attention mechanism based on parallel multi-scale learning and a series of convolution operations. Multi-scale fusion is used to guide feature enhancement to obtain an aggregate feature map K4 with more obvious features. The formula is described as follows:

[0039] K4=MF(Conv 1×1 (Concat(K6,,M4,K7)))

[0040] Concat refers to the concatenation operation, and MF refers to the MFFM multi-scale fusion operation.

[0041] S4.2.3, sample the K4 feature map and concatenate and fuse it with the M3 feature map, and enhance the features through the MFFM module to obtain the low-level output feature map K3. The formula is described as follows:

[0042] K3=MF(Conv 1×1 (Concat(M3, K4)))

[0043] S4.2.4, sample the K4 feature map and concatenate and fuse it with the M5 feature map, and enhance the features through the MFFM module to obtain the deepest semantic output feature map K5. The formula is described as follows:

[0044] K5=MF(Conv 1×1 (Concat(M5,K4)))

[0045] As a preferred solution, step S4.3 specifically includes the following steps:

[0046] S4.3, feature detection head: For the three feature maps K3, K4, and K5 obtained in the above steps, the MASH module is used to provide dynamic receptive fields for features of different scales. Taking the AD-FPN feature output K3 as an example, K3 integrates rich local context through the MASH multi-scale adaptive perception detection head, performs deep separable convolutions with different expansion ratios on K3, and modulates the different receptive fields captured by multiple parallel branches through a set of adaptive spatial parameters. Finally, the final output P3 is obtained through 1x1 convolution. Similarly, P4 and P5 can be obtained, and the formula is described as follows:

[0047]

[0048] DSConv is a depth-separable convolution, α, β, γ are a set of adaptively adjusted spatial parameters, represents element-wise multiplication, and d represents the dilation ratio.

[0049] After the training is completed, the output of S4.3 adjusts the number of channels to obtain the prediction information on the feature map, and the prediction information is used to enlarge the position of the prediction box on the original image according to the ratio of the feature map to the input image.

[0050] As a preferred solution, in step S4, the EASE module is inserted after the last C3 module to capture the semantics of irregular defect features.

[0051] As a preferred solution, in step S4, three feature maps are subjected to non-integer sampling to obtain intermediate scale feature layers, with scales of 20×20, 30x30, 40×40, 60x60 and 80×80 respectively. The units on the feature map are usually called "grid cells". These grids are used to represent different areas in the image space, and target detection prediction is performed in each grid.

[0052] In step S4, the EASE feature extraction module is embedded in the backbone network of CSPDarknet to improve the feature extraction capability, while the aggregation distribution guidance strategy is used in the path aggregation network to enhance the feature fusion capability of the network architecture.

[0053] As a preferred solution, step S5 is specifically as follows: for N predicted feature maps, the YOLO series detection processing method is used for direct prediction. The number of channels of the feature map output by the network is adjusted to 3*(5+num_class) through the lightweight YOLO Head, which is the number of channels required for network prediction, which includes the probability of each pixel belonging to an object, the center point position, the width and height of the bounding box of the object, and the probability of detecting the object category, wherein num_class represents the number of predicted categories. The corresponding prior frame is found based on the GT frame position, and the GT and label information are converted into a vector of 5+num_class length. Then the loss is calculated with each prior frame vector on the predicted feature map, and the network parameters are updated. The process of inputting all the pictures of the entire training set into the network once for forward propagation and reversely optimizing the network parameters is called an epoch. After each epoch, the network with updated parameters is used to predict each picture in the verification set, and the AP indicators of each category of the verification set are counted according to the prediction and GT of each picture. The network convergence can be judged by the AP value remaining unchanged or showing a downward trend for multiple consecutive rounds, otherwise the training of the next epoch will continue. For each image, the network outputs the corresponding N predicted feature maps, and then the algorithm directly predicts the probability of belonging to an object in the image, the probability of which category the object to be detected belongs to, and the bounding box information of the object.

[0054] As a preferred solution, step S5 uses the YOLO series detection processing method to directly predict the probability that each pixel in the image belongs to an object.

[0055] As a preferred solution, step S5 obtains prediction information, trains the network, and integrates the output features into the channel information required for prediction.

[0056] As a preferred solution, step S6 generates all prediction boxes for each image to be predicted based on the trained network weights, and then uses non-maximum suppression (NMS) to obtain the prediction boxes on the feature map.

[0057] As a preferred solution, in step S7, specifically, the prediction box on the feature image scale is finally mapped to the original image scale according to the proportional relationship, thereby realizing the positioning of defects on the fabric image.

[0058] The present invention also discloses an efficient real-time detection system for fabric surface defect detection, which is used to execute the above method and includes the following modules:

[0059] Data collection module: collects fabric surface defect data sets;

[0060] Data enhancement module: perform data enhancement on fabric dataset;

[0061] Dataset division module: divides the data set after data enhancement into training set, validation set, and test set, and saves the corresponding image name list, label position, and category information;

[0062] Prediction feature map acquisition module: randomly select X images from the training set and input them into the selected backbone network CSPDarknet for feature extraction to obtain N effective feature maps of different scales; optimize the feature extraction capability of the backbone through the EASE module; then perform feature fusion through the lightweight AD-FPN to obtain N fused feature maps of different scales, and finally obtain N prediction feature maps through the multi-scale adaptive perception detection head MASH from the fused feature maps;

[0063] Training module: For N predicted feature maps, predict the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and generate a bounding box; during the training process, calculate the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss based on the network output results and the real GT box, and repeatedly calculate the loss until the mAP is determined and the trained network is obtained;

[0064] Prediction box acquisition module: Use the trained network to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence score, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box;

[0065] Defect localization module: maps the prediction box on the feature map scale to the original image scale according to the proportional relationship to locate the defects in the fabric image.

[0066] The beneficial effects of the present invention are:

[0067] (1) The present invention improves the YOLOv5s algorithm, enhances the features of the redundant residual blocks in the deep layer of the backbone network, and proposes a C3EASE feature extraction module. By introducing C3EASE before SPPF performs multi-scale extraction, it can screen irregular defect semantic features that are more beneficial to the detection task, thereby accelerating the network and improving the detection accuracy of the network.

[0068] (2) Considering the redundant connections in the original feature fusion module PANet, the present invention removes some redundant connection nodes and designs AD-FPN to improve the efficiency of network feature transmission. Different from PANet, AD-FPN is a simplified bidirectional pyramid structure. It improves the bottom-up feature flow path based on PANet, avoids feature dislocation caused by multiple feature fusions through the strategy of aggregation distribution guidance, improves the reasoning speed of the network architecture, and efficiently guides multi-scale feature fusion.

[0069] (3) For the prediction head of the network, the present invention uses a lightweight multi-scale adaptive perceptual detection head, which modulates the local context of multiple branches in parallel through a set of adaptive spatial parameters to select a receptive field that is more favorable for the final prediction of each scale. The present invention improves the accuracy of the model in detecting small defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 It is a flow chart of an efficient real-time detection method for fabric surface defect detection according to a preferred embodiment of the present invention;

[0072] Figure 2 It is the structure diagram of EASE (Enhanced Arbitrary Shape Feature Extraction), the structure diagram of MFFM (Multi-Scale Feature Fusion Module), and the structure diagram of MASH (Multi-Scale Adaptive Awareness Detection Head) proposed in the preferred embodiment of the present invention;

[0073] Figure 3 It is a schematic diagram of the network architecture of a preferred embodiment of the present invention;

[0074] Figure 4 is a loss graph of the training process of the preferred embodiment of the present invention;

[0075] Figure 5 The block diagram of a highly efficient real-time detection system for fabric surface defect detection according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0076] The following describes the embodiments of the present invention through specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0077] This embodiment provides an efficient real-time detection method for fabric surface defect detection. Figure 1 The figure shows a flow chart of the method. Figure 2 , which is the C3EASE structure diagram, refer to Figure 3, which is a network architecture diagram, the image is processed according to the flowchart of the method of the present invention to illustrate in detail the effect of the method of the present invention on improving the detection accuracy.

[0078] An efficient real-time detection method for fabric surface defect detection comprises the steps of:

[0079] S1. Collect a fabric image dataset. Each image corresponds to an XML-formatted annotation file, in which each defect in the image is annotated with its category and location.

[0080] S2, performing data enhancement on the fabric image dataset to expand the dataset, so as to obtain a fabric image enhanced dataset;

[0081] S3, dividing the enhanced data set according to a preset ratio to obtain a training set, a validation set, and a test set;

[0082] S4, randomly select X images in the training set as the input of the network, and obtain N effective feature maps of different scales after feature extraction by the backbone network. Then the feature aggregation module further fuses the obtained effective feature maps to obtain N fused feature maps of different scales. Finally, adjust the channels of the fused feature maps to obtain N predicted feature maps.

[0083] S5. For N predicted feature maps, directly predict the probability that each pixel in the image belongs to the object to be detected and the bounding box information of the object, and then generate a bounding box based on this information; calculate the network loss value based on the predicted box obtained in S5 and the GT box of the corresponding image, and use the gradient descent method to update the network parameters; repeat until all images in the training set are input into the network once. Use the network with updated parameters to predict each image in the validation set, and output the AP value of each category in the validation set after statistics; repeat until the mAP counted in step S5 stabilizes at a certain value, and a trained network is obtained;

[0084] S6. Use the trained network to predict all images in the test set and obtain the predicted boxes on the feature map.

[0085] S7. According to the proportional relationship between the feature map and the original image (the ratio of the feature map size to the original image), the prediction box on the feature layer is mapped to the corresponding original image to locate the defect.

[0086] The following describes each step in more detail.

[0087] In step S1, the public Ali Tianchi fabric dataset is selected. Specifically, the obtained dataset includes multiple fabric images I and corresponding label files. The image size of the input network is 640×640 pixels. The label file is an XML format file that records the target location information and category information in the image.

[0088] Step S2 specifically includes expanding the fabric data set using Mosaic data enhancement, wherein the Mosaic data enhancement randomly selects four images from four training sets, randomly crops the four images, and then splices them into one image as new training data.

[0089] In step S3, the division ratio of the training set, validation set, and test set is set to 0.75:0.15:0.15, and train.txt, val.txt, and test.txt files are generated to save the corresponding image lists.

[0090] Step S4 specifically includes the following steps:

[0091] S4.1. Randomly select X images from the training set and input them into the selected backbone network CSPDarknet for multi-scale feature extraction. Output three effective feature layers of different scales, with scales of 20×20, 40×40, and 80×80, respectively. Among them, the proposed Darknet is composed of 2 Conv modules, 3 C3 modules and C3EASE modules. After the deepest feature map passes through the fast spatial pyramid pooling structure (SPPF), a feature map M5 is obtained.

[0092] S4.2. The three effective feature maps of different scales extracted by the backbone network are further fused through the designed AD-FPN. The output feature map has a scale corresponding to that of the backbone network output feature map (80×80, 40×40, and 20×20, respectively). The designed AD-FPN extracts multi-scale features from SPPF and simplifies some of the calculation nodes to obtain an approximate bidirectional pyramid structure.

[0093] S4.3. The three effective feature maps of different scales output after the feature fusion architecture is enhanced are output by integrating the effective adaptive receptive field through the designed MASH structure.

[0094] In step S5, the feature output of the lightweight detection head adjusts the number of channels of the fusion feature map to 3*(5+num_class), and outputs N prediction feature maps, where num_classe represents the number of categories. The overall network loss is calculated based on the network output feature map, the prediction box, and the corresponding GT box, specifically: the intersection over Union (IoU) loss is calculated based on the prediction box and the corresponding GT box, and the classification confidence loss and border confidence loss are calculated based on the classification confidence and border confidence of each prediction box contained in the network output feature map, and the intersection over Union loss, classification confidence loss, and border confidence loss are weighted and summed at a preset ratio to obtain the overall network loss, and back propagation is performed to optimize the network parameters. Specifically, network model training is performed, which specifically includes the following steps:

[0095] A: Configure the network environment, Python version 3.9 and deep learning framework PyTorch 2.1, and use CUDA for acceleration;

[0096] B: Set the initial learning rate to 0.001, the learning rate adjustment strategy to cosine annealing decay, and the optimizer to SGD;

[0097] C: Set the number of images per batch input to the network to 16;

[0098] D: The network does not use pre-trained weights. The overall network loss is calculated after each cycle of training. The loop is iterated until the mAP indicator of the validation set stabilizes at a certain value, at which point the model network training is stopped.

[0099] In step S6, the final prediction box is obtained from all adjusted test set prediction boxes, which specifically includes the following steps:

[0100] S6.1. Sort all adjusted prediction boxes according to their confidence scores;

[0101] S6.2. Use non-maximum suppression (NMS) to remove redundant boxes in all adjusted prediction boxes to obtain the final prediction box.

[0102] In order to verify the performance of the proposed method, the improved network is used to predict the images in the test set, and the prediction results and GT are used to calculate the mean average precision (mAP) and the accuracy (Precision) and recall (Recall) corresponding to each category. The experimental results are shown in Figure 2. Figure 4 As shown, the present invention can realize the detection of various types of fabric defects and achieve a high detection accuracy.

[0103] like Figure 5 As shown, this embodiment discloses an efficient real-time detection system for fabric surface defect detection, which is used in the above method embodiment and includes the following modules:

[0104] Data collection module: collects fabric surface defect data sets;

[0105] Data enhancement module: perform data enhancement on fabric dataset;

[0106] Dataset division module: divides the data set after data enhancement into training set, validation set, and test set, and saves the corresponding image name list, label position, and category information;

[0107] Prediction feature map acquisition module: randomly select X images from the training set and input them into the selected backbone network CSPDarknet for feature extraction to obtain N effective feature maps of different scales; optimize the feature extraction capability of the backbone through the EASE module; then perform feature fusion through the lightweight AD-FPN to obtain N fused feature maps of different scales, and finally obtain N prediction feature maps through the multi-scale adaptive perception detection head MASH from the fused feature maps;

[0108] Training module: For N predicted feature maps, predict the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and generate a bounding box; during the training process, calculate the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss based on the network output results and the real GT box, and repeatedly calculate the loss until the mAP is determined and the trained network is obtained;

[0109] Prediction box acquisition module: Use the trained network to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence score, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box;

[0110] Defect localization module: maps the prediction box on the feature map scale to the original image scale according to the proportional relationship to locate the defects in the fabric image.

[0111] For other contents of this embodiment, please refer to the above method embodiment.

[0112] The beneficial effects of the present invention are:

[0113] (1) The present invention is based on the improvement of the YOLOv5s algorithm, which simplifies the redundant residual blocks existing in the deep layer of the backbone network, thereby accelerating the network and improving the detection accuracy of the network.

[0114] (2) Considering the redundant connections in the original feature fusion module PANet, AD-FPN improves the efficiency of network feature fusion and accelerates the network reasoning speed. Unlike PANet, AD-FPN is an optimized approximate bidirectional pyramid structure. It aggregates and distributes the guided feature fusion strategy based on PANet. In addition, C3EASE is introduced before SPPF performs multi-scale extraction to screen irregular defect semantic features that are more beneficial to the detection task.

[0115] (3) For the prediction head of the network, a lightweight MASH structure is used to integrate local receptive fields of different sizes, and a set of adaptive spatial parameters is used to select the receptive field that is more favorable to the final prediction in space. In this way, the model can improve the accuracy of small defect detection.

[0116] In summary, the present invention is improved based on the YOLOv5s network, and proposes an EASE structure to extract semantic features of arbitrary shapes and a multi-scale adaptive perception detection head MASH structure to make full use of the network fusion output to improve accuracy. At the same time, a new feature fusion network architecture AD-FPN is proposed for the reasoning speed of the model. While improving the reasoning speed, the defect recognition accuracy of the fabric image surface is higher and the detection speed is faster.

[0117] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope of the present invention.

Claims

1. An efficient real-time detection method for fabric surface defect detection, characterized in that The steps include: S1, collect fabric surface defect data set; S2, perform data enhancement on the fabric dataset; S3, divide the data set after data enhancement into training set, validation set, and test set, and save the corresponding picture name list and label position and category information; S4, randomly select X images from the training set and input them into the selected backbone network CSPDarknet for feature extraction to obtain N effective feature maps of different scales; optimize the feature extraction capability of the backbone through the EASE module; then perform feature fusion through the lightweight AD-FPN to obtain N fused feature maps of different scales, and finally obtain N predicted feature maps through the multi-scale adaptive perception detection head MASH; S5. For the N predicted feature maps, predict the probability that each pixel in the image belongs to an object, the center point position, the width and height of the bounding box of the object, and the probability of detecting the object category, and generate a bounding box; During the training process, the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss are calculated based on the network output results and the real GT box. The loss is calculated repeatedly until the mAP is determined and the trained network is obtained. S6, using the network trained in step S5 to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence scores, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes, and obtain the final prediction box; S7. Map the prediction box on the feature map scale to the original image scale according to the proportional relationship to locate the fabric image defects.

2. The efficient real-time detection method for fabric surface defect detection according to claim 1, characterized in that: In step S1, the public Alibaba Tianchi fabric dataset is selected. The fabric surface defect dataset contains multiple defect images I of different categories and corresponding label files. The label file contains the real frame labeling information of the object in the image; the real frame is called GT, the object object is the fabric surface defect, and the GT frame labeling data is (c, x1, y1, x2, y2 y2), where c represents the category of the object contained in the GT frame, x1, y1 represent the x coordinate and y coordinate of the upper left corner vertex in the GT frame, respectively, and x2, y2 represent the x coordinate and y coordinate of the lower right corner vertex in the GT frame.

3. An efficient real-time detection method for fabric surface defect detection as claimed in claim 1 or 2, characterized in that: In step S1, the width and height of the fabric image are 2048×1696.

4. The efficient real-time detection method for fabric surface defect detection according to claim 1, characterized in that: In step S2, Mosaic data enhancement is used on the fabric dataset. The Mosaic data enhancement performs random scaling, random cropping, and random arrangement on any four images in the dataset to obtain a new image and add it to the dataset to expand the dataset.

5. The efficient real-time detection method for fabric surface defect detection according to claim 1, characterized in that: In step S3, the data set after data enhancement is divided into a training set, a validation set, and a test set in a ratio of 0.75:0.15:0.15, and train.txt, val.txt, and test.txt files are generated to save the corresponding image name lists and the label positions and category information therein.

6. An efficient real-time detection method for fabric surface defect detection as claimed in claim 1 or 5, characterized in that: In step S3, the generated file format is divided into rows for each image. Taking the train.txt file as an example, each row sequentially stores the path of the image divided into the training set, the position information of all the annotation boxes in the image (x1, y1, x2, y2) and the category information.

7. The efficient real-time detection method for fabric surface defect detection according to claim 1, characterized in that: Step S4 The steps include: S4.

1. Randomly select X images from the training set and input them into the selected backbone network CSPDarknet for multi-scale feature extraction. Output three effective feature layers of different scales, with scales of 20×20, 40×40, and 80×80 respectively. The deepest feature map is subjected to the fast spatial pyramid pooling structure to obtain the feature map M5. S4.2, the three effective feature maps of different scales extracted by the backbone network are fused through AD-FPN, and the scales of the output feature maps correspond to those of the backbone network output feature maps, which are 80×80, 40×40, and 20×20 respectively; S4.

3. The three feature maps K3, K4, and K5 obtained use the MASH module to provide dynamic receptive fields for features of different scales, use depthwise separable convolutions with different expansion ratios to obtain different local contexts, adjust the spatial positions of different local contexts through a set of adaptive spatial parameters, add the features after multi-branch modulation, and integrate them through 1x1 convolution to obtain the predicted outputs P3, P4, and P5.

8. The efficient real-time detection method for fabric surface defect detection according to claim 7, characterized in that: Step S4 .2 Specifically include: S4.2.

1. The M5 feature map and the M4 feature map are sampled by 1.5 times and 0.75 times respectively, and the results are added and passed through a 3x3 convolution to obtain an intermediate-scale feature map 30x30. The intermediate feature map 30x30 is added again with the result of performing non-integer sampling and adaptively supplemented with feature information of different scales through a 3x3 convolution, thereby obtaining an intermediate-scale feature map K6 with richer feature information; similarly, the M4 feature map and the M3 feature map are subjected to the same operation to obtain an intermediate-scale feature map 60x60, called K7, and the formula is as follows: K6=Conv 3×3 (Conv 3×3 (↑ ×1.5 M5+↓ ×0.75 M4)+↓ ×0.75 M4+↑ ×1.5 M5) K7=Conv 3×3 (Conv 3×3 (↑ ×1.5 M4+↓ ×0.75 M3)+↓ ×0.75 M3+↑ ×1.5 M4) ↑ ×1.5 Refers to 1.5 times upsampling,↓ ×0.75 Refers to 0.7 times downsampling; S4.2.

2. K6, K7 and M4 are concatenated and fused through 1x1 convolution to obtain the aggregate feature K4. K4 is processed through MFFM. The formula is as follows: K4=MF(Conv 1×1 (Concat(K6,,M4,K7))) Concat is a concatenation operation, and MF is a MFFM multi-scale fusion operation; S4.2.3, sample the K4 feature map and concatenate and fuse it with the M3 feature map, and enhance the feature through MFFM to obtain the low-level output feature map K3. The formula is as follows: K3=MF(Conv 1×1 (Concat(M3,K4))) S4.2.4, sample the K4 feature map and concatenate and fuse it with the M5 feature map, and perform feature enhancement through MFFM to obtain the deepest semantic output feature map K5. The formula is as follows: K5=MF(Conv 1×1 (Concat(M5,K4)))。 9. The efficient real-time detection method for fabric surface defect detection according to claim 7, characterized in that: In step 4.3, the formula is as follows: DSConv is a depth-separable convolution, α, β, γ are a set of adaptively adjusted spatial parameters, represents element-wise multiplication, and d represents the dilation ratio.

10. An efficient real-time detection system for fabric surface defect detection, used to execute the method according to any one of claims 1 to 9, characterized in that Includes the following modules: Data collection module: collects fabric surface defect data sets; Data enhancement module: perform data enhancement on fabric dataset; Dataset division module: divides the data set after data enhancement into training set, validation set, and test set, and saves the corresponding image name list, label position, and category information; Prediction feature map acquisition module: randomly select X images from the training set and input them into the selected backbone network CSPDarknet for feature extraction to obtain N effective feature maps of different scales; optimize the feature extraction capability of the backbone through the EASE module; then perform feature fusion through the lightweight AD-FPN to obtain N fused feature maps of different scales, and finally obtain N prediction feature maps through the multi-scale adaptive perception detection head MASH from the fused feature maps; Training module: For N predicted feature maps, predict the probability of each pixel in the image belonging to an object, the center point position, the width and height of the object's bounding box, and the probability of detecting the object category, and generate a bounding box; During the training process, the classification confidence loss, bounding box confidence loss, and intersection-over-union (IOU) loss are calculated based on the network output results and the real GT box. The loss is calculated repeatedly until the mAP is determined and the trained network is obtained. Prediction box acquisition module: Use the trained network to predict the validation set, generate all candidate boxes, sort the candidate boxes according to the confidence score, and then perform non-maximum suppression (NMS) on the adjusted candidate boxes to remove redundant prediction boxes and obtain the final prediction box; Defect localization module: maps the prediction box on the feature map scale to the original image scale according to the proportional relationship to locate the defects in the fabric image.

Images