A fabric defect detection model and a training method and a fabric defect detection method

Abstract

The present application belongs to the technical field of fabric defect detection, and particularly relates to a fabric defect detection model and a training method, and a fabric defect detection method. The fabric defect detection model comprises a backbone network, a conflict filtering residual shrinkage module for performing noise reduction processing on a feature map, an efficient feature pyramid network and a prediction head. By introducing the noise reduction conflict filtering residual shrinkage module at a specific position of the network, the problems of false detection and missed detection caused by complex background interference and noise in the fabric defect detection task are reduced. Compared with the existing advanced detection model, the present application has great advantages in detection accuracy and efficiency indicators, and also meets the real-time requirements in the industrial scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of fabric defect detection, and particularly relates to a fabric defect detection model and a training method and a fabric defect detection method. BACKGROUND

[0002] The textile industry has always played an important role in the development of China's national economy. In the production of the textile industry, fabric not only serves as a basic material for making clothes, but also can be used as a raw material for other decorative and industrial products for further processing. With the development of economy and the improvement of people's living standards, the market's demand for product quality is gradually increasing, and professional quality inspection personnel are needed to check the defects of textiles and remove unqualified fabric. The manual detection method is not only low in efficiency and high in cost, but also affected by the subjective factors of the detection personnel, resulting in false detection or missed detection. Therefore, it is necessary to design an efficient automatic fabric defect detection method.

[0003] In recent years, convolutional neural networks (CNNs) have played an increasingly important role in image classification, detection and segmentation tasks, and a large number of researchers have participated in this field and improved various network models. In the direction of target detection, a large number of excellent research works have emerged, such as the R-CNN and YOLO series of detection models, among which the YOLO series has the advantages of fast detection speed and light model weight, and has stood out from many models and occupied a place. However, there are problems such as noise interference and slow detection speed in the current fabric defect detection task, and the existing advanced detection models still need to be improved in terms of detection accuracy and efficiency. SUMMARY

[0004] The purpose of the present application is to provide a fabric defect detection model and a training method and a fabric defect detection method, which introduces a noise reduction conflict filtering residual shrinkage module at a specific position of the network, reduces the false detection and missed detection problems caused by complex background interference and noise in the fabric defect detection task, and has greater advantages in detection accuracy and efficiency compared with the existing advanced detection models, and also meets the real-time requirements in the industrial scene.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] The first aspect of the embodiment of the present application provides a fabric defect detection model, which comprises a backbone network, a conflict filtering residual shrinkage module for noise reduction processing of a feature map, an efficient feature pyramid network and a prediction head.

[0007] The backbone network comprises an input layer, a concentration layer, a first convolutional bottleneck layer, a second convolutional bottleneck layer, a third convolutional bottleneck layer, a fourth convolutional bottleneck layer and a spatial pyramid fast pooling module connected in sequence.

[0008] The high-efficiency feature pyramid network comprises a first convolutional block, a first fusion bottleneck layer, a first fusion convolutional layer, a second fusion bottleneck layer, a second convolutional block, a second fusion convolutional layer, a third fusion bottleneck layer, a third convolutional block, a fourth fusion bottleneck layer, a fourth convolutional block and a fifth fusion bottleneck layer connected in sequence, the first fusion convolutional layer and the fourth fusion bottleneck layer are connected, and the first convolutional block and the fifth fusion bottleneck layer are connected;

[0009] The first convolutional bottleneck layer is connected with the second fusion convolutional layer, the second convolutional bottleneck layer is connected with the conflict filtering residual shrinkage module and the second fusion bottleneck layer respectively, the third convolutional bottleneck layer is connected with the conflict filtering residual shrinkage module and the first fusion bottleneck layer respectively, the spatial pyramid fast pooling module is connected with the first convolutional block, and the first convolutional block is connected with the first fusion convolutional layer after up-sampling;

[0010] The conflict filtering residual shrinkage module is connected with the third fusion bottleneck layer and the fourth fusion bottleneck layer respectively;

[0011] The third fusion bottleneck layer, the fourth fusion bottleneck layer and the fifth fusion bottleneck layer are connected with three prediction heads respectively.

[0012] As a preferred scheme, the conflict filtering residual shrinkage module comprises an input layer, a point-wise convolutional block with an activation function, a convolutional block without an activation function, a deep convolutional block without an activation function, a first picture addition processing layer, an SE module, a soft threshold function layer, a second picture addition processing layer and a second rectified linear unit activation function;

[0013] The SE module comprises a global average pooling layer, a first full connection layer, a first rectified linear unit activation function, a second full connection layer, a first activation function layer and a picture multiplication processing layer connected in sequence, and the global average pooling layer is connected with the picture multiplication processing layer;

[0014] The input layer is connected with the convolutional block without an activation function and the point-wise convolutional block with an activation function respectively, the point-wise convolutional block with an activation function is connected with the deep convolutional block without an activation function and the second picture addition processing layer respectively, the convolutional block without an activation function and the deep convolutional block without an activation function are both connected with the first picture addition processing layer, the first picture addition processing layer is connected with the global average pooling layer and the soft threshold function layer of the SE module respectively, and the picture multiplication processing layer, the soft threshold function layer, the second picture addition processing layer and the second rectified linear unit activation function of the SE module are connected in sequence.

[0015] The second aspect of the embodiment of the application provides a fabric defect detection model training method, comprising the following steps:

[0016] S1, collect a plurality of fabric images with defects, label fabric defect data and perform data enhancement processing on each image to construct a fabric image dataset;

[0017] S2, using a clustering algorithm to cluster the target GT box of all images in the fabric image dataset to obtain K prior boxes;

[0018] S3, divide the fabric image dataset into a training set and a validation set according to a predetermined ratio;

[0019] S4, randomly select X images in the training set and input them into the fabric defect detection model according to any one of the first aspect of the embodiments of the present application for training, and obtain N training prediction feature maps for each fabric image;

[0020] S5, divide the K prior boxes into N groups in scale order, and according to the scale size of the N groups of prior boxes and the scale size of the N training prediction feature maps, assign the N groups of prior boxes to the N training prediction feature maps respectively, and adjust the corresponding prior boxes according to the anchor point information on the training prediction feature maps to obtain all training prediction boxes;

[0021] S6, calculate the overall loss of the network according to the training prediction boxes and the corresponding GT boxes, and use gradient descent method to update the parameters of the model;

[0022] S7, repeat steps S4-S6 until all images in the training set are input into the model once, and use the model with updated parameters to predict each image in the validation set, calculate the precision value AP of each fabric defect class in the validation set according to the validation prediction box and the GT box of each image, and calculate the average precision value mAP of all precision values AP;

[0023] S8, repeat step S7 until the average precision value mAP calculated multiple times is equal to a fixed value, and obtain the trained model.

[0024] As a preferred scheme, step S4 comprises:

[0025] S4.1, randomly select X images in the training set, input them into the backbone network after slice processing for hierarchical feature extraction, and take out four effective feature maps containing different scales and channel numbers, and name them as M5 feature map, M4 feature map, M3 feature map and M2 feature map in order of scale from small to large;

[0026] S4.2, input the M4 feature map and the M3 feature map into a conflict filtering residual shrinkage module for processing to obtain a K4 feature map and a K3 feature map, and input the M5 feature map, the M4 feature map, the M3 feature map, the M2 feature map, the K4 feature map and the K3 feature map into an efficient feature pyramid network for further feature fusion to obtain a P5 fusion feature map, a P4 fusion feature map and a P3 fusion feature map;

[0027] S4.3, adjust the channels of the P5 fusion feature map, the P4 fusion feature map and the P3 fusion feature map obtained in step S4.2 through a prediction head to obtain three training prediction feature maps.

[0028] As a preferred solution, in step S4.2, the M4 feature map and the M3 feature map are input into the conflict filtering residual shrinkage module for processing, including the steps of:

[0029] S4.2.1, input the M4 feature map into a point-wise convolution block with an activation function and a convolution block without an activation function for processing respectively;

[0030] S4.2.2, add the output of the point-wise convolution block with an activation function after being processed by a depth convolution block without an activation function to the output of the convolution block without an activation function to obtain an added feature map;

[0031] S4.2.3, after all values on the added feature map are processed by an absolute value, input the processed added feature map into an SE module for processing to obtain a processed feature map and a threshold value of the feature map;

[0032] S4.2.4, jointly perform soft threshold denoising processing on all values on the processed feature map output by the SE module and the added feature map obtained in step S4.2.2 to obtain a soft threshold processing result;

[0033] S4.2.5, add the soft threshold processing result and the output of the point-wise convolution block with an activation function, and output after being processed by a second rectified linear unit activation function to obtain the K4 feature map;

[0034] The M3 feature map is input into the conflict filtering residual shrinkage module for the same processing as the M4 feature map to obtain the K3 feature map.

[0035] As a preferred solution, the function used in the soft threshold denoising processing is:

[0036]

[0037] Wherein, x represents all values on the processed feature map output by the SE module and the added feature map obtained in step S4.2.2, y represents the output value of x after soft thresholding processing, and τ represents the threshold value of the feature map obtained after SE module processing.

[0038] As a preferred solution, in step S4.2.3, the addition result after absolute value processing is input into the SE module for processing, and the SE module processing steps include:

[0039] The absolute value processed added feature map is sequentially processed by a global average pooling layer, a first full connection layer, a first rectified linear unit activation function, a second full connection layer, and a first activation function layer in the SE module;

[0040] The feature map output by the global average pooling layer is multiplied by the feature map output by the first activation function layer to obtain a multiplied feature map, which is the processed feature map output by the SE module.

[0041] As a preferred solution, in step S4.2, the M5 feature map, the M4 feature map, the M3 feature map, the M2 feature map, the K4 feature map, and the K3 feature map are all input into the efficient feature pyramid network for further feature fusion to obtain the P5 fusion feature map, the P4 fusion feature map, and the P3 fusion feature map, including steps:

[0042] S4.2.6, the M5 feature map is processed by a first convolutional block to obtain a C5 feature map, the C5 feature map is upsampled and input together with the M4 feature map into a first fusion bottleneck layer for processing, the output of the first fusion bottleneck layer processing and the output of the upsampling processing are input together into a first fusion convolutional layer for processing to obtain a C4 feature map;

[0043] S4.2.7, the C4 feature map is upsampled and input together with the M3 feature map into a second fusion bottleneck layer for processing to obtain a C3 feature map;

[0044] S4.2.8, the C3 feature map is processed by a second convolutional block, and after upsampling processing, is input together with the M2 feature map into a second fusion convolutional layer for processing to obtain a C2 feature map;

[0045] S4.2.9, the C2 feature map, the C3 feature map, and the K3 feature map are input together into a third fusion bottleneck layer for processing to obtain a P3 fusion feature map;

[0046] S4.2.10, the P3 feature map is processed by a third convolutional block and input together with the C4 feature map and the K4 feature map into a fourth fusion bottleneck layer for processing to obtain a P4 fusion feature map;

[0047] S4.2.11, the P4 feature map after being processed by the fourth convolution block and the C5 feature map are jointly input into a fifth fusion bottleneck layer for processing to obtain a P5 fusion feature map.

[0048] As a preferred solution, step S6 comprises:

[0049] S6.1, calculating a bounding box position loss, a classification confidence loss and a bounding box confidence loss according to the training prediction box and the corresponding GT box;

[0050] S6.2, weighting and summing the bounding box position loss, the classification confidence loss and the bounding box confidence loss to obtain a network overall loss, and performing back propagation of the overall loss value to the fabric defect detection model and updating the parameters of the fabric defect detection model using the gradient descent method.

[0051] The third aspect of the embodiment of the present application provides a fabric defect detection method, comprising the following steps:

[0052] A, obtaining a plurality of fabric images with defects to be detected, using the fabric defect detection model obtained by the model training method of any one of the second aspect of the embodiment of the present application to predict the fabric images to be detected, and outputting N detection prediction feature maps;

[0053] B, adjusting the prior box according to the adjustment vector corresponding to each anchor point on each detection prediction feature map to obtain a preliminary detection prediction box of each image;

[0054] C, using non-maximum suppression to remove redundant boxes in the preliminary detection prediction box to obtain a detection prediction box on the detection prediction feature map;

[0055] D, mapping the detection prediction box on the detection prediction feature map scale to the original image scale according to the proportional relationship to complete the detection and positioning of the fabric defects.

[0056] The beneficial effects of the present application are:

[0057] The fabric defect detection model provided by the present application introduces a jump mapping of a deep residual shrinkage module on the basis of a YOLOv5 network, and improves a threshold function and a convolution layer module on the basis of the deep residual shrinkage module to obtain a conflict filtering residual shrinkage module for denoising processing of a feature map. Meanwhile, a convolution side branch is added to the conflict filtering residual shrinkage module, and a construction method of a Ghost Module is used for reference, that is, Depthwise convolution and Pointwise convolution are used to efficiently learn features, and are connected in an additive manner, which can more efficiently use computing resources to generate feature maps and can well reduce costs.

[0058] The fabric defect detection model provided by the application also improves the feature pyramid network on the basis of the YOLOv5 network, designs an efficient feature pyramid network of increasing fine-grained feature extraction branches and receptive field branches of small and medium targets, and in the efficient pyramid feature network, not only increases the fine-grained feature extraction branch in the top-down path, and fuses the shallow position and texture information upward, but also adds the receptive field branch of the small and medium target through the skip connection, so that the small and medium target can be distinguished from the complex background. Meanwhile, the connection mode between the main network of the model, the conflict filtering residual shrinkage module for denoising the feature map and the efficient feature pyramid network is improved, and the way of the model extracting and predicting the feature map is improved.

[0059] The fabric defect detection model provided by the application introduces a deep residual shrinkage module on the basis of the YOLOv5 network, and improves the way of the model extracting and predicting the feature map, which can effectively reduce the influence of small defects and noise on the detection of medium and large size defects in the fabric image, can well fit the original structure of the YOLOv5 network, and can fuse more features without bringing noise, so that the denoised feature map can not only provide important information, but also reduce the introduction of noise as much as possible, improve the precision and speed of the network, and strengthen the detection ability of the fabric defect, while speeding up the detection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0061] Figure 1 It is a network structure schematic diagram of a fabric defect detection model.

[0062] Figure 2 It is a structure schematic diagram of a noise reduction conflict filtering residual shrinkage module.

[0063] Figure 3 It is a correct schematic diagram of a fabric defect detection anchor frame.

[0064] Figure 4 It is a fabric defect prediction frame schematic diagram after adding an initial RSBU module in the YOLOv5 network.

[0065] Figure 5 It is a fabric defect prediction frame schematic diagram after adding an RSBU module using a thresholding function in the YOLOv5 network.

[0066] Figure 6is a fabric defect prediction frame schematic diagram after adding a CF-RSBU module in the YOLOv5 network.

[0067] Figure 7 is a fabric defect detection heat comparison diagram after adding a residual shrinkage module and a conflict filtering residual shrinkage module in the YOLOv5 network, respectively.

[0068] Figure 8 is a structure comparison diagram of PAN, BiFPN and CF-RSBU-YOLOv5.

[0069] Figure 9 is a detection effect diagram of a fabric image sample. DETAILED DESCRIPTION

[0070] The embodiments of the present application will be described in detail below with specific specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied by different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0071] Example 1

[0072] Referring to Figure 1 , the embodiment provides a fabric defect detection model, which comprises a backbone network Backbone, a conflict filtering residual shrinkage module CF-RSBU for denoising feature maps, an efficient feature pyramid network EFPN and a prediction head Head;

[0073] The backbone network comprises an input layer Inputs, a focus layer Focus, a first convolutional bottleneck layer CBL+CSP_3(160, 160, 64), a second convolutional bottleneck layer CBL+CSP_6(80, 80, 128), a third convolutional bottleneck layer CBL+CSP_9(40, 40, 256), a fourth convolutional bottleneck layer CBL+CSP_3(20, 20, 512) and a spatial pyramid fast pooling module SPPF connected in sequence.

[0074] The high-efficiency feature pyramid network comprises a first convolution block CBL, a first fusion bottleneck layer Concat+CSP_3, a first fusion convolution layer Concat+CBL, a second fusion bottleneck layer Concat+CSP_3, a second convolution block CBL, a second fusion convolution layer Concat+CBL, a third fusion bottleneck layer Concat+CSP_3, a third convolution block CBL, a fourth fusion bottleneck layer Concat+CSP_3, a fourth convolution block CBL and a fifth fusion bottleneck layer Concat+CSP_3 connected in sequence, the first fusion convolution layer Concat+CBL and the fourth fusion bottleneck layer Concat+CSP_3 are connected, the first convolution block CBL and the fifth fusion bottleneck layer Concat+CSP_3 are connected;

[0075] The first convolution bottleneck layer CBL+CSP_3 (160, 160, 64) is connected with the second fusion convolution layer Concat+CBL, the second convolution bottleneck layer CBL+CSP_6 (80, 80, 128) is respectively connected with the conflict filtering residual shrinkage module CF-RSBU and the second fusion bottleneck layer Concat+CSP_3, the third convolution bottleneck layer CBL+CSP_9 (40, 40, 256) is respectively connected with the conflict filtering residual shrinkage module CF-RSBU and the first fusion bottleneck layer Concat+CSP_3, the spatial pyramid fast pooling module SPPF is connected with the first convolution block CBL, and the first convolution block CBL is connected with the first fusion convolution layer Concat+CBL after being up-sampled;

[0076] The conflict filtering residual shrinkage module CF-RSBU is respectively connected with the third fusion bottleneck layer Concat+CSP_3 and the fourth fusion bottleneck layer Concat+CSP_3.

[0077] The third fusion bottleneck layer Concat+CSP_3, the fourth fusion bottleneck layer Concat+CSP_3 and the fifth fusion bottleneck layer Concat+CSP_3 are respectively connected with three prediction heads YOLOHead.

[0078] Specifically:

[0079] Each layer of the convolution bottleneck layer in the backbone network Backbone comprises a first convolution block CBL and a bottleneck structure module BottleneckCSP connected in sequence, and the number of bottleneck modules Bottleneck in the bottleneck structure modules BottleneckCSP of the four layers of convolution bottleneck layers is 3, 6, 9 and 3 in turn.

[0080] The first fusion bottleneck layer, the second fusion bottleneck layer, the third fusion bottleneck layer, the fourth fusion bottleneck layer and the fifth fusion bottleneck layer in the high-efficiency feature pyramid network each comprise a fusion function Concat and a bottleneck structure module BottleneckCSP connected in sequence, and the number of bottleneck modules Bottleneck in the bottleneck structure module BottleneckCSP of each fusion bottleneck layer is 3;

[0081] The first fusion convolution layer and the second fusion convolution layer in the high-efficiency feature pyramid network each comprise a fusion function Concat and a convolution block CBL connected in sequence.

[0082] Further, referring to Figure 2 , the conflict filtering residual shrinking module CF-RSBU comprises an input layer Input, a 1×1 pointwise convolution block PW-CBL with an activation function SiLU, a 3×3 convolution block CBL without an activation function SiLU, a 3×3 depth convolution block DW-CBL without an activation function SiLU, a first picture addition processing layer, an SE module, a soft thresholding function layer, a second picture addition processing layer and a second rectified linear unit activation function ReLU;

[0083] The SE module comprises a global average pooling layer, a first full connection layer FC, a first rectified linear unit activation function ReLU, a second full connection layer FC, a first activation function layer Sigmoid, a picture multiplication processing layer connected in sequence, and the global average pooling layer and the picture multiplication processing layer are connected;

[0084] The input layer Input is connected with the 3×3 convolution block CBL without an activation function SiLU and the 1×1 pointwise convolution block PW-CBL with an activation function SiLU, respectively, the 1×1 pointwise convolution block PW-CBL with an activation function SiLU is connected with the 3×3 depth convolution block DW-CBL without an activation function SiLU and the second picture addition processing layer, respectively, the 3×3 convolution block CBL without an activation function SiLU and the 3×3 depth convolution block DW-CBL without an activation function SiLU are both connected with the first picture addition processing layer, the first picture addition processing layer is connected with the global average pooling layer of the SE module and the soft thresholding function layer, respectively, and the picture multiplication processing layer of the SE module, the soft thresholding function layer, the second picture addition processing layer and the second rectified linear unit activation function ReLU are connected in sequence.

[0085] Ghost Module borrows depthwise separable convolution to generate feature maps at the cost of fewer convolution kernels, that is, it is more efficient in terms of computational resources for generating feature maps, and can well reduce costs. Inspired by this idea, to mine potential effective features, we added a convolution side branch to the conflict filtering residual shrinkage module CF-RSBU, and borrowed the construction method of Ghost Module, that is, to efficiently learn features by using Depthwise convolution and Pointwise convolution, and to map and connect in an additive manner.

[0086] On the basis of RSBU, the thresholding function and convolution layer module are improved. To further demonstrate the effectiveness of the improvement points, we use the final output prediction map and Gradient-weighted Class Activation Mapping (Grad-CAM) to visualize the decision of the output layer.

[0087] The prediction map is generated according to the best.pt weight file under different structure design, and the final detected defect frame class and coordinates are obtained, and the prediction frame is drawn on the image to be predicted with defects. Different shades of defect frames in the figure represent different defect classes. It can be seen that Figure 4 In the middle, too many prediction frames will be output around small targets, such as dark frames representing dirty defects, which will be repeatedly detected and output. In Figure 5 After the thresholding function is improved, this situation will be reduced, and defect prediction frames that have not been detected before will be output. After further improving the convolution layer, Figure 6 The prediction frame obtained is more similar to the correct diagram Figure 3 Most defect targets are detected, and can be accurately positioned.

[0088] Figure 7 (a) is the fabric detection heat map after adding residual shrinkage module in YOLOv5 network, Figure 7 (b) is the fabric detection heat map after adding conflict filtering residual shrinkage module in YOLOv5 network. As can be seen from the figure, after adding residual conflict filtering residual shrinkage module in YOLOv5 network, the influence of small defects and noise on the detection of medium and large size defects in fabric image is reduced, which can well fit the original structure of YOLOv5 network, and fuse more features without introducing noise. The noise-reduced feature map not only provides important information, but also tries to reduce the introduction of noise, which can effectively improve the precision and speed of fabric defect detection network and enhance the detection ability of fabric defects.

[0089] As Figure 8As shown, the feature pyramid network (FPN) provides rich semantic information for the fine-grained information of the shallow layer feature map, greatly improving the detection efficiency of small targets. FPN uses a top-down path to obtain a larger receptive field for small targets, while the path aggregation network (PANet) reversely adds a bottom-up path to strengthen the fusion of features of different granularities. The weighted bidirectional feature pyramid network (BiFPN) further weights the same layer features of the intermediate layer and removes the nodes with only a single input edge in the top and bottom layers to construct a simplified high-level fusion network. The CF-RSBU-YOLOv5 network model designed in the application not only increases the fine-grained feature extraction branch in the top-down path to fuse the rich position and texture information of the shallow layer upwards, but also uses a skip connection to add a receptive field branch for small targets to distinguish them from complex backgrounds.

[0090] Embodiment two:

[0091] The embodiment provides a fabric defect detection model training method, including the following steps:

[0092] S1, a plurality of fabric images with defects are collected, and each image is subjected to fabric defect data labeling and data enhancement processing to construct a fabric image dataset;

[0093] S2, a clustering algorithm is used to cluster the target GT boxes of all images in the fabric image dataset to obtain K prior boxes;

[0094] S3, the fabric image dataset is divided into a training set and a validation set according to a predetermined proportion;

[0095] S4, X images in the training set are randomly selected and input into the fabric defect detection model as described in embodiment one of the application for training, and each fabric image obtains N training prediction feature maps;

[0096] S5, the K prior boxes are evenly divided into N groups according to the scale order, the N groups of prior boxes are respectively assigned to the N training prediction feature maps according to the scale size of the N groups of prior boxes and the scale size of the N training prediction feature maps, and the corresponding prior boxes are adjusted according to the anchor point information on the training prediction feature maps to obtain all training prediction boxes;

[0097] S6, the overall loss of the network is calculated according to the training prediction boxes and the corresponding GT boxes, and the gradient descent method is used to update the parameters of the model;

[0098] S7, repeat steps S4-S6 until all pictures in the training set are input into the model once, and the model after updating the parameters is used to predict each image in the validation set, the accuracy value AP of each fabric defect class in the validation set is calculated according to the validation prediction frame and the GT frame of each image, and the average accuracy value mAP of all accuracy values AP is calculated;

[0099] S8, repeat step S7 until the average accuracy value mAP calculated for multiple times is equal to a fixed value, and a trained model is obtained.

[0100] Further, in step S1, the data of each fabric defect in each fabric image is labeled in terms of class and position, each fabric defect is completely boxed by a rectangular target GT frame, and each target GT frame is labeled as (class, xmin, ymin, xmax, ymax), wherein class represents the class of the fabric defect contained in the target GT frame, xmin and ymin represent the x-coordinate and y-coordinate of the top left corner of the target GT frame, and xmax and ymax represent the x-coordinate and y-coordinate of the bottom right corner of the target GT frame.

[0101] Further, in step S1, the data is enhanced using Mosaic.

[0102] Further, in step S2, the width and height of the target GT frame are obtained through the data labeling of the target GT frame, and the Kmeans++ algorithm is used to cluster all target GT frames in the fabric image data set according to the width and height of the target GT frame, to obtain K cluster center coordinates, and the K cluster center coordinates are used as the width and height to form K prior boxes.

[0103] Further, step S4 includes:

[0104] S4.1, randomly select X images in the training set, input them into the backbone network Backbone after slice processing for hierarchical feature extraction, and take out four effective feature maps containing different scales and channel numbers, and name them as M5 feature map, M4 feature map, M3 feature map and M2 feature map in order of increasing scale;

[0105] S4.2, input the M4 feature map and the M3 feature map into the conflict filtering residual shrinkage module CF-RSBU for processing to obtain the K4 feature map and the K3 feature map, and input the M5 feature map, the M4 feature map, the M3 feature map, the M2 feature map, the K4 feature map and the K3 feature map into the efficient feature pyramid network EFPN for further feature fusion to obtain the P5 fusion feature map, the P4 fusion feature map and the P3 fusion feature map;

[0106] S4.3, the P5 fusion feature map, the P4 fusion feature map, and the P3 fusion feature map obtained in step S4.2 are input into a prediction head YOLOHead for channel adjustment to obtain three training prediction feature maps.

[0107] Further, in step S4.2, the M4 feature map and the M3 feature map are input into a conflict filtering residual shrinkage module CF-RSBU for processing, including steps of:

[0108] S4.2.1, the M4 feature map is input into a 1*1 pointwise convolution block PW-CBL with an activation function SiLU and a 3*3 convolution block CBL without an activation function SiLU for processing;

[0109] S4.2.2, the output of the 1*1 pointwise convolution block PW-CBL with an activation function SiLU is processed by a 3*3 depthwise convolution block DW-CBL without an activation function SiLU, and then added to the output of the 3*3 convolution block CBL without an activation function SiLU to obtain an added feature map;

[0110] S4.2.3, after all values on the added feature map are processed by an absolute valueization, the processed added feature map is input into an SE module for processing to obtain a processed feature map and a threshold value of the feature map;

[0111] S4.2.4, all values on the processed feature map output by the SE module and the added feature map obtained in step S4.2.2 are collectively processed by a soft threshold denoising to obtain a soft threshold processing result;

[0112] S4.2.5, the soft threshold processing result and the output of the pointwise convolution block with an activation function are added, and then output after a second rectified linear unit activation function ReLU processing to obtain a K4 feature map;

[0113] The M3 feature map is input into the conflict filtering residual shrinkage module CF-RSBU for the same processing as the M4 feature map to obtain a K3 feature map.

[0114] Further, the function used in the soft threshold denoising is:

[0115]

[0116] Wherein, x represents all values on the processed feature map output by the SE module and the added feature map obtained in step S4.2.2, y represents the output value of x after soft threshold processing, and τ represents the threshold value of the feature map obtained after SE module processing.

[0117] Specifically:

[0118] The part of x>τ is not reserved, and only the features after filtering noise in the part of x<-τ are reserved, because the identity mapping adds the original features after dimensionality reduction by 1x1 convolution, so the information loss is not too large, which to some extent preserves the detailed features that may be the defects to be detected.

[0119] Further, in step S4.2.3, the addition result after absolute value processing is input into the SE module for processing, and the processing steps of the SE module include:

[0120] In the SE module, the addition feature map after absolute value processing is sequentially processed by a global average pooling layer, a first fully connected layer FC, a first rectified linear unit activation function ReLU, a second fully connected layer FC, and a first activation function layer Sigmoid.

[0121] The feature map output by the global average pooling layer is multiplied by the feature map output by the first activation function layer Sigmoid to obtain a multiplication feature map, which is the processed feature map output by the SE module.

[0122] Further, in step S4.2, the M5 feature map, the M4 feature map, the M3 feature map, the M2 feature map, the K4 feature map, and the K3 feature map are input into the efficient feature pyramid network EFPN for further feature fusion to obtain a P5 fusion feature map, a P4 fusion feature map, and a P3 fusion feature map, including steps of:

[0123] S4.2.6, the M5 feature map passes through a first convolution block CBL to obtain a C5 feature map, the C5 feature map is upsampled, and the M4 feature map is input into a first fusion bottleneck layer Concat+CSP_3 for processing, the output of the first fusion bottleneck layer Concat+CSP_3 and the output of the upsampling processing are input into a first fusion convolution layer for processing to obtain a C4 feature map.

[0124] S4.2.7, the C4 feature map is upsampled and input into a second fusion bottleneck layer Concat+CSP_3 together with the M3 feature map for processing to obtain a C3 feature map.

[0125] S4.2.8, the C3 feature map is processed by a second convolution block CBL and then upsampled, and the M2 feature map is input into a second fusion convolution layer Concat+CBL for processing to obtain a C2 feature map.

[0126] S4.2.9, the C2 feature map, the C3 feature map, and the K3 feature map are input into a third fusion bottleneck layer Concat+CSP_3 for processing to obtain a P3 fusion feature map.

[0127] S4.2.10, the P3 feature map after being processed by the third convolutional block CBL and the C4 feature map and the K4 feature map are jointly input into a fourth fusion bottleneck layer Concat+CSP_3 for processing to obtain a P4 fusion feature map;

[0128] S4.2.11, the P4 feature map after being processed by the fourth convolutional block CBL and the C5 feature map are jointly input into a fifth fusion bottleneck layer Concat+CSP_3 for processing to obtain a P5 fusion feature map.

[0129] Further, in step S5, adjusting the corresponding prior box according to the anchor point information on the training prediction feature map comprises:

[0130] Each training prediction feature map is divided into a grid of HxW, wherein the center of each grid unit is referred to as an anchor point, and the information of each anchor point includes position and scale adjustment information, classification confidence and bounding box confidence. Then, each anchor point is covered by K / N prior boxes belonging to the corresponding training prediction feature map.

[0131] Each anchor point on the training prediction feature map corresponds to a vector of length num_anchor*(5+num_class), and by dimensionally splitting the vector, a one-dimensional adjustment vector of length 5+num_class corresponding to each prior box is obtained.

[0132] Based on the position and scale adjustment information of each one-dimensional adjustment vector, the position and scale of the corresponding prior box are adjusted to obtain K / N training prediction boxes.

[0133] Further, step S6 comprises:

[0134] S6.1, calculating the bounding box position loss, the classification confidence loss and the bounding box confidence loss according to the training prediction box and the corresponding GT box;

[0135] S6.2, weighted summing the bounding box position loss, the classification confidence loss and the bounding box confidence loss to obtain the overall loss of the network, and using gradient descent method to update the parameters of the fabric defect detection model.

[0136] Specifically:

[0137] According to the position information of each one-dimensional GT vector and each one-dimensional adjustment vector, the CIoU loss is used to calculate the bounding box position loss; according to the classification confidence of each one-dimensional GT vector and each one-dimensional adjustment vector, the binary cross entropy loss is used to calculate the classification confidence loss; and according to the bounding box confidence of each one-dimensional GT vector and each one-dimensional adjustment vector, the binary cross entropy loss is used to calculate the bounding box confidence loss.

[0138] Embodiment three:

[0139] The embodiment provides a fabric defect detection method, comprising the following steps:

[0140] A. Obtain a plurality of fabric images with defects to be detected, and use a fabric defect detection model obtained by the model training method of embodiment two of the application to predict the fabric images to be detected, and output N detection prediction feature maps;

[0141] B. Adjust the prior box on each detection prediction feature map according to the adjustment vector corresponding to each anchor point to obtain a preliminary detection prediction box for each image;

[0142] C. Remove redundant boxes in the preliminary detection prediction box using non-maximum suppression to obtain a detection prediction box on the detection prediction feature map;

[0143] D. Map the detection prediction box on the detection prediction feature map to the original image scale according to the proportional relationship to complete the detection and positioning of the fabric defects.

[0144] The fabric defect detection method described in the embodiment is used to detect and position the fabric image samples, and the detection result is shown in Figure 8 .

[0145] Specifically:

[0146] The size of the fabric image to be detected is 4096x1696 pixels, 9 prior boxes are obtained on the image, and the sizes of the 9 prior boxes are (7, 8), (15, 14), (30, 33), (25, 188), (639, 34), (639, 49), (71, 636), (639, 71), (639, 637), and the sizes of the output 3 detection prediction feature maps of different scales are 20x20, 40x40, and 80x80. For each detection prediction feature map, 3 prior boxes are allocated. For example, for the 20x20 scale detection prediction feature map, the largest 3 size prior boxes (71, 636), (639, 71), and (639, 637) are allocated because the receptive field is the largest; for the 40x40 scale detection prediction feature map, the 3 larger size prior boxes (25, 188), (639, 34), (639, 49), and (639, 75) are allocated; and for the 80x80 scale detection prediction feature map, the 3 prior boxes with the smallest size (7, 8), (15, 14), and (30, 33) are allocated.

[0147] The fabric defect detection model obtained by using the model training method according to any one of Embodiment Two of the present application is used to predict the fabric image to be detected, and the average precision mean mAP and the accuracy (Precision) and recall rate (Recall) corresponding to each fabric defect category are calculated. The selected fabric defect categories in the fabric image to be detected are: sewing, sewing print, bug, hole, scrimp, flaw, miss print, color shade, fold, dirty, pattern error, watermark, hair, defect and wax spot.

[0148] The experimental results are shown in Table 1, and the comparison of detection of overall fabric defect categories before and after the improvement of the network structure is shown in Table 2.

[0149] Table 1

[0150] Class P (Accuracy) R (Recall) mAP (Mean Average Precision) all 0.616 0.465 0.506 sewing head 0.914 0.107 0.537 sewing print 0.789 0.108 0.559 bug 0.77 0.877 0.847 hole 0.587 0.461 0.504 scrimp 0.677 0.5 0.535 flaw 0.799 0.532 0.602 miss print 0.849 0.604 0.622 color shade 0.416 0.636 0.585 fold 0.848 1 0.995 wax spot 0.149 0.455 0.232 defect 0.794 0.8 0.774 hair 0.319 0.136 0.073 watermark 0.489 0.467 0.449 pattern error 0.35 0.0799 0.0774 dirty 0.489 0.213 0.191

[0151] Table 2

[0152]

[0153]

[0154] As shown in Table 1, the present application can realize detection of various categories of fabric defects and achieve high accuracy. As shown in Table 2, the present application is based on YOLOv5, increases the fine-grained feature extraction branch and the receptive field branch of small and medium targets on the efficient feature pyramid network, introduces the jump mapping of the deep residual shrinkage module RSBU and improves the module, obtains the noise reduction conflict filtering residual shrinkage module CF-RSBU, uses the fabric defect detection model obtained by training to detect the fabric image to be detected, and the average precision mean mAP increases by nearly one point, which enhances the detection ability of fabric defects.

[0155] The above-described embodiments only describe the preferred embodiments of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art shall fall within the protection scope of the present application.

Claims

1. A fabric defect detection system based on a fabric defect detection model, characterized in that, The detection model includes a backbone network, a conflict filtering residual shrinking module for noise reduction of feature maps, an efficient feature pyramid network, and a prediction head; The backbone network consists of an input layer, a central layer, a first convolutional bottleneck layer, a second convolutional bottleneck layer, a third convolutional bottleneck layer, a fourth convolutional bottleneck layer, and a spatial pyramid fast pooling module, connected in sequence. The efficient feature pyramid network includes a first convolutional block, a first fusion bottleneck layer, a first fusion convolutional layer, a second fusion bottleneck layer, a second convolutional block, a second fusion convolutional layer, a third fusion bottleneck layer, a third convolutional block, a fourth fusion bottleneck layer, a fourth convolutional block, and a fifth fusion bottleneck layer connected in sequence. The first fusion convolutional layer and the fourth fusion bottleneck layer are connected, and the first convolutional block and the fifth fusion bottleneck layer are connected. The first convolutional bottleneck layer is connected to the second fused convolutional layer. The second convolutional bottleneck layer is connected to the conflict filtering residual shrinkage module and the second fused bottleneck layer, respectively. The third convolutional bottleneck layer is connected to the conflict filtering residual shrinkage module and the first fused bottleneck layer, respectively. The spatial pyramid fast pooling module is connected to the first convolutional block. The first convolutional block is connected to the first fused convolutional layer after upsampling. The conflict filtering residual shrinkage module is connected to the third fusion bottleneck layer and the fourth fusion bottleneck layer, respectively. The third, fourth, and fifth fusion bottleneck layers are connected to the three prediction heads, respectively.

2. The fabric defect detection system based on a fabric defect detection model according to claim 1, characterized in that, The conflict filtering residual shrinkage module includes an input layer, a pointwise convolutional block with an activation function, a convolutional block without an activation function, a depthwise convolutional block without an activation function, a first image addition processing layer, an SE module, a soft thresholding function layer, a second image addition processing layer, and a second rectified linear unit activation function. The SE module includes a global average pooling layer, a first fully connected layer, a first rectified linear unit activation function, a second fully connected layer, a first activation function layer, and an image multiplication processing layer connected in sequence. The global average pooling layer and the image multiplication processing layer are connected together. The input layer is connected to a convolutional block without an activation function and a pointwise convolutional block with an activation function. The pointwise convolutional block with an activation function is connected to a depthwise convolutional block without an activation function and the second image addition processing layer. The convolutional block without an activation function and the depthwise convolutional block without an activation function are both connected to the first image addition processing layer. The first image addition processing layer is connected to the global average pooling layer and the soft thresholding function layer of the SE module. The image multiplication processing layer, the soft thresholding function layer, the second image addition processing layer, and the second rectified linear unit activation function of the SE module are connected in sequence.

3. A method for training a fabric defect detection model, characterized in that, Includes the following steps: S1. Collect several images of fabrics with defects, and perform fabric defect data annotation and data augmentation on each image to construct a fabric image dataset. S2. Use a clustering algorithm to cluster the target ground truth boxes of all images in the fabric image dataset to obtain K prior boxes; S3. Divide the fabric image dataset into a training set and a validation set according to a preset ratio; S4. Randomly select X images from the training set and input them into the fabric defect detection model as described in any one of claims 1-2 for training. Each fabric image will generate N training prediction feature maps. S5. Divide the K prior boxes into N groups in scale order. Based on the scale of the N prior boxes and the scale of the N training prediction feature maps, assign the N prior boxes to the N training prediction feature maps respectively. Adjust the corresponding prior boxes according to the anchor point information on the training prediction feature maps to obtain all training prediction boxes. S6. Calculate the overall network loss based on the training prediction boxes and corresponding ground truth boxes, and use gradient descent to adjust the model parameters. S7. Repeat steps S4-S6 until all images in the training set are input into the model once. Based on the updated model, predict each image in the validation set. Calculate the precision value AP for each fabric defect category in the validation set based on the validation prediction box and GT box of each image, and calculate the average precision value mAP of all precision values ​​AP. S8. Repeat step S7 until the average precision value mAP calculated multiple times equals a fixed value, and the trained model is obtained.

4. The method for training a fabric defect detection model according to claim 3, characterized in that, Step S4 includes: S4.1 Randomly select X images from the training set, slice them, and input them into the backbone network for stepwise feature extraction. Extract four effective feature maps with different scales and number of channels, and name them M5 feature map, M4 feature map, M3 feature map, and M2 feature map in ascending order of scale. S4.2 Input the M4 feature map and M3 feature map into the conflict filtering residual shrinkage module for processing to obtain the K4 feature map and K3 feature map. Input the M5 feature map, M4 feature map, M3 feature map, M2 feature map, K4 feature map and K3 feature map into the efficient feature pyramid network for further feature fusion to obtain the P5 fused feature map, P4 fused feature map and P3 fused feature map. S4.

3. The P5 fusion feature map, P4 fusion feature map, and P3 fusion feature map obtained in step S4.2 are channel-adjusted through the prediction head to obtain three training prediction feature maps.

5. The method for training a fabric defect detection model according to claim 4, characterized in that, Step S4.2 involves inputting the M4 feature map and the M3 feature map into the conflict filtering residual shrinkage module for processing, including the following steps: S4.2.

1. Input the M4 feature map into pointwise convolutional blocks with activation functions and convolutional blocks without activation functions for processing; S4.2.

2. The output of the pointwise convolutional block with activation function is processed by the depthwise convolutional block without activation function, and then added to the output of the convolutional block without activation function to obtain the summed feature map. S4.2.3 After all values ​​on the additive feature map are converted to absolute values, the processed additive feature map is input into the SE module for further processing to obtain the processed feature map and the threshold of the feature map. S4.2.

4. Perform soft thresholding on all values ​​on the processed feature map output by the SE module and the summed feature map obtained in step S4.2.2 to obtain the soft thresholding result. S4.2.

5. Add the soft thresholding result and the output of the pointwise convolutional block with activation function, process it through the second rectified linear unit activation function, and output it to obtain the K4 feature map. The M3 feature map is input into the conflict filtering residual shrinkage module and processed in the same way as the M4 feature map to obtain the K3 feature map.

6. The method for training a fabric defect detection model according to claim 5, characterized in that, The function used for soft thresholding in noise reduction is: ， in, This represents the processed feature map output by the SE module and all values ​​on the summed feature map obtained in step S4.2.

2. express The output value after soft thresholding This represents the threshold of the feature map obtained after processing by the SE module.

7. The method for training a fabric defect detection model according to claim 5, characterized in that, In step S4.2.3, the addition result is converted to absolute value and then input into the SE module for processing. The SE module processing steps include: In the SE module, the summed feature map after absolute value processing is processed sequentially through a global average pooling layer, a first fully connected layer, a first rectified linear unit activation function, a second fully connected layer, and a first activation function layer. The feature map output by the global average pooling layer and the feature map output by the first activation function layer are multiplied to obtain a multiplied feature map, which is the processed feature map output by the SE module.

8. The method for training a fabric defect detection model according to claim 5, characterized in that, In step S4.2, the M5, M4, M3, M2, K4, and K3 feature maps are all input into the efficient feature pyramid network for further feature fusion to obtain the P5 fused feature map, P4 fused feature map, and P3 fused feature map. This includes the following steps: S4.2.6 The M5 feature map is processed by the first convolutional block to obtain the C5 feature map. The C5 feature map is upsampled and then fed together with the M4 feature map into the first fusion bottleneck layer for processing. The output of the first fusion bottleneck layer and the output of the upsampling process are fed together into the first fusion convolutional layer for processing to obtain the C4 feature map. S4.2.7 and C4 feature maps, after being upsampled, are input together with M3 feature map into the second fusion bottleneck layer for processing to obtain C3 feature map; After the S4.2.8 and C3 feature maps are processed by the second convolutional block and upsampled, they are input together with the M2 feature map into the second fusion convolutional layer for further processing to obtain the C2 feature map. S4.2.9, C2 feature map, C3 feature map, and K3 feature map are input together into the third fusion bottleneck layer for processing to obtain the P3 fusion feature map; After processing by the third convolutional block, the S4.2.10 and P3 fused feature maps, together with the C4 and K4 feature maps, are input into the fourth fusion bottleneck layer for further processing to obtain the P4 fused feature map. The S4.2.11 and P4 fused feature maps, after being processed by the fourth convolutional block, are input together with the C5 feature map into the fifth fusion bottleneck layer for further processing to obtain the P5 fused feature map.

9. The method for training a fabric defect detection model according to claim 3, characterized in that, Step S6 includes: S6.1 Calculate the bounding box position loss, classification confidence loss, and bounding box confidence loss based on the trained predicted boxes and corresponding ground truth boxes; S6.

2. The bounding box position loss, classification confidence loss, and border confidence loss are weighted and summed to obtain the overall network loss. The overall loss value is backpropagated to the fabric defect detection model, and the parameters of the fabric defect detection model are updated using the gradient descent method.

10. A method for detecting fabric defects, characterized in that, Includes the following steps: A. Obtain several images of fabrics to be detected with defects, and use the fabric defect detection model obtained by the model training method as described in any one of claims 3-9 to predict the fabric images to be detected, and output N detection prediction feature maps. B. Adjust the prior box on each detection prediction feature map according to the adjustment vector corresponding to each anchor point to obtain the preliminary detection prediction box for each image. C. Use non-maximum suppression to remove redundant boxes in the initial detection prediction boxes to obtain the detection prediction boxes on the detection prediction feature map; D. Based on the proportional relationship, the detection prediction box on the scale of the detection prediction feature map is mapped to the scale of the original image to complete the detection and localization of fabric defects.

Images