A fabric defect detection method based on improved RefineDet
By improving the RefineDet network, adding attention mechanism and SE module, the problems of traditional fabric defect detection methods in manual dependence and missed detection are solved, and more efficient and accurate defect detection is achieved, especially in the identification of complex features. Significant improvement.
Patent Information
- Application Number
- CN202111302358.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-04
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-11-04
AI Technical Summary
Traditional fabric defect detection methods rely on manual detection and are susceptible to subjective factors and external environment, resulting in mis-checking and missed detection. The automation method has poor results in identifying defects with complex characteristics.
Using the improved RefineDet network, the attention mechanism and SE module are added, and through the asynchronous gradient descent training strategy, multi-scale features are extracted and the classification capabilities of the model are enhanced, and detection accuracy and speed are improved.
It significantly improves the accuracy and speed of fabric defect detection, reduces the number of parameters, can extract fabric characteristics more effectively, and enhances the detection ability and classification effect of the model.
Smart Images

Figure CN114065847B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of computer vision, and in particular to a fabric defect detection method based on improved RefineDet. Background Art
[0002] In the production process of textile products, quality control runs through the entire process. Fabrics with quality problems will greatly affect their prices. Therefore, fabric defect detection is the last process for finished fabrics to leave the factory, and it is also a key link in controlling the quality of finished fabrics. Traditional fabric defect detection methods are mainly based on manual detection. Manual detection requires a lot of labor and is easily affected by objective factors such as the subjective factors of the inspectors and the external environment, resulting in false detection and missed detection, resulting in poor detection results. In order to improve the efficiency of fabric defect detection, researchers have proposed a large number of automatic fabric defect detection methods based on image processing since the 1990s. In addition, since 2015, deep learning methods have been introduced into the field of fabric defect detection. These methods have broken through the limitations of traditional fabric defect detection and the algorithm performance has been greatly improved. Thus, an efficient method for fabric defect detection using deep learning has been realized. Summary of the invention
[0003] The purpose of the present invention is to provide a fabric defect detection method based on improved RefineDet, and the obtained concrete has excellent working performance, mechanical properties, crack resistance and abrasion resistance.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is:
[0005] A fabric defect detection method based on improved RefineDet comprises the following steps:
[0006] Step 1, collect fabric images and preprocess them to obtain a data set, and divide the data set into a training set and a test set in proportion;
[0007] Step 2: input the training samples in the training set of step 1 into the fabric defect detection network based on the improved RefineDet in sequence, and the improved RefineDet fabric defect detection network uses the feature extraction module with the attention mechanism to extract features, and obtain the extracted feature map;
[0008] Step 3: After the extracted feature map is fused, it is input into the SE module, and the defects in the feature map are classified by the SE module. The target location module ODM is used to predict the confidence value of the target of each category of defects, and the fabric defect detection network of RefineDet is improved to finally predict the coordinates of the defect target of each category of the training sample;
[0009] Step 4: Use the asynchronous gradient descent training strategy to adjust the parameters of the fabric defect detection network of the improved RefineDet to obtain a fabric defect detection model based on the improved RefineDet;
[0010] Step 5: Input the test samples in the test set into the fabric defect detection model based on the improved RefineDet in sequence, obtain the coordinates of the defect targets of each category in the test samples, and frame the defect targets according to different categories in the test sample images.
[0011] Furthermore, the preprocessing in step 1 is to collect the fabric images and mark the defects, record the locations and categories of the defects in the fabric images, and enhance the training data by random cropping and rotation to obtain a data set.
[0012] Furthermore, the number of the SE modules is four, which can process feature maps of four different sizes.
[0013] Furthermore, the feature extraction module to which the attention mechanism is added in step 2 is an improved ANCHOR refinement module. The improved ANCHOR refinement module is improved compared to the conventional ANCHOR refinement module in that, based on the VGG16 network in the ANCHOR refinement module, the network is modified by removing the last two fully connected layers and the last pooling layer of the VGG16 network, modifying the VGG16 network into a fully convolutional network, and retraining the modified network.
[0014] Furthermore, in step 4, the parameters of the fabric defect detection network of RefineDet are adjusted and improved using the gradient descent training strategy, and the training method of the fabric defect detection model is obtained as follows:
[0015] Step 41, set the learning rate and weight decay value, and set the maximum number of iterations;
[0016] Step 42, inputting the training samples of the training set images into the fabric defect detection network based on the improved RefineDet in sequence;
[0017] Step 43, improving the fabric defect detection network of RefineDet to predict the coordinates of defect targets of each category of training samples;
[0018] Step 44, calculating the difference between the coordinates of the defect target predicted in step 43 and the actual position coordinates of the defect marked on the image through a loss function;
[0019] Step 45, adjusting the parameters of the fabric defect detection network based on the improved RefineDet according to the learning rate, the weight decay value and the difference value in step 44, and returning to step 42 until all the training samples in the training set are traversed;
[0020] Step 46, the number of iterations is increased by 1, and it is determined whether the number of iterations reaches the maximum number of iterations. If not, go to step 42. If yes, save the parameters of the last iteration and generate a fabric defect detection model based on the improved RefineDet.
[0021] Furthermore, the loss calculated by the loss function in step 45 is the sum of the ARM and ODM losses, and the total loss loss function is calculated as:
[0022]
[0023] Where i is the index of the labeled box in the training batch; N arm and N odm They refer to the number of positive sample anchor steps in the anchor refinement module and the target detection module respectively; p i Refers to the confidence that the predicted anchor i is a target; x i Refers to the coordinates of Anchor i predicted after ARM refinement; c i is the object category of the predicted label box in the object detection module; t i are the coordinates of the predicted marker box in the object detection module; L b represents the binary classification loss, that is, the cross entropy loss between the target and the non-target; L r represents the regression loss; L m Represents multi-class classification loss; exponential function Indicates that when box i belongs to the defect class, the output is 1, otherwise it is 0; is the real category label of anchor i; It is the actual position and size of Anchor i.
[0024] The beneficial effects of the present invention are:
[0025] 1. The improved VGG16 used in the present invention can reduce the number of parameters without significantly affecting the performance. The multi-scale feature extraction method extracts features from feature maps of different downsampling layers, which can simultaneously utilize more comprehensive fabric features and improve the detection capability of the model;
[0026] 2. The present invention uses an improved RefineDet network model, which can highlight the detailed information of defects and improve the classification effect of the model after adding the attention mechanism and SE module;
[0027] 3. The present invention improves the traditional RefineDet network model so that the model has better detection accuracy and greatly improves the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 It is a schematic diagram of the overall process structure of fabric defect detection of the present invention;
[0029] Figure 2 The structural diagram of the present invention adding the attention mechanism to the ANCHOR refinement module;
[0030] Figure 3 The present invention introduces a schematic diagram of the SE module;
[0031] Figure 4 Comparison effect diagram of accuracy results before and after the improvement of the method of the present invention;
[0032] Figure 5 Comparison diagram of defect location results before and after the improvement of the method of the present invention. DETAILED DESCRIPTION
[0033] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.
[0034] like Figure 1 As shown, the embodiment of the present invention provides a fabric defect detection method based on improved RefineDet, and the specific steps are as follows:
[0035] Step 1. Select holes (plain background), holes (knitted background), stains (plain background), stains (striped background), yarn defects (plain background), yarn defects (striped background), linear (plain background), and linear (checkered background) from the TILDA fabric database. The four defect types of holes, linear, yarn defects, and stains are used as image datasets. The TILDA dataset contains 1600 images with a resolution of 768×512, from which 1200 images are selected. The input image size is 320×320, and the training data is enhanced by random cropping and rotation. Randomly select images from the image dataset collected in step 11 for cropping; then, the cropped images are rotated separately. Each image type used in the experiment is randomly selected as a training set and a test set, of which 70% are used as a training set and 30% are used as a test set.
[0036] Step 2: Input the fabric images in the training set in step 1 into the fabric defect detection network model for training. The fabric defect detection model extracts the features of the fabric images through the ANCHOR refinement module. In order to allow the model to extract the detailed information of the fabric image, the attention mechanism is innovatively added to the feature extraction module. The principle of the attention mechanism is to quickly scan the entire image to obtain the target area that needs to be focused on, which is generally referred to as the focus of attention, and then invest more attention resources in this area to obtain more detailed information of the target that needs to be focused on, while suppressing other useless information, such as Figure 2 shown.
[0037] Step 3: In order to improve the classification performance of the model, the present invention innovatively adds the SE module (step squeeze and excitation, SE), such as Figure 3 As shown in Figure 2, the features extracted in step 2 are fused, and the fused feature map is input into the SE module. The SE module is simple in concept, easy to implement, and can be easily loaded into the existing network model framework. Not only that, the SE module can significantly improve the classification ability of the network, and the module has less negative impact on the network, specifically:
[0038] Step 31: The feature map output by the ANCHOR refinement module is used as the input of TCB. At this time, four features of different sizes are obtained.
[0039] Step 32: The result after TCB processing is used as the input of the SE module, that is, four SE modules are added after TCB. The SE module is composed of a fully connected layer and a nonlinear constraint layer. It can be easily loaded into the existing network model. After processing by four SE modules, four feature maps of different sizes are obtained. The SE module performs weighted operations on the channels of the convolutional features, enhances the channel features with valid classification, and suppresses the channel features with invalid classification;
[0040] Step 33. In order to match the dimensions, the high-level feature map is enhanced by inverse convolution, and then the corresponding elements of the high-level feature map are summed. The sizes of the high-level feature maps are 40×40×256, 20×20×256, 10×10×256, and 5×5×256, respectively. Finally, a convolution layer is added to the summed result to improve the discernibility of the detection results. Finally, the feature map is classified after processing according to the four SE modules to identify whether the defects belong to the four categories of holes, lines, yarn defects, and stains.
[0041] Step 4: ODM (object detection module) regresses the results to the exact defect location and predicts the multi-category labels of the defects.
[0042] The feature extraction network of the present invention not only increases the receptive field, but also obtains more comprehensive fabric features through the fusion of features of different scales and different levels. A series of different modules are designed in the network, among which the goal of ARM (ANCHOR refinement module) is to eliminate the negative sample Anchor step in order to reduce the search space of the classifier, and roughly adjust the size and position of the Anchor step to provide better initialization results for the subsequent regressor. TCB transmits the features in the Anchor refinement module, which can not only predict the position, size and category label of the target in ODM, but also inherit large-scale context by adding the advanced features of ARM to the transmitted features to improve the accuracy of detection. ODM classifies and locates the target area. These three categories are all possessed by the traditional RefineDet model, on the basis of which, the attention mechanism and SE module are innovatively added. Among them, the attention mechanism is introduced in ARM, which can make the model focus on the important features of the defect and suppress unnecessary features, so as to enhance the ability of defect detection; the SE module structure is introduced in TCB to improve the classification performance of the network. To ensure that the network can meet the multi-scale fabric defect detection.
[0043] Step 5: Use the gradient descent training strategy to adjust the parameters of the fabric defect detection network of RefineDet, and obtain the specific training method of the fabric defect detection network model as follows:
[0044] Step 51, set the learning rate θ to 0.0001, the weight decay value ω to 0.0005, and the maximum number of iterations to 100000.
[0045] Step 52: randomly select N fabric images and defect position coordinates from the training set and input them into the fabric defect detection network.
[0046] Step 53: Input the feature map extracted by the fabric defect detection network into the network model. The fused features can better obtain the category and location information of the defects.
[0047] Step 54: The number of bounding boxes in each grid is predicted by the fabric defect detection network, the SE classification module is used to classify the fabric image, and the ODM target positioning module predicts the confidence value of the defect target in each frame; during the iteration process of the network, the loss function is used to calculate the difference between the output of the neural network and the expected output to determine the adjustment direction of the iteration. The loss function can reflect the quantified inconsistency between the output of the current network and the actual result, so as to adjust the network parameters during the iteration process.
[0048] Step 55: Calculate the difference between the confidence of the defect target predicted in step 44 and the defect position coordinates through the loss function; the loss function in step 45 is the sum of ARM (VGG16) and ODM losses, and the total loss loss function is calculated as follows:
[0049]
[0050] Where i is the index of the labeled box in the training batch; N arm and N odm They refer to the number of positive sample Anchor steps in ARM (VGG16) and ODM respectively; p i Refers to the confidence that the predicted anchori is a target; x i Refers to the coordinates of Anchori predicted after ARM (VGG16) refinement; c i is the object category of the predicted label box in ODM; t i are the coordinates of the predicted marker box in ODM; L b represents the binary classification loss, that is, the cross entropy loss between the target and the non-target; L r represents the regression loss; L m Represents multi-class classification loss; exponential function Indicates that when box i belongs to the defect class, the output is 1, otherwise it is 0; is the real category label of anchori; It is the real location and size of Anchori;
[0051] Step 56, adjusting the parameters of the fabric defect detection network of the improved RefineDet according to the learning rate, the weight decay value and the difference value in step 45, until all images in the training set are traversed;
[0052] Step 57, the number of iterations is increased by 1, and steps 42 to 46 are executed cyclically until the maximum number of iterations is reached, and the parameters of the fabric defect detection network of the last iteration are retained to generate a fabric defect detection network model;
[0053] Step 6: Input the fabric images in the test set into the fabric defect detection model obtained in step 4, obtain the target defects and position coordinates in the image, and frame the location of the defects in the image. Input the fabric images in the test set into the fabric defect detection model, and after the image is subjected to layer-by-layer convolution operations of the network model, obtain the confidence that the pixels in each area belong to the fabric defect target, and predict the position coordinates of the defect target in the image. By setting the confidence threshold, the target greater than the threshold is determined as the defect target, and the position of the defect in the image is accurately framed according to the coordinate value;
[0054] The defect detection performance indicators mainly include accuracy and mean average accuracy, and the hardware resources used are also listed. The defect detection indicators used in Table 2 are given below.
[0055] 1) Confusion Matrix
[0056] Table 1 Confusion matrix
[0057]
[0058] TP, TN, FP, and FN are the abbreviations of true positive, true negative, false positive, and false negative. Positive and negative represent the predicted results. If the prediction is positive, it is positive, and if the prediction is negative, it is negative. True and false represent whether the predicted result is the same as the true result. If they are the same, it is true, and if they are different, it is false.
[0059] 2) Precision
[0060] That is, the model judges that the image is positive, and the probability that the image is indeed positive. The accuracy of the positive class refers to the number of images that the model judges to be positive and the true class is also positive / the number of images judged by the model to be positive. It measures the probability that a positive class classified by a classifier is indeed positive. The calculation formula is as follows:
[0061]
[0062] 3) Recall
[0063] That is, if there are 5 positive classes in all the images, and the model can correctly judge these 5 images as positive, then the recall rate is 100%. The recall rate refers to the number of images that the model judges as positive and the true class is also positive / the number of images that the true class is positive. It measures the ability of a classifier to find all positive classes. The calculation formula is:
[0064]
[0065] 4) AP and mAP
[0066] Calculate the Precision and Recall of each prediction result and draw a curve according to the relationship, which is the PR curve. Before calculating AP, in order to smooth the PR curve and reduce the impact of curve jitter, first interpolate the PR curve. By calculating the average value of the Precision value corresponding to each recall value, you can get a numerical evaluation indicator: AP, which is used to measure the detection ability of the trained model on the category of interest.
[0067] AP measures the quality of detection of one class, while mAP measures the quality of detection of multiple classes. For example, if there are two classes, the AP value of class A is 0.5 and the AP value of class B is 0.2, then mAP = (0.5 + 0.2) / 2 = 0.35.
[0068] The images in the test set were input into the fabric defect detection model to obtain the detection results of the model. The results were compared with other neural networks. The results showed that good results were achieved in the four classification effects of TILDA dataset, hole, line, yarn defect and stain. Compared with the traditional RefineDet defect detection method, the method proposed in the present invention has lower losses and better fitting effect. In terms of accuracy, the accuracy of the improved model increased by 5.0%, with good improvement effect, such as Figure 4 shown.
[0069] The performance of this model is compared with that of the RefineDet network model before improvement, and the results are shown in Table 1.
[0070] Table 2 Average accuracy before and after the test method improvement
[0071]
[0072] The experimental results show that in the TILDA dataset, holes, lines, stains, and yarn defects, it can be seen from the improved AP in Table 1 that the method proposed in the present invention has a good classification effect on holes and stains, but the classification of yarn defects and lines is slightly worse. The main reason for the low average accuracy of yarn defects is that the characteristics of yarn defects are relatively complex and diverse, and some yarn defects have large gaps, which are similar to the characteristics of holes, resulting in low network recognition accuracy for yarn defects. The main reason for the low average accuracy of lines is that it is difficult to identify lines when they are close to the background of the fabric image. The model cannot distinguish between the background and the line, and some lines are lighter in color and difficult to identify.
[0073] The model proposed by the present invention can accurately detect common defects in fabrics. In addition, while obtaining accurate detection results, the improved model has better positioning effect, such as Figure 5 shown.
[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A fabric defect detection method based on improved RefineDet, characterized in that: The following steps are involved: Step 1: Collect fabric images and preprocess them to obtain a dataset, and divide the dataset into a training set and a test set in proportion; Step 2: The training samples in the training set of step 1 are sequentially input into the fabric defect detection network based on the improved RefineDet. The improved RefineDet fabric defect detection network uses a feature extraction module with an attention mechanism to extract features and obtain an extracted feature map. The feature extraction module with the attention mechanism added in step 2 is an improved ANCHOR refinement module. Compared with the conventional ANCHOR refinement module, the improved ANCHOR refinement module is improved by modifying the VGG16 network in the ANCHOR refinement module by removing the last two fully connected layers and the last pooling layer of the VGG16 network, modifying the VGG16 network into a fully convolutional network, and retraining the modified network. Step 3: The extracted feature maps are fused and input into the SE module. The SE module is used to classify the defects in the feature maps, and the object location module (ODM) is used to predict the confidence value of the target of each category of defects. The improved RefineDet fabric defect detection network finally predicts the coordinates of the defect target of each category of the training sample. Step 4: Use the asynchronous gradient descent training strategy to adjust the parameters of the improved RefineDet fabric defect detection network to obtain a fabric defect detection model based on the improved RefineDet; The loss calculated by the loss function in step 4 is the sum of ARM and ODM losses. The total loss function is calculated as: ; in, i is the index of the labeled box in the training batch; N arm and N odm They refer to the number of positive sample anchor steps in the Anchor refinement module and the target detection module respectively; p i Refers to the confidence that the predicted anchor i is a target; x i Refers to the Anchor predicted after ARM refinement i coordinates of c i is the object category of the predicted label box in the target detection module; t i are the coordinates of the predicted marker box in the target detection module; L b Represents the binary classification loss, that is, the cross entropy loss between target and non-target; L r represents the regression loss; L m Represents multi-class classification loss; exponential function Indicates when the frame i If it belongs to the defect class, output 1, otherwise 0; is the true category label of Anchor i; is the actual position and size of Anchor i; Step 5: Input the test samples in the test set into the fabric defect detection model based on the improved RefineDet in sequence to obtain the coordinates of the defect targets of each category in the test samples, and frame the defect targets according to different categories in the test sample images.
2. The fabric defect detection method based on the improved RefineDet according to claim 1, characterized in that: The preprocessing in step 1 is to collect the fabric images and mark the defects, record the location and category of the defects in the fabric images, and enhance the training data by random cropping and rotation to obtain a data set.
3. The fabric defect detection method based on the improved RefineDet according to claim 1, characterized in that: The number of the SE modules is four, which can process feature maps of four different sizes.
4. The fabric defect detection method based on the improved RefineDet according to claim 1, characterized in that: In step 4, the parameters of the fabric defect detection network of RefineDet are adjusted and improved using the gradient descent training strategy, and the training method of the fabric defect detection model is obtained as follows: Step 41: Set the learning rate and weight decay value, and set the maximum number of iterations; Step 42: input the training samples of the training set images into the fabric defect detection network based on the improved RefineDet in sequence; Step 43: Improve the fabric defect detection network of RefineDet to predict the coordinates of each category of defect targets in the training samples; Step 44: Calculate the difference between the coordinates of the defect target predicted in step 43 and the actual position coordinates of the defect marked on the image using a loss function; Step 45: Adjust the parameters of the fabric defect detection network based on the improved RefineDet according to the learning rate, the weight decay value, and the difference value in step 44, and return to step 42 until all training samples in the training set are traversed; Step 46: The number of iterations is increased by 1, and it is determined whether the number of iterations reaches the maximum number of iterations. If not, go to step 42. If so, save the parameters of the last iteration and generate a fabric defect detection model based on the improved RefineDet.
