Fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion
Through the hybrid expansion convolution and multi-scale feature fusion method, the problems of low efficiency and large error in traditional fabric defect detection are solved, and high-precision and efficient automated detection are achieved, which is suitable for the identification and positioning of multiple fabric defects.
Patent Information
- Application Number
- CN202111247663.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-10-26
AI Technical Summary
Traditional fabric defect detection relies on manual operation, resulting in low efficiency and high error rate, making it difficult to meet high-quality production needs, especially when the fabric moves quickly, workers' fatigue increases, and errors are inevitable.
Using fabric defect detection method based on hybrid expansion convolution and multi-scale feature fusion, image features are extracted through hybrid expansion convolution module, combined with regional suggestions module and classification regression module, the network parameters are adjusted using asynchronous gradient descent training strategy to realize automated detection .
It improves detection accuracy and speed, reduces calculation amount, can effectively detect defects in different sizes and positions, and improves textile product quality and production efficiency.
Smart Images

Figure CN114067170B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of combining computer vision technology with textile technology, and relates to a fabric defect detection method based on mixed dilated convolution and multi-scale feature fusion. Background Art
[0002] Defect detection is an important part of improving product quality for textile enterprises. However, due to the wide variety of textile products and the randomness of defect colors, structures and shapes, automatic defect detection has become a very challenging task. Traditionally, fabric defect detection is done manually by workers, but due to the limitations of the human eye, if the fabric moves too fast, manual operation will inevitably cause fatigue, leading to errors, which undoubtedly limits the speed of high-quality fabric production. If enterprises want to improve the efficiency of fabric defect detection without reducing the quality of textile products, it is necessary for textile enterprises to replace manual detection with intelligent detection. Deep learning is a new type of artificial intelligence technology that has emerged in recent years. Through the construction of high-level neural networks, it can comprehensively express the characteristics of fabric images and the essential characteristics of defects in a concise and effective form, thereby realizing the detection of fabric defects. Summary of the invention
[0003] The purpose of the present invention is to provide a fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion, and the resulting concrete has excellent working performance, mechanical properties, crack resistance and abrasion resistance.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is:
[0005] The fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion includes the following steps:
[0006] Step 1, collect fabric images and preprocess them to obtain a data set, and divide the data set into a training set and a test set in proportion;
[0007] Step 2: Input the training samples in the training set in step 1 into the fabric defect detection network for training. The fabric defect detection network extracts the features of the image through a hybrid dilated convolution module. Four parallel paths are designed in the hybrid dilated convolution module, three of which are used to extract features of different scales for detecting fabric defects of different sizes, and the other path is used to obtain nonlinear features of defects at different positions.
[0008] Step 3: Use the optimal feature fusion method to perform feature fusion, and input the obtained features into the region proposal module to obtain the defect suggestion box;
[0009] Step 4: The classification and regression module combines the features obtained in step 2 with the defect suggestion box obtained in step 3 to perform regression of the final defect position and classification of the defect type, and obtain the final defect detection result of each training sample;
[0010] Step 5: Using the asynchronous gradient descent training strategy to adjust the parameters of the fabric defect detection network to obtain a fabric defect detection model;
[0011] Step 6: Input the test set in step 1 into the fabric defect detection network model to obtain the target defects and position coordinates in the image, and use the candidate box to frame the defect target in the image.
[0012] Furthermore, the preprocessing method in step 1 is to collect fabric images with defects, enhance the training data by random cropping and rotation, and mark the position coordinates of the actual defects on the images.
[0013] Furthermore, the fabric defect detection network includes a feature extraction module, a region proposal module, and a classification and regression module. The feature extraction module includes an image shallow processing module and a hybrid dilated convolution module processing module. The image shallow processing module contains four convolution layers and two pooling layers for capturing the texture and edge features of the fabric and reducing the size of the feature map. The hybrid dilated convolution module processing module includes nine hybrid dilated convolution modules for obtaining multi-scale features while reducing the number of parameters.
[0014] Furthermore, the candidate frame setting in step 3 includes 3 pixel ratios and 3 aspect ratios, which are combined to obtain 9 different selection methods. The pixel ratio is the ratio of the number of pixels in the candidate frame to the number of pixels in the complete image, and the aspect ratio is the ratio of the width to the length of the candidate frame; the area calculation formula of the candidate frame is:
[0015] S i =(λ i B) 2 ,i=1,2,3
[0016] The calculation formula for the length and width of the candidate box is:
[0017]
[0018] Among them, S i represents the area of the candidate box, λ i Represents the three ratios of the candidate box, B represents the size of each pixel on the feature map corresponding to the original image, R j Indicates the three aspect ratios of the candidate box, H i,j Indicates the width of the candidate box, W i,j Indicates the length of the candidate box.
[0019] Furthermore, the training method of the fabric defect detection model is as follows:
[0020] Step 21, set the learning rate θ and weight decay ω, and set the maximum number of iterations t_max;
[0021] Step 22, inputting the training samples of the training set images into the fabric defect detection network in sequence;
[0022] Step 23, dividing the feature map extracted by the fabric defect detection network into a plurality of grids of the same size, wherein each grid predicts B bounding boxes, where B is a constant;
[0023] Step 24: Use the logistic classifier to predict the confidence value of the defect target in each border, and give the center coordinates and length and width values of each border;
[0024] Step 25, calculating the difference between the confidence value of the defect target predicted in step 24 and the position coordinates of the actual defect on the image in step 22 by using the loss function;
[0025] Step 26, adjusting the parameters of the fabric defect detection network according to the learning rate θ, the weight decay ω and the difference value in step 25, returning to step 22, until all the training samples in the training set are traversed, and then going to the next step;
[0026] Step 27: The number of iterations increases by 1, and it is determined whether the maximum number of iterations t is reached. max If not, go to step 22. If yes, save the parameters of the last iteration and generate a fabric defect detection network model.
[0027] Furthermore, the loss function in step 25 is: Loss = Error iou +Error cls +Error bbl , where Error iou Represents the confidence loss function, Error cls is the logistic classifier loss function, Error bbl is the bounding box loss function;
[0028] The confidence loss function Error iou for:
[0029]
[0030] Among them, S k represents the kth network, k = 1, 2, ..., i = 0, 1 ..., j = 0, 1 ..., B represents the jth bounding box, Or 1 indicates whether there is a fabric defect target in the jth bounding box in the ith network. If it exists, it is 1, and if it does not exist, it is 0. nofbd is the weight of the confidence loss of the target without defects, C i is the confidence score, is the intersection ratio between the predicted bounding box and the true bounding box;
[0031] The logistic classifier loss function is:
[0032]
[0033] Among them, p i (c) is the target true category score, Score the predicted target class;
[0034] The bounding box loss function Error bbl for:
[0035]
[0036] Among them, λ bbl is the weight of the coordinate error of the defect, (x i ,y i ,ω i ,h i ) are the center coordinates and length and width values of the real defect, are the center coordinates and length and width values of the predicted defect target.
[0037] Furthermore, the optimal feature fusion method in step 3 is obtained by training with multiple fusion methods, comparing the feedback effects obtained from the training, and screening the optimal fusion method. The beneficial effects of the present invention are:
[0038] 1) The present invention adopts a multi-scale feature extraction method to extract features from feature maps of different downsampling layers, which can simultaneously utilize more comprehensive fabric features and improve the detection ability of the model;
[0039] 2) The present invention uses a hybrid dilated convolution module to build a lightweight convolutional neural network model, which increases the receptive field of the convolution kernel while keeping the number of parameters unchanged, greatly reduces the amount of network calculation, avoids reducing the information of the feature map due to downsampling, and improves the detection speed;
[0040] 3) The present invention combines the multi-scale feature extraction method and the hybrid dilated convolution module to build a convolutional neural network model, so that the model has good detection accuracy and greatly improves the detection speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1It is a schematic diagram of the fabric defect detection structure of the present invention;
[0042] Figure 2 Schematic diagram of the feature fusion module of the present invention;
[0043] Figure 3 Schematic diagram of the hybrid dilated convolution module of the present invention;
[0044] Figure 4 Detection effect diagram of the method of the present invention. DETAILED DESCRIPTION
[0045] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.
[0046] like Figure 1 As shown, the embodiment of the present invention provides a fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion, and the specific steps are as follows:
[0047] S1. We selected white-gray fabrics, dark red fabrics, and mesh fabrics from the TILDA fabric database as image datasets. We selected 360 defective images, which contain six types of defects: broken yarn, thread ends, holes, fuzz balls, scratches, and stains. Each type contains 60 samples, and the size of the original image is 768×512 pixels. The dark red fabric and mesh fabric were collected using industrial cameras in a textile professional laboratory environment. The dark red fabric contains four defects: thread ends, broken yarn, cotton knots, and fuzz balls. The mesh fabric contains four defects: thread ends, cotton knots, holes, and stains. Each defect in the two fabrics contains 60 fabric image samples, and the image size is 800×600.
[0048] The training data was enhanced by random cropping and rotation. Images were randomly selected from the collected image dataset for cropping, and the cropped image size was 256×256; then, the cropped images were rotated 90 degrees, 180 degrees, and 270 degrees respectively. The final number of samples used for the experiment was 8400, of which 3600 were selected from the TILDA database fabrics, and 2400 were dark red fabrics and mesh fabrics. Each image type was randomly selected as a training set and a test set. 80% was used as a training set and 20% as a test set.
[0049] S2. Build the network structure. The network structure consists of three main parts: feature extraction, region proposal network, and classification and regression. The feature extraction part can be divided into two stages: the shallow processing stage and the mixed dilated convolution module processing stage. The shallow processing stage contains four convolution layers and two pooling layers, which can capture the texture and edge features of the fabric and reduce the size of the feature map. The processing stage of the mixed dilated convolution module contains nine dilated convolution modules to obtain multi-scale features while reducing the number of parameters. Figure 2 shown.
[0050] S3. In the entire fabric defect detection network structure, the feature extraction network not only increases the receptive field, but also obtains more comprehensive fabric features through the fusion of features of different scales and levels. A series of candidate frames of different sizes are designed in the network to ensure that the network can meet the multi-scale fabric defect detection. Four parallel paths are designed in the hybrid dilated convolution module, three of which are used to extract features of different scales for detecting fabric defects of different sizes, and the other path is used to obtain nonlinear features of defects at different positions. Figure 3 shown.
[0051] S4. The candidate box setting includes 3 pixel ratios (8, 16, 32) and 3 aspect ratios (1:2, 1:1, 2:1). They can be combined to get 9 different options. The area calculation formula of the candidate box is:
[0052] S i =(λ i B) 2 ,i=1,2,3
[0053] The calculation formula for the length and width of the candidate box is:
[0054]
[0055] Among them, S i represents the area of the candidate box, λ i Represents the three pixel ratios of the candidate box, B represents the size of each pixel on the feature map corresponding to the original image, and R j Indicates the three aspect ratios of the candidate box, H i,j Indicates the width of the candidate box, W i,j Indicates the length of the candidate box.
[0056] S5, inputting the training set in step S1 into the fabric defect detection network for training, and adjusting the parameters of the fabric defect detection network using an asynchronous gradient descent training strategy to obtain a fabric defect detection network model; the specific training method is:
[0057] S51, set the learning rate θ to 0.001, the weight decay ω to 0.0005, and set the maximum number of iterations t max It is 400500.
[0058] S52, randomly selecting N fabric images and defect position coordinates from the training set and inputting them into the fabric defect detection network.
[0059] S53, dividing the feature map extracted by the fabric defect detection network into S×S grids, and predicting B bounding boxes for each grid. Figure 2 As shown in the figure, the fused features can better obtain the category and location information of the defects.
[0060] S54, predict the number of bounding boxes in each grid through the fabric defect detection network, use the logistic classifier to predict the confidence value of the defect target in each border, and give the center coordinates and length and width values of each border; in the iterative process of the network, it is necessary to use the loss function to calculate the difference between the output of the neural network and the expected output to determine the adjustment direction of the iteration. The loss function can reflect the quantified inconsistency between the output of the current network and the actual result, so as to adjust the network parameters in the iterative process.
[0061] S55, calculating the difference between the confidence of the defect target predicted in S54 and the defect position coordinates through a loss function; the loss function is: Loss = Error iou +Error cls +Error bbl , where Error iou Represents the confidence loss function, Error cls is the logistic classifier loss function, Error bbl is the bounding box loss function;
[0062] The confidence loss function Error iou for:
[0063]
[0064] Among them, S k represents the kth network, k = 1, 2, ..., i = 0, 1 ..., j = 0, 1 ..., B represents the jth bounding box, Or 1 indicates whether there is a fabric defect target in the jth bounding box in the ith network. If it exists, it is 1, and if it does not exist, it is 0. nofbd is the weight of the confidence loss of the target without defects, C i is the confidence score, is the intersection ratio between the predicted bounding box and the true bounding box;
[0065] The logistic classifier loss function is:
[0066]
[0067] Among them, p i (c) is the target true category score, Score the predicted target class;
[0068] The bounding box loss function Error bbl for:
[0069]
[0070] Among them, λ bbl is the weight of the coordinate error of the defect, (x i ,y i ,ω i ,h i ) are the center coordinates and length and width values of the real defect, are the center coordinates and length and width values of the predicted defect target.
[0071] S56, adjusting the parameters of the fabric defect detection network according to the learning rate, weight decay and the difference value in step S55, until all images in the training set are traversed.
[0072] S57, the number of iterations is increased by 1, and steps S52 to S56 are executed in a loop until the maximum number of iterations is reached, the parameters of the fabric defect detection network of the last iteration are retained, and a fabric defect detection network model is generated.
[0073] S6. Input the fabric image in the test set into the fabric defect detection model obtained in step S5, obtain the target defect and position coordinates in the image, and frame the location of the defect in the image. Input the fabric image in the test set into the fabric defect detection model, and after the image is subjected to layer-by-layer convolution operations of the network model, obtain the confidence that the pixels in each area belong to the fabric defect target, and predict the position coordinates of the defect target in the image. By setting a confidence threshold, the target greater than the threshold is determined as a defect target, and the position of the defect in the image is accurately framed according to the coordinate value.
[0074] The defect detection performance indicators mainly include accuracy and running speed (real-time), and the hardware resources used are also listed. The defect detection indicators used are given below.
[0075] 1) Confusion Matrix
[0076] Table 1 Confusion matrix
[0077] Tab.2 Confusion matrix
[0078]
[0079] TP, TN, FP, and FN are the abbreviations of true positive, true negative, false positive, and false negative. Positive and negative represent the predicted results. If the prediction is positive, it is positive, and if the prediction is negative, it is negative. True and false represent whether the predicted result is the same as the true result. If they are the same, it is true, and if they are different, it is false.
[0080] 2) Precision
[0081] That is, the model judges that the image is positive, and the probability that the image is indeed positive. The accuracy of the positive class refers to the number of images that the model judges to be positive and the true class is also positive / the number of images judged by the model to be positive. It measures the probability that a positive class classified by a classifier is indeed positive. The calculation formula is as follows:
[0082]
[0083] 3) Recall
[0084] That is, if there are 5 positive classes in all the images, and the model can correctly judge these 5 images as positive, then the recall rate is 100%. The recall rate refers to the number of images that the model judges as positive and the true class is also positive / the number of images that the true class is positive. It measures the ability of a classifier to find all positive classes. The calculation formula is:
[0085]
[0086] 4) AP and mAP
[0087] Calculate the Precision and Recall of each prediction result and draw a curve according to the relationship, which is the PR curve. Before calculating AP, in order to smooth the PR curve and reduce the impact of curve jitter, first interpolate the PR curve. By calculating the average value of the Precision value corresponding to each recall value, you can get a numerical evaluation indicator: AP, which is used to measure the detection ability of the trained model on the category of interest.
[0088] AP measures the quality of detection of one class, while mAP measures the quality of detection of multiple classes. For example, if there are two classes, the AP value of class A is 0.5 and the AP value of class B is 0.2, then mAP = (0.5 + 0.2) / 2 = 0.35.
[0089] The images in the test set are input into the fabric defect detection model to obtain the detection results of the model. Compared with other neural networks, the results show that good results have been achieved on the TILDA dataset, dark red fabric images and grid fabric images. The detection results of the fabric defect detection model are as follows: Figure 4 As shown in Table 2, the fabric defect detection network can not only accurately detect the location of the defect, but also has fewer model parameters and faster detection speed. The performance of the model is compared with other mainstream neural network models, and the results are shown in Table 2.
[0090] Table 2 Performance comparison of network models with other mainstream neural network models
[0091]
[0092] Experimental results show that the proposed model can achieve an accuracy of more than 97% on the TILDA dataset, dark red fabric images, and grid fabric images, and can accurately detect common defects in fabrics, such as broken yarns, cotton knots, fuzz balls, holes, scratches, and stains. Moreover, while obtaining accurate detection results, the size of the model is reduced.
[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion, characterized in that: The following steps are involved: Step 1, collect fabric images and preprocess them to obtain a data set, and divide the data set into a training set and a test set in proportion; Collect fabric images with defects, enhance the training data by random cropping and rotation, and mark the coordinates of the actual defects on the images; Step 2: Input the training samples in the training set in step 1 into the fabric defect detection network for training. The fabric defect detection network extracts the features of the image through a hybrid dilated convolution module. Four parallel paths are designed in the hybrid dilated convolution module, three of which are used to extract features of different scales for detecting fabric defects of different sizes, and the other path is used to obtain nonlinear features of defects at different positions. The fabric defect detection network includes a feature extraction module, a region proposal module, and a classification and regression module. The feature extraction module includes an image shallow processing module and a mixed dilated convolution module processing module. The image shallow processing module includes four convolution layers and two pooling layers, which are used to capture the texture and edge features of the fabric and reduce the size of the feature map. The mixed dilated convolution module processing module includes nine mixed dilated convolution modules, which are used to obtain multi-scale features while reducing the number of parameters. Step 3: Use the optimal feature fusion method to perform feature fusion, and input the obtained features into the region proposal module to obtain the defect suggestion box; The candidate frame setting in step 3 includes 3 pixel ratios and 3 aspect ratios, which are combined to obtain 9 different selection methods. The pixel ratio is the ratio of the number of pixels in the candidate frame to the number of pixels in the complete image, and the aspect ratio is the ratio of the width to the length of the candidate frame. The area calculation formula of the candidate frame is: S i =(λ i ·B) 2 ,i=1,2,3 The calculation formula for the length and width of the candidate box is: Among them, S i represents the area of the candidate box, λ i Represents the three ratios of the candidate box, B represents the size of each pixel on the feature map corresponding to the original image, R j Indicates the three aspect ratios of the candidate box, H i,j Indicates the width of the candidate box, W i,j Indicates the length of the candidate box; Step 4: The classification and regression module combines the features obtained in step 2 with the defect suggestion box obtained in step 3 to perform regression of the final defect position and classification of the defect type, and obtain the final defect detection result of each training sample; Step 5: Using the asynchronous gradient descent training strategy to adjust the parameters of the fabric defect detection network to obtain a fabric defect detection model; Step 6: Input the test set in step 1 into the fabric defect detection network model to obtain the target defects and position coordinates in the image, and use the candidate box to frame the defect target in the image.
2. The fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion according to claim 1, characterized in that: The training method of the fabric defect detection model is as follows: Step 21, set the learning rate θ and weight decay ω, and set the maximum number of iterations t_max; Step 22, inputting the training samples of the training set images into the fabric defect detection network in sequence; Step 23, dividing the feature map extracted by the fabric defect detection network into a plurality of grids of the same size, wherein each grid predicts B bounding boxes, where B is a constant; Step 24: Use the logistic classifier to predict the confidence value of the defect target in each border, and give the center coordinates and length and width values of each border; Step 25, calculating the difference between the confidence value of the defect target predicted in step 24 and the position coordinates of the actual defect on the image in step 22 by using the loss function; Step 26, adjusting the parameters of the fabric defect detection network according to the learning rate θ, the weight decay ω and the difference value in step 25, returning to step 22, until all the training samples in the training set are traversed, and then going to the next step; Step 27: The number of iterations increases by 1, and it is determined whether the maximum number of iterations t is reached. max If not, go to step 22. If yes, save the parameters of the last iteration and generate a fabric defect detection network model.
3. The fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion according to claim 2, characterized in that: The loss function in step 25 is: Loss = Error iou +Error cls +Error bbl , where Error iou Represents the confidence loss function, Error cls is the logistic classifier loss function, Error bbl is the bounding box loss function; The confidence loss function Error iou for: Among them, S k represents the kth network, k = 1, 2, ..., i = 0, 1 ..., j = 0, 1 ..., B represents the jth bounding box, Or 1 indicates whether there is a fabric defect target in the jth bounding box in the ith network. If it exists, it is 1, and if it does not exist, it is 0. nofbd is the weight of the confidence loss of the target without defects, C i is the confidence score, is the intersection ratio of the predicted bounding box to the true bounding box: The logistic classifier loss function is: Among them, p i (c) is the target true category score, Score the predicted target class; The bounding box loss function Error bbl for: Among them, λ bbl is the weight of the coordinate error of the defect, (x i ,y i ,ω i ,h i ) are the center coordinates and length and width values of the real defect, are the center coordinates and length and width values of the predicted defect target.
4. The fabric defect detection method based on hybrid dilated convolution and multi-scale feature fusion according to claim 1, characterized in that: The optimal feature fusion method in step 3 is to perform training by using multiple fusion methods, compare the feedback effects obtained from the training, and screen out the optimal fusion method.
