Fabric defect detection method and system based on ResNet
By adopting ResNet-based detection methods in fabric defect detection, combined with Partial Convolution and CA module, the problems of low efficiency and poor accuracy of traditional detection methods are solved, and efficient and accurate fabric defect detection and automated sorting are achieved.
Patent Information
- Application Number
- CN202411951331.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional fabric defect detection methods have problems such as low detection efficiency, high cost, missed detection rate and error rate. Traditional machine learning algorithms perform poorly when processing high-dimensional data, making it difficult to be competent for large-scale image classification tasks.
Using ResNet-based fabric defect detection method, the camera captures fabric defect images, and through preprocessing and feature extraction, a ResNet-50 model integrating Partial Convolution and CA module is constructed to perform end-to-end learning, automatically learn feature representations from the original image, and sorting operations are performed through the robotic arm.
It improves the speed and accuracy of fabric defect detection, enhances the robustness of the identification system, realizes automated fabric defect sorting, reduces labor costs, and adapts to different scenarios and needs.
Smart Images
Figure CN119863686A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to a fabric defect detection method and system based on ResNet. Background Art
[0002] Traditional fabric defect detection uses manual observation, which has many disadvantages such as low detection efficiency, high cost, high missed detection rate and error rate, and cannot meet the requirements of the textile industry. Therefore, proposing an effective fabric defect recognition method has become the key to fabric detection. At present, the common fabric defect detection methods are manual classification and machine learning methods; manual classification requires a lot of manpower and equipment, which is time-consuming and costly; in the case of long-term continuous operation, people will make mistakes when they are tired, resulting in a decrease in the accuracy of fabric defect detection; using traditional machine learning algorithms, such as support vector machines, random forests, etc., to extract features from data and classify fabric defects by training classifiers. This method requires manual design of feature engineering, and different feature representation methods need to be used for different types of fabrics. At the same time, traditional machine learning algorithms perform poorly when processing high-dimensional data and are difficult to handle large-scale image classification tasks. Summary of the invention
[0003] The present invention provides a fabric defect detection method and system based on ResNet, which improves the fabric defect detection speed, improves the recognition accuracy, enhances the robustness of the recognition system, and provides a more efficient recognition method for an automated fabric defect sorting device.
[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0005] A cloth defect detection method based on ResNet includes the following steps:
[0006] Use a camera to capture fabric defect images and build a data set;
[0007] Preprocess the images in the dataset and divide the dataset into training set and test set according to a given ratio;
[0008] The convolutional neural network of ResNet-50 integrated with Partial Convolution is selected as the pre-training model, and the CA module is introduced into the pre-training model to build a fabric defect detection model;
[0009] Input the data set into the fabric defect detection model for training to obtain a trained detection model;
[0010] The model is exported, and the location and type information of fabric defects are obtained through image processing. The defective fabrics are then sorted by controlling the robotic arm.
[0011] Preferably, the ratio of the number of training sets to the number of test sets in the data set is 8:2.
[0012] Preferably, preprocessing the images in the data set includes data labeling, data analysis, image scaling and cropping, image data standardization, and data augmentation, wherein data labeling refers to classifying and numbering each image in the data set; data analysis refers to removing images in the fabric data set that have uneven overall data and poor quality due to differences in image type, length and width, and different shooting environments; image data standardization refers to normalizing the image to adjust the eigenvalues of different dimensions to a similar range; data augmentation refers to expanding the data set to increase the number of images in the data set.
[0013] Preferably, the ResNet-50 convolutional neural network consists of a convolutional layer, multiple layer modules, an average pooling layer, a maximum pooling layer and a softmax fully connected layer, and the layer module includes multiple bottlenecks. The key to integrating the Resnet-50 convolutional neural network with Partial Convolution is to replace the two 1×1 Convolutions in the original bottleneck with Partial Convolution.
[0014] Preferably, the CA module is introduced into the pre-trained model, and the construction of the fabric defect detection model includes the following steps:
[0015] Input feature arbitrary tensor size is X = [x1, x2..., x c , …, x h , …, x w ], respectively decompose the global average pooling, where c, h, and w are the number of channels, height, and width of the feature map, respectively;
[0016] For the input feature size c×h×w, the features are pooled in the X and Y directions respectively, generating z in the horizontal and vertical directions. h and z w The feature maps of size c×h×1 and c×1×w, where the calculation formulas for X and Y are as follows:
[0017] The generated feature map is concatenated with two feature maps with specific direction information in the third dimension, and then convolution is used to reduce the dimension. The nonlinear activation function generates the intermediate feature vector f = R (c / r)×1×(h+w) Contains horizontal and spatial vertical information, where r is the downsampling ratio during the channel process;
[0018] Split the feature map f along the spatial dimension to generate two feature vectors f h =R (c / r)×h×1 and f w =R (c / r)×1×w , and then use 1×1 convolution to increase the dimension, and then combine the activation function Sigmoid to get the attention g h =R c×h×1 and g w =R c×1×w ;
[0019] In the Resnet-50 network structure integrated with Partial Convolution, a CA module is added to the first convolutional layer to build a fabric defect detection model based on Resnet-50.
[0020] Preferably, the ResNet-based fabric defect detection method further includes the step of testing the fabric defect detection model using a training set and a test set, including the following steps:
[0021] Evaluate the accuracy Train_Accuracy and loss Train_Loss of the training set, and the accuracy Val_Accuracy and loss Val_Loss of the test set of the fabric defect detection model. The accuracy is: TP represents the number of correctly classified as positive examples, that is, the number of instances that are actually positive examples and are classified as positive examples by the classifier; TN represents the number of correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the classifier; FP represents the number of incorrectly classified as positive examples, that is, the number of instances that are actually negative examples but are classified as positive examples by the classifier; FN represents the number of incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but are classified as negative examples by the classifier. Among them, the loss value is: N is the total number of samples, i represents one of the output samples, y is the actual value, and x is the predicted value.
[0022] A system for fabric defect detection using the aforementioned ResNet-based fabric defect detection method comprises:
[0023] An image preprocessing module, used to capture images and perform preprocessing operations;
[0024] An image recognition module, the image recognition module comprising a data set establishment unit, a fabric defect detection unit, a model training unit and an image detection unit;
[0025] A fabric defect sorting module drives a sorting device to perform a fabric defect sorting operation based on the image recognition result of the image recognition module; and
[0026] The sorting device is used to perform sorting operations.
[0027] Preferably, the sorting device is a robotic arm.
[0028] It can be seen from the above technical solutions that the present invention has the following beneficial effects:
[0029] 1. In the present invention, the model used is an end-to-end learning model, which can automatically learn feature representation from the original image without relying on manual feature engineering, reducing the workload and improving the defect detection performance. In addition, the model in the present invention can enhance the feature extraction and expression capabilities, and has strong robustness to interference such as noise and occlusion, and has the following main advantages:
[0030] 1) Automation and efficiency: The deep learning model combined with the robotic arm can check whether the fabric has defects without human intervention. Compared with manual sorting, automated processing is more efficient and can process large amounts of fabric quickly and continuously.
[0031] 2) Accuracy and consistency: The deep learning model has high accuracy in fabric defect tasks, is not affected by factors such as fatigue and emotions, and maintains consistency and stability. Compared with manual sorting, it reduces classification errors caused by human factors and improves the accuracy of fabric defect identification.
[0032] 3) Reduce labor costs: The fabric defect detection system can significantly reduce labor input and save labor costs. The robotic arm can work continuously without rest and replenishment of manpower, thus reducing operating costs.
[0033] 4) Highly customizable: The fabric defect detection system can be customized and adjusted according to actual needs. The parameters of the deep learning model and the robotic arm can be optimized according to different fabric classification tasks, making the system more adaptable to different scenarios and needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A block diagram of a system for detecting fabric defects provided by the present invention;
[0035] Figure 2 A flow chart of the cloth defect detection method provided by the present invention;
[0036] Figure 3 This is a comparison chart between the bottleneck that integrates Partial Convolution and the traditional bottleneck;
[0037] Figure 4 This is the flowchart of the CA attention mechanism algorithm;
[0038] Figure 5A flowchart for building a fabric defect detection model based on Resnet-50;
[0039] In the figure: 10, image preprocessing module; 20, image recognition module; 210, data set establishment unit; 220, fabric defect detection unit; 230, model training unit; 240, image detection unit; 30, fabric defect sorting module. DETAILED DESCRIPTION
[0040] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.
[0041] To achieve the above purpose, the embodiment of the present invention adopts the following technical solution: Figure 2 , a cloth defect detection method based on ResNet, comprising the following steps:
[0042] S10: Use a camera to capture fabric defect images and establish a data set.
[0043] S20: Preprocess the images in the dataset and divide the dataset into a training set and a test set according to a given ratio.
[0044] S30: Select the convolutional neural network of ResNet-50 integrated with Partial Convolution as the pre-trained model, and introduce the CA module into the pre-trained model to build a fabric defect detection model; it should be noted that the CA module is the Coordinate attention mechanism.
[0045] S40: Input the data set into the fabric defect detection model for training to obtain a trained detection model.
[0046] S50: Export the model, obtain the location and type information of fabric defects through image processing, and control the robotic arm to sort the defective fabric to achieve sorting tasks for different defects. The present invention integrates Partial Convolution in the Resnet-50 network to repair the occluded area information, suppress the influence of occlusion on the gradient, improve the robustness of the model, accelerate the convergence speed, and simplify the post-processing process. At the same time, the added CA module (i.e., Coordinate attention mechanism) can capture cross-channel information as well as direction perception and position perception information, which can help the model locate and identify targets of interest more accurately, and enhance features by strengthening information representation.
[0047] It should be noted that this embodiment uses a PC as the image recognition processing end, and is configured with an i7-13700KF CPU, a 64-bit Windows 10 operating system, and an RTX4060 GPU;
[0048] The camera is a camera developed by Stereolabs, model ZED Mini, as the camera of the present invention, with a binocular depth resolution of 1920x1080 and a binocular detection range of 1m to 15m;
[0049] The ABB YuMi robot arm of AbbVie's ABB series robot is used as the sorting device of the present invention, with a payload of 5kg. The fabric defect detection method based on Resnet-50 is deployed on the ABB YuMi robot arm of AbbVie's ABB series robot, which has the advantages of automation, high efficiency, accuracy, cost reduction, adaptability to diversified fabrics, high customization and high safety. Such a system is expected to play a greater role in fabric defect processing, improve the efficiency and intelligence level of fabric detection and processing, and promote the development of the industry.
[0050] Reference Figure 2 As a preferred technical solution of this embodiment, the ratio of the number of training sets and test sets in the data set is 8:2.
[0051] Furthermore, fabric defect images are captured by a camera and the images are preprocessed. Specifically, the preprocessing of images in the data set includes data labeling, data analysis, image scaling and cropping, image data standardization, and data augmentation. Among them, data labeling refers to the classification and numbering of each image in the data set; data analysis refers to removing images in the fabric data set that have uneven overall data and poor quality due to differences in image type, length and width, and different shooting environments; image data standardization refers to normalizing the image to adjust the eigenvalues of different dimensions to a similar range; data augmentation refers to expanding the data set to increase the number of images in the data set.
[0052] Reference Figure 3In some embodiments, the steps of constructing and training a convolutional neural network based on Resnet-50 are as follows: The Resnet-50 convolutional neural network consists of a convolutional layer, multiple layer modules, an average pooling layer, a maximum pooling layer, and a softmax fully connected layer. The layer module contains multiple bottlenecks. The key to the Resnet-50 convolutional neural network that integrates Partial Convolution is to replace the two 1×1 Convolutions in the original bottleneck with Partial Convolution. Taking the bottleneck in layer1 as an example, the specific comparison structure is as follows Figure 3 ; PartialConvolution greatly improves the computational efficiency by adding a mask to the convolution operation, and distinguishes pixels in damaged and non-damaged areas, thereby improving its sensitivity.
[0053] In order to make full use of channel attention information and spatial attention information, improve the precision and accuracy of model recognition, make the model pay more attention to important features and suppress useless information. The present invention adds a Coordinate attention mechanism, namely, a CA module. Specifically, first, for any input feature tensor size X = [x1, x2…, x c , …x h , …x w ], where c, h, and w are the number of channels, height, and width of the feature map, respectively, and are used to decompose the global average pooling, so that it can effectively capture the long-range interactive features of precise location information and avoid the spatial information being compressed into the channel; then, for the features of the input feature size c×h×w, pooling is performed in the X and Y directions respectively, generating z in the horizontal and vertical directions h and z w The feature maps of size c×h×1 and c×1×w, where the calculation formulas for X and Y are as follows: Next, the generated feature map is concatenated with two feature maps with specific direction information in the third dimension, and then convolution is used for dimensionality reduction. The nonlinear activation function generates an intermediate feature vector f = R (c / r)×1×(h+w) Contains horizontal and spatial vertical information, where r is the downsampling ratio in the channel process; finally, the feature map f is split along the spatial dimension to generate two feature vectors f h =R (c / r)×h×1 and f w =R (c / r)×1×w , and then use 1×1 convolution to increase the dimension, and then combine the activation function Sigmoid to get the attention g h =Rc×h×1 and g w =R c×1×w Specific structure such as Figure 4 .
[0054] In the Resnet-50 network structure fused with Partial Convolution, a CA module is added to the first convolution layer to build a fabric defect detection model based on Resnet-50. The specific structure is as follows: Figure 5 ;
[0055] The above model is tested using the training set and the test set to evaluate the model's accuracy Train_Accuracy and loss Train_Loss on the training set, and the accuracy Val_Accuracy and loss Val_Loss on the test set. The accuracy is: TP represents the number of correctly classified as positive examples, that is, the number of instances that are actually positive examples and are classified as positive examples by the classifier; TN represents the number of correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the classifier; FP represents the number of incorrectly classified as positive examples, that is, the number of instances that are actually negative examples but are classified as positive examples by the classifier; FN represents the number of incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but are classified as negative examples by the classifier. The loss value is: N is the total number of samples, i represents one of the output samples, y is the actual value, and x is the predicted value.
[0056] The present invention also provides a system for fabric defect detection using the aforementioned ResNet-based fabric defect detection method, comprising an image preprocessing module 10, an image recognition module 20, a fabric defect sorting module 30 and a sorting device, wherein the image preprocessing module 10 is used to capture images and perform preprocessing operations, and the image recognition module 20 includes a data set establishment unit 210, a fabric defect detection unit 220, a model training unit 230 and an image detection unit 240; the fabric defect sorting module 30 can drive the sorting device to perform a material defect sorting operation based on the image recognition result of the image recognition module, and the sorting device is used to perform the sorting operation.
[0057] The above-described embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.
Claims
1. A cloth defect detection method based on ResNet, characterized in that: The following steps are involved: Use a camera to capture fabric defect images and build a data set; Preprocess the images in the dataset and divide the dataset into training set and test set according to a given ratio; The convolutional neural network of ResNet-50 integrated with PartialConvolution is selected as the pre-training model, and the CA module is introduced into the pre-training model to build a fabric defect detection model; Input the data set into the fabric defect detection model for training to obtain a trained detection model; The model is exported, and the location and type information of fabric defects are obtained through image processing. The defective fabrics are then sorted by controlling the robotic arm.
2. The fabric defect detection method based on ResNet according to claim 1, characterized in that: The ratio of the number of training sets and test sets in the dataset is 8:
2.
3. The fabric defect detection method based on ResNet according to claim 1, characterized in that: The preprocessing of images in the dataset includes data labeling, data analysis, image scaling and cropping, image data standardization, and data augmentation. Data labeling refers to the classification and numbering of each image in the dataset; data analysis refers to the removal of images in the fabric dataset that have uneven overall data and poor quality due to differences in image type, length and width, and shooting environments; image data standardization refers to the normalization of images to adjust the feature values of different dimensions to a similar range; data augmentation refers to the expansion of the dataset to increase the number of images in the dataset.
4. The fabric defect detection method based on ResNet according to claim 1, characterized in that: The ResNet-50 convolutional neural network consists of a convolutional layer, multiple layer modules, an average pooling layer, a maximum pooling layer and a softmax fully connected layer. The layer module includes multiple bottlenecks. The key to integrating the Resnet-50 convolutional neural network with Partial Convolution is to replace the two 1×1 Convolutions in the original bottleneck with PartialConvolution.
5. The fabric defect detection method based on ResNet according to claim 4, characterized in that: Introduce the CA module into the pre-trained model to build a fabric defect detection model The following steps are involved: Input feature arbitrary tensor size is X = [x1, x2..., x c , …x h , …x w ], respectively decompose the global average pooling, where c, h, and w are the number of channels, height, and width of the feature map, respectively; For the input feature size c×h×w, the features are pooled in the X and Y directions respectively, generating z in the horizontal and vertical directions. h and z w The feature maps of size c×h×1 and c×1×w, where the calculation formulas for X and Y are as follows: The generated feature map is concatenated with two feature maps with specific direction information in the third dimension, and then convolution is used to reduce the dimension. The nonlinear activation function generates the intermediate feature vector f = R (c / r)×1×(h+w) Contains horizontal and spatial vertical information, where r is the downsampling ratio during the channel process; Split the feature map f along the spatial dimension to generate two feature vectors f h =R (c / r)×h×1 and f w =R (c / r)×1×w , and then use 1×1 convolution to increase the dimension, and then combine the activation function Sigmoid to get the attention g h =R c×h×1 and g w =R c×1×w ; In the Resnet-50 network structure integrated with PartialConvolution, a CA module is added to the first convolutional layer to build a fabric defect detection model based on Resnet-50.
6. The fabric defect detection method based on ResNet according to claim 5, characterized in that: The ResNet-based fabric defect detection method further includes the step of testing the fabric defect detection model using a training set and a test set, including the following steps: Evaluate the accuracy Train_Accuracy and loss Train_Loss of the training set, and the accuracy Val_Accuracy and loss Val_Loss of the test set of the fabric defect detection model. The accuracy is: TP represents the number of correctly classified as positive examples, that is, the number of instances that are actually positive examples and are classified as positive examples by the classifier; TN represents the number of correctly classified as negative examples, that is, the number of instances that are actually negative examples and are classified as negative examples by the classifier; FP represents the number of incorrectly classified as positive examples, that is, the number of instances that are actually negative examples but are classified as positive examples by the classifier; FN represents the number of incorrectly classified as negative examples, that is, the number of instances that are actually positive examples but are classified as negative examples by the classifier. Among them, the loss value is: N is the total number of samples, i represents one of the output samples, y is the actual value, and x is the predicted value.
7. A system for fabric defect detection using the ResNet-based fabric defect detection method according to any one of claims 1 to 6, characterized in that: include: An image preprocessing module, used to capture images and perform preprocessing operations; An image recognition module, the image recognition module comprising a data set establishment unit, a fabric defect detection unit, a model training unit and an image detection unit; A fabric defect sorting module drives a sorting device to perform a fabric defect sorting operation based on the image recognition result of the image recognition module; as well as The sorting device is used to perform sorting operations.
8. The cloth defect detection system according to claim 7, characterized in that: The sorting device is a robotic arm.