Fabric defect image classification method based on deep learning

By introducing an attention mechanism and a robust optimization algorithm into the ResNet50 network, the problems of poor feature extraction and Softmax classifier in existing fabric defect detection are solved, and efficient and accurate fabric defect classification is achieved.

CN119048802APending Publication Date: 2024-11-29ZHEJIANG SCI-TECH UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411037825.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing fabric defect detection methods suffer from problems such as a lack of attention mechanisms leading to poor feature extraction, shallow depth networks, high computational complexity of Softmax classifiers, overfitting, sensitivity to outliers, and imbalanced classification, all of which affect model performance and effectiveness.

Method used

We employ a ResNet50 network with a CBAM (Concentration-Based Attention) module, combine it with a Robust Optimization (ROA) algorithm to optimize model parameters, replace fully connected layers with a width learning approach, and combine it with a BLS classifier to improve feature extraction and classification accuracy.

Benefits of technology

The model improves the feature extraction effect and classification accuracy of fabric defect detection. It has good stability and robustness and is suitable for automated fabric defect detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048802A_ABST
    Figure CN119048802A_ABST
Patent Text Reader

Abstract

The invention discloses a fabric defect image classification method based on deep learning, and the method comprises the steps: taking a pre-trained ResNet50 as a backbone training network, and adding a mixed attention (CBAM) module of a space channel in the network to capture an image; preprocessing the data set; in the back propagation process, an ROA algorithm is used for replacing an original optimizer to adjust model parameters, so that the model is converged; replacing the last full connection layer of the network with the BLS to serve as classifier output; and outputting an image classification result. According to the invention, the influence of low manual detection efficiency can be eliminated, and technical support is provided for automatic textile defect detection. The model has excellent prediction precision and good convergence and stability, and the classification performance of a textile defect image algorithm is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to a method for classifying fabric defect images based on deep learning. Background Art

[0002] As various new fabric models and various attractive patterns are added to textiles, the textile industry is facing a major quality test, and textile defect detection is an important part of promoting the high-quality development of the textile industry. Considering that a defective fabric will drop its sales price by 45-65% if it has defects, textile defect detection is an important part of many textile manufacturing companies to improve product quality and increase economic benefits. There are more than 70 types of fabric defects defined by the textile industry. However, in most textile manufacturing companies in the world, the quality inspection process still relies mainly on human vision. Due to the physiological and psychological limitations of human eyestrain, slow visual scanning speed, etc., manual inspection efficiency is low, and even well-trained workers have a success rate of only 60-75%. With the continuous improvement of the high-quality development requirements of the textile industry, automated textile defect detection technology is indispensable.

[0003] In the past decade, with the rapid popularization of deep learning, it has been widely used in many fields such as image detection and object recognition. This also provides a new idea for the research on textile defect classification. Many scholars have proposed the use of deep learning methods to realize the automatic detection of fabric defects. Castilho et al. combined neural network (NN), fuzzy modeling (FM) based on product space fuzzy clustering, and fuzzy inference system (ANFIS) based on adaptive network to accurately classify fabric defects. In order to achieve efficient classification, the thresholding technology was used to process the output of these three intelligent technologies, and the optimal threshold was determined by the optimization method based on the confusion matrix. This method not only has a high classification accuracy, but also has a fast detection speed and good adaptability, providing an efficient and reliable quality control method for the textile industry. Zhang et al. proposed an improved Elman network to classify fabric defects with proportional (P), integral (I) and derivative (D) properties. This method uses a two-dimensional optimal Gabor filter method to process the image. The solution of parameter optimization is based on a niche genetic algorithm. After image fusion and binarization, the binarized image is obtained by threshold processing, from which four feature variables are collected and input into the three-layer modified PID Elman classifier for recognition. Experimental results show that the method proposed by Zhang not only has a high recognition accuracy, but also has strong performance, which can well meet the needs of today's textile production factories for defect detection and classification. Liu et al. proposed a defect detection algorithm based on end-to-end convolutional neural network. First, a series of potential defect area proposals are automatically generated using the region proposal network (RPN). Subsequently, these proposal areas are passed to the FastR-CNN model to accurately determine whether each area actually contains fabric defects. In order to further improve the accuracy of detection, the soft non-maximum suppression (soft-NMS) technology is introduced, which can effectively handle overlapping defect areas and avoid missed detection and false detection. Experimental results show that the method of Liu et al. can more accurately locate fabric defect areas and show good adaptability on various fabric images. Jing et al. proposed a convolutional neural network (CNN) based on improved AlexNet for color-dyed fabric defect classification. The local response normalization layer is replaced by the batch normalization layer. During the network training process, defect features are gradually extracted, and edge details are fused through multiple convolution operations to capture the essential features of the image. The maximum pooling layer, dropout layer, and fully connected layer are introduced into the classification model. These layers can effectively reduce the number of parameters, prevent overfitting, and improve the generalization ability of the model. Finally, the softmax function is used to predict the defect classification results to ensure the accuracy and reliability of the classification results. Biradar et al. proposed a fabric defect detection method based on deep convolutional neural network (DCNN) to process fabrics with complex patterns and no patterns.The proposed method effectively extracts and represents the features of normal and defective fabric patches through a three-layer DCNN structure. The defect detection accuracy of patterned and unpatterned fabrics in the internal database reached 98.33% and 90.39%, respectively. Sabeenian et al. designed a deep learning framework to identify various fabric types and classify their defects using artificial intelligence technology. The framework achieves efficient preprocessing and accurate classification of fabric images by integrating pseudo-convolutional neural networks (P-CNN) and carefully designed convolutional neural networks (CNN). The algorithm is also able to filter nonlinear mixed noise in fabric datasets using PCNN. The improved convolutional neural network classifies various fabric defects from a database consisting of defect and non-defect samples. Compared with the traditional VGG network, the proposed method by Sabeenian et al. performs well in reducing computational complexity and improving computational efficiency. Through quantitative and qualitative evaluations, we verify that the proposed architecture is comparable to other existing methods in performance and shows high accuracy in tests on different fabric types. Alruwais et al. proposed a deep learning-based intelligent fabric defect detection technology (HMFODL-FDD), which combines the advantages of computer vision (CV) and deep learning (DL) to achieve automatic detection of fabric defects in sustainable manufacturing processes. For feature extraction, the HMFODL-FDD technology uses the Inception v3 model and a hyperparameter optimizer based on the HMFO algorithm. Finally, the back propagation neural network (BPNN) model is used to classify fabric defects. Experimental results confirm that the HMFODL-FDD technology has achieved better performance than DL technology in effective defect classification of fabric images. Jun et al. proposed a learning-based automatic fabric defect detection framework, which uses a fixed-size square slider to crop the original image to a certain step size and rule, and then uses improved histogram equalization to enhance each cropped image and uses the Inception-V1 model to predict whether there is a defect in the local area. The experimental results show that this method has excellent performance in fabric defect detection.

[0004] Fabric defects are of various types and in various forms, ranging from minor yarn breaks to obvious stains and holes. These defects may appear anywhere on the fabric in different shapes, sizes and colors. In addition, the texture background of the fabric is often complex and changeable, which increases the difficulty of automatic detection and classification. However, the above methods use shallow layers of deep learning networks in fabric defect detection and lack the use of attention mechanism modules. The core of the attention mechanism is that it can simulate the attention allocation process of the human visual system, that is, when processing a large amount of information, it can automatically and selectively focus on key information and ignore unimportant details. By calculating the importance of each area to the classification results, the attention mechanism can guide the model to pay more attention to those areas containing defects, thereby ignoring insignificant background textures and reducing the interference of background noise.

[0005] At the same time, in the fully connected layer stage, the Softmax function is often simply used for classification, which can easily lead to sensitivity to class imbalance, overconfidence in output probability, high computational complexity, and sensitivity to outliers, which affect the performance and effect of the model in practical applications. Therefore, existing models often face problems such as poor feature extraction and low classification accuracy. Summary of the invention

[0006] The purpose of the present invention is to address the problems of existing fabric defect detection, lack of attention mechanism, shallow number of deep network layers resulting in poor feature extraction effect; Softmax classifier has high computational complexity, easy overfitting, sensitivity to outliers, classification imbalance, etc., and propose an image classification algorithm based on deep learning network, which uses ResNet50 network to extract features of textile defects, adds an attention mechanism module (CBAM) to the network, and uses ROA algorithm to modify the optimizer in the hope of obtaining better gradient descent, and then applies the width learning idea to replace the fully connected layer of the ResNet50 network, combining the advantages of BLS classifier in speed and accuracy and the advantages of ResNet50 in feature extraction performance, to achieve the purpose of efficient classification.

[0007] The present invention provides 1. a method for classifying fabric defects in an image based on deep learning, characterized in that the method comprises the following steps:

[0008] Step 1, obtain images and build a dataset:

[0009] The images include images of normal fabrics and fabrics with defects; each image in the data set includes an image label, and the image label is a defect type;

[0010] Step 2: construct an image fabric defect classification model, wherein the image fabric defect classification model uses ResNet50 as the backbone network, and the structure of the image fabric defect classification model includes, from input to output: input layer Input, Stage0 layer, CBAM1 layer, Stage 1 layer, Stage 2 layer, Stage 3 layer, Stage 4 layer, CBAM2 layer, global average pooling layer and BLS layer;

[0011] The input layer Input is the image obtained in step 1; both the CBAM1 layer and the CBAM2 layer include: a channel attention module CAM and a spatial attention module SAM ; Use the global average pooling layer to perform global average pooling on the output of the CBAM2 layer ;

[0012] The feature map output by the global average pooling layer is expanded into a one-dimensional vector and passed into the BLS layer. After passing through the feature layer and the enhancement layer, it is mapped to the dimension of the number of categories for classification tasks.

[0013] Step 3: Use the robust optimization algorithm ROA as the optimizer to optimize the fabric defect classification model and obtain the trained Resnet50-CBAM-ROA-BLS;

[0014] Step 4: Model classification prediction

[0015] Get the image to be classified, pass it into the trained Resnet50-CBAM-ROA-BLS model for prediction, and output the classification result.

[0016] Preferably, in the fabric defect classification model, the Stage0 layer includes: a convolutional layer and a BN layer; a ReLU activation function is used to perform nonlinear activation on the output of the BN layer; and a maximum pooling layer is used to further reduce the size of the feature map.

[0017] The CBAM1 layer includes: a channel attention module CAM and a spatial attention module SAM, which obtain the output feature map of Stage 0, recalibrate the input feature map, and enhance the model's attention on key features.

[0018] Stage 1 layer includes: 3 residual blocks. Each residual block consists of several convolutional layers and a cross-layer connection. Inside each residual block, the number of channels is first reduced by a 1x1 convolutional layer, then feature extraction is performed by a 3x3 convolutional layer, and finally the number of channels is restored by a 1x1 convolutional layer. The cross-layer connection is used to add the input of the residual block directly to the output to avoid the gradient vanishing problem.

[0019] The Stage 2 layer includes: 4 residual blocks. The residual block structure is similar to that of Stage 1, but the number of input and output channels of each residual block increases.

[0020] The Stage 3 layer includes: 6 residual blocks, which continue to increase the number of channels and reduce the size of the feature map.

[0021] Stage 4 layer includes: 3 residual blocks, which is the last stage to further extract high-level features.

[0022] The CBAM2 layer includes: channel attention module CAM and spatial attention module SAM, which obtain the output feature map of Stage 4, recalibrate the input feature map, and enhance the model's attention on key features.

[0023] The output of the CBAM2 layer is globally average pooled using a global average pooling layer to reduce the size of the feature map in order to reduce the number of parameters and avoid overfitting.

[0024] The feature map output by the global average pooling layer is expanded into a one-dimensional vector and passed into the BLS layer. After passing through the feature layer and the enhancement layer, it is mapped to the dimension of the number of categories for classification tasks.

[0025] Preferably, the training process of the image fabric defect classification model comprises the following steps:

[0026] The robust optimization algorithm ROA is used as an optimizer, and the image fabric defect classification model is trained using the data set obtained in step 1 until the image fabric defect classification model converges.

[0027] Preferably, in step 1, the data set is enhanced by rotation, Gaussian noise, contrast enhancement, image flipping, and brightness enhancement, and is divided into a training set and a test set in a ratio of 8:2. Increasing the diversity of training samples helps reduce the overfitting of the model on the training data, enables the model to perform well on unseen data, and improves the generalization ability, robustness, and recognition accuracy of the model.

[0028] Preferably, in step 2, the learning rate of the ResNet50 backbone network is 1*10 -4 , the number of Epochs for training is 20, and BATCH_SIZE is 16. The learning rate is an important hyperparameter that determines the step size of weight updates during the optimization process. A learning rate that is too large may cause the model to directly skip the local optimal solution or the global optimal solution during training, thus failing to find the optimal model parameters. A learning rate that is too small may cause the model to easily fall into the local optimal solution during training and be unable to jump out, thus missing the global optimal solution. This will limit the performance of the model and fail to achieve the best results.

[0029] Preferably, in step 5, the number of mapping nodes of the BLS algorithm is 10, and the number of enhancement nodes is 10; the mapping nodes play the role of information conversion in the BLS network, and they convert the original features of the input data into enhanced features that can be further processed by the enhancement nodes. The enhancement nodes process the features of the input data through linear transformation (or more complex transformations) to generate new feature representations, which help improve the network's expressiveness and generalization capabilities.

[0030] The algorithm proposed in the present invention combines the CBAM-optimized ResNet50 model, the ROA optimizer and the BLS model to classify and predict fabric defect images. It not only has the fast convergence ability of ROA, but also has the excellent prediction performance of CBAM-ResNet50. The Resnet50-CBAM-ROA-BLS algorithm is mainly divided into three parts. The first part is to use the optimized ResNet50 convolutional neural network to perform convolution pooling and other feature extraction on textile defects, overcoming the limited feature extraction ability of traditional methods and realizing efficient feature extraction tasks. The second part is to optimize the model parameters through ROA, so that the prediction results of the model on the training data are closer to the true label, and the stability of the algorithm is improved. The third part uses BLS as a classifier to solve the problems of high computational complexity, easy overfitting, sensitivity to outliers, and imbalanced classification of the traditional Softmax classifier.

[0031] The present invention has the beneficial effects:

[0032] As the requirements for high-quality development of the textile industry continue to increase, automated textile defect detection technology is indispensable. In the field of automation, fabric defect detection (also known as inspection) is a quality control process that aims to identify and locate defects. Among them, the textile defect detection algorithm is the most critical part of textile defect detection technology. It is particularly important to design an efficient, convenient and accurate textile detection algorithm. The method of this article is to use a neural network to perform textile defect detection, learn through an existing training set with correct results, and then perform classification prediction on unknown images to achieve defect detection of unknown images.

[0033] The present invention uses Resnet50-CBAM-ROA-BLS. The accuracy of traditional classification methods is often limited by the quality of feature extraction and the effect is not good enough, so this paper proposes a new classification model. The attention mechanism is introduced into Resnet50, and the ROA algorithm is used to optimize the model parameters, and then the BLS algorithm is used to classify and output the feature vector, thereby greatly improving the performance and accuracy of image classification, and improving stability to a certain extent.

[0034] The invention can classify fabric defect images, and the classification accuracy of the method in two data sets is as high as 95.13% and 92.19% respectively. The model has excellent prediction accuracy, good convergence and stability, and can be directly used for automatic detection of fabric defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a Resnet50-CBAM-ROA-BLS model diagram constructed by the present invention;

[0036] Figure 2 It is a specific fabric diagram of the Tianchi cloth data set used in the present invention; in the diagram, a is a broken warp, b is a starch spot, c is a skipped yarn, d is a loose warp, e is a repair mark, and f is a skipped flower;

[0037] Figure 3 This is a specific fabric image of the TILDA data set used in the present invention; in the image, a is normal, b is holes, c is stains, d is broken warps, e is dragging yarn, and f is wrinkles;

[0038] Figure 4 It is a box plot of the classification accuracy of the experimental class compared with the method used in the present invention and other seven algorithms on the Tianchi cloth data set;

[0039] Figure 5 This is a time box plot of comparative experiment classification of the method used in the present invention and other seven algorithms on the Tianchi cloth dataset. DETAILED DESCRIPTION

[0040] The present invention will be further described below in conjunction with the accompanying drawings.

[0041] The present invention comprises the following steps:

[0042] Step 1: Obtain fabric defect classification dataset and preprocess it:

[0043] The experimental process of this invention uses a total of 2 data sets. The first one is the Tianchi cloth defect detection data set. It contains 2264 pictures with a resolution of 2446*1000, including 1139 defective image data and 1125 normal textile data. The defects include broken warp, starch spots, loose warp, etc. The specific textile images are shown in the attached figure. Figure 2 As shown in the figure, during training, the image is resized to a fixed size of 244*100 for training.

[0044] In addition, in order to verify the defect detection performance of the proposed algorithm on complex textured fabric surfaces, the second one uses data from the TILDA dataset. The TILDA dataset contains 8 fabric categories, each of which corresponds to 6 defect categories. We selected a floral pattern fabric as the dataset, and the final training set contains 1500 defective image data and 300 normal images. The specific textile images are shown in the attached figure. Figure 3 As shown in the figure, during training, the image is resized to a fixed size of 224*224 for training.

[0045] We perform data augmentation on the dataset by rotation, Gaussian noise, contrast enhancement, image flipping, and brightness enhancement, and divide it into training and test sets in a ratio of 8:2. The training set is used to train the network model, and the test set is used to test the model classification accuracy.

[0046] Step 2: Build the ResNet50-CBAM-ROA-BLS model:

[0047] The specific model structure diagram is as follows Figure 2 As shown in the figure, we use the pre-trained ResNet50 as the backbone network to obtain the features of textile defect images. The structure of ResNet50-CBAM-ROA-BLS can be divided into several main stages, from input to output: input layer (Input), Stage 0, CBAM, Stage 1, Stage 2, Stage 3, Stage 4, CBAM and BLS. Each stage consists of multiple residual blocks, which contain convolutional layers, batch normalization layers (Batch Normalization, referred to as BN), ReLU activation functions and cross-layer connections (Shortcut Connection). Among them, ROA is used to optimize the gradient update algorithm during training.

[0048] Input layer: Input the preprocessed initial image

[0049] Stage 0: Contains a convolutional layer for preliminary feature extraction of the input image; the BN layer performs batch normalization on the output of the convolutional layer to accelerate training and stabilize the gradient; the ReLU activation function performs nonlinear activation on the output of the BN layer; and the maximum pooling layer further reduces the size of the feature map.

[0050] CBAM: Connect the channel attention module (CAM) and the spatial attention module (SAM) in series to obtain the output feature map of Stage 0, recalibrate the input feature map, and enhance the model's attention on key features.

[0051] Stage 1: Contains 3 residual blocks (Bottleneck), each of which consists of multiple convolutional layers and a cross-layer connection. Inside each residual block, the number of channels is first reduced through a 1x1 convolutional layer (dimensionality reduction), then feature extraction is performed through a 3x3 convolutional layer, and finally the number of channels is restored through a 1x1 convolutional layer (dimensionality increase). The cross-layer connection adds the input of the residual block directly to the output to avoid the gradient vanishing problem.

[0052] Stage 2: Contains 4 residual blocks. The structure is similar to Stage 1, but the number of input and output channels of each residual block increases.

[0053] Stage 3: Contains 6 residual blocks, continuing to increase the number of channels and reduce the size of feature maps.

[0054] Stage 4: Contains 3 residual blocks. This is the last stage to further extract high-level features.

[0055] CBAM: Connect the channel attention module (CAM) and the spatial attention module (SAM) in series to obtain the output feature map of Stage 4, recalibrate the input feature map, and enhance the model's attention on key features.

[0056] Global Average Pooling: Perform global average pooling on the output of Stage 4 to reduce the size of the feature map to reduce the number of parameters and avoid overfitting.

[0057] BLS layer: The feature map of the global average pooling layer is expanded into a one-dimensional vector and passed into the BLS layer. After passing through the feature layer and the enhancement layer, it is mapped to the dimension of the number of categories for classification tasks.

[0058] This step mainly discusses the choice of attention mechanism.

[0059] The ResNet50 network with added attention mechanism and the ResNet50-BLS network were tested on the Tianchi cloth dataset, where ResNet50-SAM represents the ResNet50 network with added spatial attention module, ResNet50-CAM represents the ResNet50 network with added channel attention module, and ResNet50-CBAM represents the ResNet50 network with added lightweight hybrid attention module. The same is true for the following ResNet50-BLS network. The following table 1 shows the results of their detection accuracy.

[0060] Table 1 Experimental results of classification rate of different attention mechanisms on Tianchi cloth dataset (the best results are shown in bold)

[0061]

[0062] From Table 1, we can see that from the average results, the ResNet50 convolutional neural network and the ResNet50-BLS convolutional neural network have

[0063] In terms of classification accuracy, the results of adding the hybrid attention module are better.

[0064] Therefore, we sequentially add the hybrid attention module to ResNet50 using different types of optimizers.

[0065] Convolutional Neural Networks and ResNet50-BLS Convolutional Neural Networks.

[0066] Step 3: Train the ResNet50-CBAM-ROA-BLS model using the fabric defect classification dataset:

[0067] After preprocessing, the training input image is resized to 3x224x224 (number of channels, height, width), and BATCH_SIZE is 16, that is, the data input to the input layer is 16x3x224x224.

[0068] Stage 0: The first convolution layer uses a 7x7 convolution kernel with a stride of 2 and an output channel number of 64. MaxPooling layer: Uses a 3x3 pooling kernel with a stride of 2 to further reduce the size of the feature map, and the output size becomes 56x56. The final output is 16x64x56x56, which is passed to the CBAM module.

[0069] CBAM: This module mainly consists of the CA layer and the SA layer.

[0070] The CA layer performs global average pooling and global maximum pooling operations on the input feature map (shape is (16, 64, 56, 56). Both operations are performed on the spatial dimension (H×W). The output results are two feature maps with shapes of (16, 64, 1, 1), representing the average and maximum values ​​of each channel respectively. The above two feature maps are transformed through a shared multi-layer perceptron (MLP). The MLP contains two fully connected layers and a ReLU activation function. The first fully connected layer reduces the number of channels to reduce the number of parameters, and the second fully connected layer restores the number of channels to the original size. The two feature maps that have passed through the MLP are added element by element, and then the sigmoid activation function is used to obtain the final channel attention weight map Mc(N), whose dimension is 16×1×1. The channel attention weight map Mc(N) is multiplied channel by channel with the original feature map to obtain the weighted feature map M for subsequent spatial attention module processing.

[0071] The SA layer performs average pooling and maximum pooling operations on the feature map processed by the channel attention module in the channel dimension. Both operations are performed in the channel dimension, and the output results are two feature maps with shapes of (16, 1, 56, 56), which represent the average value and maximum value of each spatial position respectively. The above two feature maps are spliced ​​together in the channel dimension to form a new feature map (shape is (16, 2, 56, 56)). Then, a convolution layer (convolution kernel size is 7x7, step size is 1, padding is 3) is used to perform feature fusion and mapping, and the number of channels is changed to 1, and the weight coefficient of each spatial position is obtained (shape is (16, 1, 56, 56)). The output of the convolution layer is passed through the Sigmoid activation function to obtain the final spatial attention weight. These weights are multiplied with the original feature map (or the feature map processed by the channel attention module) to achieve feature recalibration of the spatial position.

[0072] The CBAM layer does not change the size of the feature map, but only changes the weight distribution in space and channels. The feature map size remains 16x64x56x56.

[0073] Stage 1: Contains 3 residual blocks (Bottleneck), each of which consists of multiple convolutional layers and a cross-layer connection. It receives the feature map passed in by the CBAM layer, the number of output channels increases from 64 to 256, the feature map size remains at 56x56, and the final output is 16x256x56x56.

[0074] Stage 2: Contains 4 residual blocks, and the structure is similar to Stage 1, but the number of input and output channels of each residual block increases. The number of output channels increases to 512, and the feature map size is halved to 28x28 through cross-layer connections or convolutional layers with a stride of 2, and the final output is 16x512x28x28.

[0075] Stage 3: Contains 6 residual blocks, continues to increase the number of channels and reduce the size of feature maps. The number of output channels increases to 1024, the size of feature maps is halved to 14x14, and the final output is 16x1024x14x14.

[0076] Stage 4: Contains 3 residual blocks. This is the last stage to further extract high-level features. The number of output channels increases to 2048, the feature map size is halved to 7x7, and the final output is 16x2048x7x7.

[0077] CBAM: Consistent with the previous CBAM layer, it obtains the output feature map of Stage 4, recalibrates the input feature map, and enhances the model's attention on key features.

[0078] Global Average Pooling: Perform global average pooling on the output of Stage 4 to reduce the size of the feature map, and the final output is 16x2048x1x1.

[0079] BLS layer: Expand the feature map of the global average pooling layer into a one-dimensional vector, i.e. 16x2048, and pass it to the BLS layer. After passing through the feature layer and the enhancement layer, it is mapped to the dimension of the number of categories for classification tasks. For example, if the Tianchi cloth defect detection dataset is trained with 5 categories, the final output is 16x5; if the TILDA dataset is trained with 6 categories, the result is 16x6. The largest value in each column vector corresponds to the predicted class.

[0080] In the process of using ROA (Robust Optimization Algorithm) as an optimizer, we use its unique mechanism to perform gradient updates to optimize the model parameters. As an advanced optimization method, ROA is particularly suitable for processing complex data sets that may contain noise or nonlinear characteristics. By fine-tuning the model parameters, ROA enables the model to better fit the objective function on the training data and gradually converge to the optimal solution.

[0081] At each step of the gradient update, ROA takes into account the diversity and potential uncertainty of the data, which makes it more robust when facing complex or noisy data. Through continuous iteration and optimization, the model gradually learns to extract the most valuable information from the input data and make accurate predictions based on it.

[0082] Thanks to ROA's optimization, the gap between the model's prediction results on the training data and the true labels is significantly narrowed. This not only improves the accuracy of the model, but also enhances its generalization ability when faced with new data or noisy data. At the same time, because ROA fully considers the complexity and uncertainty of the data during the optimization process, the stability of the algorithm has also been significantly improved. This makes the model more robust during training and less susceptible to interference from outliers or noise, so that it can converge to the optimal solution more reliably.

[0083] Step 4: Model classification prediction:

[0084] First, obtain the test set images. These images are not seen by the model during training, so they can provide an objective evaluation standard to help understand the performance of the model in practical applications. These images are passed to the trained Resnet50-CBAM-ROA-BLS model for prediction. During the prediction process, the model will perform a series of complex processing on the input test set images. The specific process is shown in the attached figure. Figure 4As shown in the figure. First, the high-level features of the image are extracted through the deep convolutional network of Resnet50; then, the CBAM module will weight these features to highlight important features and suppress unimportant features; then, the ROA optimization algorithm will fine-tune the model parameters based on these weighted features to ensure that the model can make the most accurate predictions; finally, the BLS system will use these optimized parameters and features to output the final prediction results.

[0085] Then a comparative analysis of algorithm performance is conducted.

[0086] 1. Classification accuracy analysis

[0087] In order to study the image classification effect of the ResNet50-CBAM-ROA-BLS model, the comparative experimental comparison method was adopted. The comparative experimental algorithm models used include ResNet50-CBAM algorithm model, ResNet50-CBAM-ROA algorithm model, ResNet50-CBAM-SFO algorithm model, ResNet50-CBAM-WOA algorithm model, ResNet50-CBAM-BLS algorithm model, ResNet50-CBAM-ROA-BLS algorithm model, ResNet50-CBAM-SFO-BLS algorithm model, and ResNet50-CBAM-WOA-BLS algorithm model.

[0088] Calculate the maximum value (Max), minimum value (Min), average value (Avg) and standard deviation (Stdv) of the accuracy of each algorithm. Run each algorithm ten times and take the average value as the corresponding result. The results obtained are as follows:

[0089] Table 2 Experimental results of detection and classification rate on Tianchi cloth dataset (the best results are shown in bold)

[0090]

[0091] As can be seen from Table 2, the minimum and maximum accuracy and average accuracy of the ResNet50-CBAM-ROA-BLS algorithm are better than those of other algorithms, and the standard deviation is slightly worse than the best ResNet50-CBAM-WOA-BLS.

[0092] On the TILDA dataset, we use the same method to test and compare eight algorithms. The experimental results are shown in Table 3. a to h are the confusion matrices of various algorithms, where 0 represents normal, 1 represents holes, 2 represents broken warps, 3 represents dragging yarn, 4 represents stains, and 5 represents wrinkles.

[0093] Table 3 Experimental results of detection and classification rate on TILDA dataset (the best results are shown in bold)

[0094]

[0095] As shown in Table 3, in the classification of complex texture fabric surfaces, the accuracy of other algorithms has dropped significantly compared to the previous data set, while ResNet50-CBAM-ROA-BLS demonstrates its strong anti-interference ability, with an average accuracy of 92.19%, which is better than other algorithms. In terms of stability, the optimizer SFO performed the worst, while ROA was second only to WOA with a small difference. In terms of classifiers, replacing the traditional Softmax classifier with BLS has significantly improved the classification accuracy.

[0096] 1. Algorithm stability analysis

[0097] In order to verify the stability of the ResNet50-CBAM-ROA-BLS algorithm, we compared it with the box plots of several algorithms such as ResNet50-CBAM and ResNet50-CBAM-ROA. The box plot data comes from each algorithm running ten times under 20 epoch conditions. The classification accuracy box plot is shown in the attached figure. Figure 4 As shown, the time box diagram for classification is shown in the attached Figure 5 As shown, A represents ResNet50-CBAM, B represents ResNet50-CBAM-ROA, C represents ResNet50-CBAM-SFO, D represents ResNet50-CBAM-WOA, E represents ResNet50-CBAM-BLS, F represents ResNet50-CBAM-ROA-BLS, G represents ResNet50-CBAM-SFO-BLS, and H represents ResNet50-CBAM-WOA-BLS.

[0098] By the attached Figure 4 and attached Figure 5 From the box plot, we can see that the stability of ResNet50-CBAM-ROA-BLS is slightly worse than that of ResNet50-CBAM-WOA-BLS in terms of classification accuracy, but its accuracy is the best among all other algorithms, and the ResNet50 network uses its own Softmax classifier, so the classification accuracy of the series of algorithms is not as good as ResNet50-BLS. In terms of classification time, the use of BLS increases the number of computing nodes, which increases the accuracy while also increasing the training time. We can see that the time used by the ResNet50-BLS series of algorithms is longer than that of ResNet50, but its improvement in accuracy performance is worth the loss in training classifier time.

Claims

1. A method for classifying fabric defects in images based on deep learning, characterized in that: The method comprises the following steps: Step 1, obtain images and build a dataset: The images include images of normal fabrics and fabrics with defects; each image in the data set includes an image label, and the image label is a defect type; Step 2: construct a fabric defect classification model. The image fabric defect classification model uses ResNet50 as the backbone network. The structure of the image fabric defect classification model includes, from input to output: input layer Input, Stage 0 layer, CBAM1 layer, Stage 1 layer, Stage 2 layer, Stage 3 layer, Stage 4 layer, CBAM2 layer, global average pooling layer and BLS layer; The input layer Input is the image obtained in step 1; both the CBAM1 layer and the CBAM2 layer include: a channel attention module CAM and a spatial attention module SAM ; Use the global average pooling layer to perform global average pooling on the output of the CBAM2 layer ; The feature map output by the global average pooling layer is expanded into a one-dimensional vector and passed into the BLS layer. After passing through the feature layer and the enhancement layer, it is mapped to the dimension of the number of categories for classification tasks. Step 3: Use the robust optimization algorithm ROA as the optimizer to optimize the fabric defect classification model and obtain the trained Resnet50-CBAM-ROA-BLS; Step 4: Model classification prediction Get the image to be classified, pass it into the trained Resnet50-CBAM-ROA-BLS model for prediction, and output the classification result.

2. The method for classifying fabric defect images based on deep learning according to claim 1, characterized in that: The training process of the image fabric defect classification model includes the following steps: The robust optimization algorithm ROA is used as an optimizer, and the image fabric defect classification model is trained using the data set obtained in step 1 until the image fabric defect classification model converges.

3. The method for classifying fabric defect images based on deep learning according to claim 1, characterized in that: In step 1, the data set is enhanced by rotation, Gaussian noise, contrast enhancement, image flipping, and brightness enhancement, and is divided into a training set and a test set in a ratio of 8:

2.

4. The method for classifying fabric defect images based on deep learning according to claim 1, characterized in that: In step 2, the learning rate of ResNet50 is 1*10 -4 , the number of training rounds Eporch is selected as 20 rounds, and BATCH_SIZE is 16.

5. The method for classifying fabric defect images based on deep learning according to claim 1, characterized in that: In step 5, the number of hidden nodes of the BLS algorithm is 10, and the number of enhanced nodes is 10.

Citation Information

Cited By

  • Small sample image classification method and device and medium

    CN120510459A

  • Small sample image classification method, device and medium

    CN120510459B