CTPN-based fabric defect detection method

Abstract

This invention discloses a fabric defect detection method based on CTPN, comprising the following steps: S1: acquisition of fabric dataset image information; S2: division of fabric dataset image samples; S3: feature extraction of fabric dataset images; S4: CTPN-based model construction for fabric dataset: (1) image preprocessing module; (2) network framework optimization: replacing with Mobilenetv2 network as feature extractor; (3) using bidirectional LSTM; (4) specific anchor design module; (5) post-processing module; S5: defect detection of fabric dataset. This invention is stable, reliable, and has strong generalization ability, and can be directly promoted. It completely abandons traditional algorithms, increases the robustness of the model, and greatly improves the accuracy of fabric defect localization. It not only meets the accuracy requirements of fabric defect detection, but also reduces the inference time by 3 / 4, accelerating the implementation of industrial-grade textile mill fabric detection algorithms.

Description

Technical Field

[0001] This invention relates to the field of fabric inspection technology, and in particular to a fabric defect detection method based on CTPN. Background Technology

[0002] Fabric defect detection is a crucial step in the textile industry. Currently, manual inspection remains the primary method of quality control in the textile sector. However, in recent years, due to rising labor costs and the inherent problems of manual inspection, such as slow speed, high false negative rates, poor consistency, and high staff turnover, more and more factories are beginning to utilize machines to replace manual labor for quality inspection, thereby improving production efficiency and saving labor costs. Therefore, fabric defect detection technology is of paramount importance in the textile industry.

[0003] Currently, most defect detection methods for industrial fabrics use traditional algorithms, such as edge detection and region segmentation. However, fabric defects (such as warp breaks) are highly variable, and these methods have extremely poor generalization ability. Furthermore, fabric defects are highly sensitive to environmental factors, which can affect the overall results. Therefore, these methods are only suitable for experimental applications in traditional chemical enterprises and cannot be widely adopted. Summary of the Invention

[0004] This invention discloses a fabric defect detection method based on CTPN, aiming to solve the technical problems in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The CTPN-based fabric defect detection method includes the following steps:

[0007] S1: Acquisition of image information for the fabric dataset: Multiple images of fabric defects were captured using a camera;

[0008] S2: Division of image samples in the fabric dataset;

[0009] S3: Feature extraction from fabric dataset images;

[0010] S4: Fabric dataset model built based on CTPN:

[0011] (1) Image preprocessing module: The preprocessing module of industrial instruments is mainly for different lighting conditions such as strong light and dark light. It averages the brightness of the whole image and uses histogram equalization to adjust the brightness to a uniform level.

[0012] (2) Network framework optimization: In response to the requirements of algorithm deployment, the network framework was further optimized by replacing the original VGG16 with Mobilenetv2 as the feature extractor.

[0013] (3) Use bidirectional LSTM: The CTPN algorithm is used for text box detection. The main consideration for transferring the CTPN algorithm to fabric defect detection is that bidirectional LSTM plays a key role in temporal feature detection.

[0014] (4) Specific anchor design module: Observation of fabric defects. The vertical anchor settings in the CTPN algorithm are fully applied to fabric detection. A set of 10 anchors of equal width is used to locate the position of fabric defects.

[0015] (5) Post-processing module: used for defect detection. A defect detection image of a piece of fabric is preprocessed by histogram equalization of the image, and then the trained model is loaded. The model outputs the classification branch, the vertical coordinate of the defect box and the offset of the side-refinement.

[0016] S5: Defect detection in fabric datasets:

[0017] (1) After preprocessing the image and histogram equalization, the defects of the fabric are more obvious, which is beneficial to the training of the model.

[0018] (2) Input the image into the backbone network of the CTPN algorithm, extract features through the backbone network, generate an N×C×H×W feature map, slide 3×3 convolution in the feature map, and then perform im2col operation. Each slide will result in a 3×3×C feature vector. Finally, a new N×9C×H×W feature map is generated and then input into BLSTM (bi-directional long short-term memory) for sequence feature extraction. Then, it is passed into the fully connected layer for further feature extraction. The fully connected layer is followed by 3 fully connected layer branches to predict vertical coordinate regression, classification score and horizontal translation regression respectively. Finally, the graph-based text construction algorithm is used to obtain the thin rectangular box.

[0019] In a preferred embodiment, in step S2, the collected images are randomly sampled and divided into independent and non-repeating validation and test sets according to a certain ratio.

[0020] In a preferred embodiment, in step S3, features of the detected image are extracted, and a recognition model for the detected image dataset is constructed on the validation set. The parameters of the recognition model are determined, and then the recognition effect is detected using the test set to verify the model performance.

[0021] In a preferred embodiment, in S4, Mobilenetv2 uses a convolution with a 1x1xC1 kernel and LSTM, and the feature map output by v2 is convolved by a specially designed CONVLSTM. The resulting feature map is then input into the RPN network for learning.

[0022] In a preferred embodiment, the main implementation method of bidirectional LSTM in S4 is as follows: the bottom layer uses VGG16 features, which consists of a W*H*C Conv5 feature map. A 3*3 spatial window is used to slide the window on the feature map of the last convolutional layer (VGG16 Conv5). The sequential windows in each row are cyclically connected by BLSTM (bi-directional long short-term memory). The convolutional features (3x3xC) of each window are used as the input of BLSTM. Then, bidirectional BLSTM is implemented to enhance the learning of information of associated sequences. Finally, the feature map output by the last convolutional layer of VGG is converted into vector form for subsequent BLSTM training.

[0023] In a preferred embodiment, in S4, the specific anchor design, due to network optimization, makes the width and height of the feature map of the last layer output by the feature extractor 1 / 16 of the width and height of the input image. At the same time, the feature map of this layer is stretched into a column vector for subsequent BLSTM training. The output of BLSTM is fed into the fully connected layer, and the cloth detection model is learned by using the three predicted values ​​and the ground truth error output by the network.

[0024] In a preferred embodiment, the post-processing module's stage training model in S4 is mainly divided into three stages: the first stage is data preprocessing, the second stage is training the network framework using the Mobilenetv2 framework as a feature extractor, and the third stage is training specific anchors.

[0025] In a preferred embodiment, after the post-processing module has trained the fabric defect detection models, the following steps are required when deploying these models:

[0026] A. An RGB image of a fabric defect was captured from the camera;

[0027] B. The image obtained after image preprocessing;

[0028] C. Then it is fed into the improved Mobilenetv2 network framework;

[0029] D. The output of the network framework optimization module is fed into the BLSTM module to learn the temporal features of the fabric; then the output features are input into the fully connected layer, and finally the model outputs the classification branch, the vertical coordinates of the defect box, and the offset of the side-refinement.

[0030] As can be seen from the above, the CTPN-based fabric defect detection method provided by this invention is stable, reliable, and has strong generalization ability. It can be directly promoted, completely abandoning traditional algorithms and using deep learning methods to improve generalization ability, accuracy, and reduce false detections. By applying the text detection framework CTPN algorithm to fabric defect detection, the model learns the relationship of temporal features, increasing the robustness of the model and greatly improving the accuracy of fabric defect localization. The optimization of the network framework not only meets the accuracy requirements of fabric defect detection but also reduces the inference time by 3 / 4, accelerating the implementation of industrial-grade textile mill fabric detection algorithms. Attached Figure Description

[0031] Figure 1 This is a flowchart of the fabric defect detection method based on CTPN proposed in this invention.

[0032] Figure 2 This is an optimized diagram of the network framework for the CTPN-based fabric defect detection method proposed in this invention.

[0033] Figure 3 This is the original image of the fabric defect in the fabric defect detection method based on CTPN proposed in this invention.

[0034] Figure 4 This is the equalization image of the original fabric defect image for the fabric defect detection method based on CTPN proposed in this invention.

[0035] Figure 5 This is a label segmentation diagram of the fabric defect map in the fabric defect detection method based on CTPN proposed in this invention.

[0036] Figure 6 This is a design diagram of the anchor for the fabric defect detection method based on CTPN proposed in this invention.

[0037] Figure 7 The image shows the fabric defect detection effect of the CTPN-based fabric defect detection method proposed in this invention.

[0038] Figure 8 This is a fabric defect detection image based on the CTPN-based fabric defect detection method proposed in this invention. Detailed Implementation

[0039] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0040] Reference Figure 1-8 The fabric defect detection method based on CTPN includes the following steps:

[0041] S1: Acquisition of image information for the fabric dataset: Multiple images of fabric defects were captured using a camera;

[0042] S2: Division of image samples in the fabric dataset;

[0043] S3: Feature extraction from fabric dataset images;

[0044] S4: Fabric dataset model built based on CTPN:

[0045] Image preprocessing module: The preprocessing module of industrial instruments is mainly designed for different lighting conditions such as strong light and low light. It averages the brightness of the entire image and uses histogram equalization to adjust the brightness to a uniform level.

[0046] like Figure 3-5 Histogram equalization was applied to overly dark images to adjust their brightness to a reasonable level, which is helpful for subsequent analysis.

[0047] Network framework optimization: In response to the requirements of algorithm deployment, the network framework was further optimized by replacing the original VGG16 network with Mobilenetv2 as the feature extractor.

[0048] The original CTPN text box detection algorithm uses the VGG16 network framework, which cannot meet the real-time requirements of industrial-grade fabric defect detection. Ordinary LSTM plays an important role in processing two-dimensional temporal data, but for three-dimensional images, which have rich spatial information and strong correlations between each pixel and its surrounding pixels, ordinary LSTM performs only moderately. CONVLSTM, on the other hand, not only adds convolution operations to LSTM to capture spatial features for three-dimensional image processing, but also makes feature extraction from images more effective.

[0049] (3) Use bidirectional LSTM: The CTPN algorithm is used for text box detection. The main consideration for transferring the CTPN algorithm to fabric defect detection is that bidirectional LSTM plays a key role in temporal feature detection.

[0050] (4) Specific anchor design module: Observation of fabric defects. The vertical anchor settings in the CTPN algorithm are fully applied to fabric detection. A set of 10 anchors of equal width is used to locate the position of fabric defects.

[0051] The backbone is an improved version of Mobilenetv2. The model is compressed to a very small size, which can solve most of the speed and accuracy problems. In addition, the addition of CONVLSTM makes the features extracted by the feature extractor have stronger spatial correlation, which is more conducive to the learning of temporal features of the BLSTM module. It achieves excellent results in fabric defect detection.

[0052] (5) Post-processing module: used for defect detection. A defect detection image of a piece of fabric is preprocessed by histogram equalization of the image, and then the trained model is loaded. The model outputs the classification branch, the vertical coordinate of the defect box and the offset of the side-refinement.

[0053] S5: Defect detection in fabric datasets:

[0054] (1) After preprocessing the image and histogram equalization, the defects of the fabric are more obvious, which is beneficial to the training of the model.

[0055] (2) Input the image into the backbone network of the CTPN algorithm, extract features through the backbone network, generate an N×C×H×W feature map, slide 3×3 convolution in the feature map, and then perform im2col operation. Each slide will result in a 3×3×C feature vector. Finally, a new N×9C×H×W feature map is generated and then input into BLSTM (bi-directional long short-term memory) for sequence feature extraction. Then, it is passed into the fully connected layer for further feature extraction. The fully connected layer is followed by 3 fully connected layer branches to predict vertical coordinate regression, classification score and horizontal translation regression respectively. Finally, the graph-based text construction algorithm is used to obtain the thin rectangular box.

[0056] In a preferred embodiment, in S2, the collected images are randomly sampled and divided into independent and non-repeating validation and test sets according to a certain ratio.

[0057] In a preferred embodiment, in S3, features of the detected image are extracted, and a recognition model of the detected image dataset is constructed on the validation set. The parameters of the recognition model are determined, and then the recognition effect is detected using the test set to verify the model performance.

[0058] In a preferred embodiment, in S4, Mobilenetv2 uses a convolution with a 1x1xC1 kernel and LSTM, and the feature map output by v2 is convolved by a specially designed CONVLSTM. The resulting feature map is then input into the RPN network for learning.

[0059] In a preferred embodiment, in S4, the main implementation method of bidirectional LSTM is as follows: the bottom layer uses VGG16 features, which consists of a W*H*C Conv5 feature map. A 3*3 spatial window is used to slide the window on the feature map of the last convolutional layer (VGG16 Conv5). The sequential windows in each row are cyclically connected by BLSTM (bi-directional long short-term memory). The convolutional features (3x3xC) of each window are used as the input of BLSTM. Then, bidirectional BLSTM is implemented to enhance the learning of information of associated sequences. Finally, the feature map output by the last convolutional layer of VGG is converted into vector form for subsequent BLSTM training.

[0060] In a preferred embodiment, in S4, the specific anchor design, due to network optimization, makes the width and height of the feature map of the last layer output by the feature extractor 1 / 16 of the width and height of the input image. At the same time, the feature map of this layer is stretched into a column vector for subsequent BLSTM training. The output of BLSTM is fed into the fully connected layer, and the cloth detection model is learned by using the three predicted values ​​and the ground truth error output by the network.

[0061] As attached Figure 6 As shown in the small and medium squares, the width of the Anchor is widths=

[16] , heights=[11,16,23,33,48,68,97,139,198,283].

[0062] As attached Figure 7 As shown in the experimental diagram, the lines represent defects (broken warp sections) in the test set.

[0063] Training parameters were set with the learning rate decreasing periodically during training. The total training period was 100 epochs, with the learning rate decreasing by 0.1 in each epoch. The epochs were [45, 90]. The initial learning rate was 0.001, and the final learning rate was 10 to the power of negative 5. The network structure diagram is attached. Figure 8 .

[0064] In a preferred embodiment, in S4, the stage training model of the post-processing module is mainly divided into three stages: the first stage is data preprocessing, the second stage is the training of the network framework using the Mobilenetv2 framework as the feature extractor, and the third stage is the training of specific anchors.

[0065] In a preferred embodiment, after the post-processing module has trained the fabric defect detection models, when deploying these models, the following process steps need to be invoked:

[0066] A. An RGB image of a fabric defect was captured from the camera;

[0067] B. The image obtained after image preprocessing;

[0068] C. Then it is fed into the improved Mobilenetv2 network framework;

[0069] D. The output of the network framework optimization module is fed into the BLSTM module to learn the temporal features of the fabric; then the output features are input into the fully connected layer, and finally the model outputs the classification branch, the vertical coordinates of the defect box, and the offset of the side-refinement.

[0070] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting defects in a cloth based on CTPN, characterized in that, The method comprises the following steps: S1: acquisition of cloth dataset image information: multiple cloth defect pictures are collected by a camera; S2: division of cloth dataset image samples; S3: feature extraction of the cloth dataset image; S4: model construction of the cloth dataset based on CTPN: (1) image preprocessing module: the preprocessing module of the industrial instrument averages the brightness of the whole picture in different light conditions such as strong light and dim light, and adjusts the brightness to a uniform level by using the histogram equalization method; (2) network framework optimization: the network framework is further optimized according to the requirements of algorithm deployment on the landing side, and the original VGG16 is replaced by the network of Mobilenetv2 as a feature extractor; (3) use of bidirectional LSTM: the CTPN algorithm is used for text box detection, and the CTPN algorithm is migrated to cloth defect detection, mainly considering that bidirectional LSTM plays a key role in time sequence feature detection; (4) specific anchor design module: the vertical direction anchor setting in the CTPN algorithm is applied to cloth detection, and a set of 10 equal-width anchors are used to locate the position of the cloth defect; (5) post-processing module: for defect detection, the defect detection image of a piece of cloth is preprocessed by histogram equalization of the image, then the trained model is loaded, and the model outputs the classification branch, the vertical coordinate of the defect frame and the offset of side-refinement training; S5: defect detection of the cloth dataset: (1) histogram equalization is performed on the picture, and the cloth defects are more obvious, which is conducive to the training of the model; (2) the picture is input into the backbone network of the CTPN algorithm for feature extraction to generate an NxCxHxW feature map, then 3x3 convolution is performed on the feature map, then im2col operation is performed, then a 3x3xC feature vector is obtained each time the sliding is performed, finally a new N*9C*H*W feature map is generated, then the feature map is input into BLSTM for sequence feature extraction, and then the BLSTM is input into a full connection layer for further feature extraction, the full connection layer is connected with three full connection layers, respectively, to predict the vertical coordinate regression, the classification score and the horizontal translation amount regression, and finally the algorithm based on the graph text construction is used to obtain an elongated rectangular frame; In S4, the feature map of the last layer output by the feature extractor has a width and height of 1 / 16 of the input image, and the layer feature map is pulled into a column vector for subsequent BLSTM training, and the output of the BLSTM is input into a full connection layer, and the three prediction values output by the network are used to learn the model for cloth detection. 2.The CTPN-based cloth defect detection method of claim 1, wherein, In S2, the collected pictures are divided into independent and non-repeating verification set and test set in a certain proportion by random sampling. 3.The CTPN-based cloth defect detection method of claim 1, wherein, In S4, Mobilenetv2 uses a convolution kernel of 1x1xC1 and a convolution with LSTM, and the feature map output by Mobilenetv2 is subjected to a specific design CONVLSTM convolution to generate a feature map to be input into an RPN network for learning. 4.The CTPN-based cloth defect detection method of claim 1, wherein, In the S4, the main implementation method of the bidirectional LSTM is as follows: the bottom layer uses VGG16 features, a W*H*C Conv5 feature map is used, a spatial window with a size of 3*3 is used, the window is slid on the feature map of the last convolutional layer (Conv5 of VGG16), the sequential windows in each row are connected in a BLSTM loop, the convolutional features (3x3xC) of each window are used as the input of the BLSTM, the bidirectional BLSTM is implemented, the information learning of the associated sequence is enhanced, and the feature map output by the last convolutional layer of VGG is converted into a vector form for the subsequent BLSTM training. 5.The CTPN-based cloth defect detection method of claim 1, wherein, In the S4, the post-processing module is divided into three stages: the first stage is data preprocessing, the second stage is network framework training, and the MobileNetV2 framework is used as a feature extractor, and the third stage is specific anchor training. 6.The CTPN-based cloth defect detection method of claim 5, wherein, After the fabric defect detection model is trained by the post-processing module, the models are deployed, and the process steps are as follows: A. An RGB fabric defect picture is collected from a camera; B. Picture preprocessing; C. The picture is sent to the improved MobileNetV2 network framework; D. The output is sent to the BLSTM module to learn the time sequence features of the fabric; The output features are input into the full connection layer, and finally the model outputs the classification branch, the vertical coordinate of the defect frame and the offset of the side-refinement.

Images