A defective fabric generation method based on improved pix2pixGAN network model
Through the improved joint training of pix2pixGAN network model, generator and discriminator, the problem of limited labor and data enhancement in the existing technology is solved, and high-quality fabric defect data set generation is achieved, which significantly improves the accuracy of defect detection.
Patent Information
- Application Number
- CN202111567060.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2025-03-25
- Estimated Expiration
- 2041-12-20
AI Technical Summary
The prior art relies on manual detection in fabric defect detection, which is time-consuming and labor-intensive. The dependence of deep learning models on large-scale labeled data sets leads to limited data enhancement methods and is unable to effectively generate high-quality fabric defect data sets.
Using the improved pix2pixGAN network model, the generator and discriminator are trained to generate a new fabric defect data set by stitching and separation of semantic segmentation diagrams, defects are generated on different textured fabrics through the DPGAN model.
The generated defect data set can effectively improve the accuracy of defect model detection, and the model detection rate has increased from 73% to 84%, an increase of 11%.
Smart Images
Figure CN114240904B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of fabric defect detection, and in particular to a method for generating defective fabrics based on an improved pix2pixGAN network model. Background Art
[0002] Fabric defect detection is crucial in the textile industry. Traditional manual detection is time-consuming, highly repetitive, labor-intensive, and not very stable. With the development of computer technology, people have applied machine vision and image processing to defect detection, which has the advantages of high detection accuracy, good objective repeatability, and fast speed. In recent years, deep learning has occupied a core position, but deep learning mainly relies on data-driven. How to obtain a large-scale labeled data set has become the research goal of relevant personnel in recent years. Commonly used methods include image flipping, random cropping, color jitter, and noise perturbation, but these methods are some transformations on the original data set, which is not very meaningful for neural networks. With the emergence of GAN networks in recent years, real data set enhancement can be achieved. The GAN network is inspired by the two-person zero-sum game theory in game theory, and its unique adversarial training idea can generate high-quality samples. At the same time, many improved models have been derived, such as DCGAN, CylcleGAN, pix2pixGAN, etc. The literature [Liu Juhua, Wang Chaoyue, Su Hai, Du Bo, Tao Dacheng. Multistage GAN for Fabric Defect Detection. [J]. IEEE transactions on image processing: a publication of the IEEE Signal Processing Society, 2019.] uses a multi-layer GAN network to first generate fabric with defects, and then smoothly merges the defects with the fabric to obtain a new fabric defect dataset. However, there are problems such as the network being complex and applicable to specific models. Summary of the invention
[0003] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method for generating defective fabrics based on an improved pix2pixGAN network model.
[0004] In order to achieve the above object, the present invention adopts the following technical solutions:
[0005] A defective fabric generation method based on an improved pix2pixGAN network model comprises the following steps:
[0006] S1. Obtain a fabric yarn defect data set, mark the defect locations of the fabric, and obtain a semantic segmentation map of the fabric and the defects;
[0007] S2, horizontally splicing the semantic segmentation map, the defect-free map and the defect map to obtain a spliced image;
[0008] Separate the stitched image into semantic segmentation map, defect-free map and defect map by image segmentation Figure 3 The sizes of the three groups of images are normalized; the semantic segmentation map and the defect-free image are packaged and imported as the training set of the generator using the map function, and the defect image is imported into the discriminator at the same time;
[0009] S3, inputting the training set into the generator, inputting the defect image into the discriminator, and optimizing the generator and the discriminator by taking the sum of the cross entropy and L1Loss loss function between the generator generated image and the discriminator defect image as the objective function, so that the generator outputs the target domain image, and obtaining the DPGAN model;
[0010] S4. Randomly perform semantic segmentation on the new textured fabric, concatenate the semantically segmented image with the original defect-free fabric image, input it into the DPGAN model, and generate a new texture defect fabric image.
[0011] Furthermore, the specific process of generating the semantic segmentation map of fabric and defects in S1 is as follows:
[0012] Modify the texture yarn defect data to JPEG format and change the size to 600×600;
[0013] Circle the fabric defects and change them to white to obtain a semantic segmentation map.
[0014] Furthermore, the generator in S3 is a U-net model consisting of a downsampling path and an upsampling path;
[0015] The downsampling path consists of the first downsampling path and the second downsampling path, which are connected to 4×4 convolution kernels respectively, and ReLU is used for correction after convolution;
[0016] The deconvolution kernel size of the upsampling path is 4×4, and ReLU is used for correction after deconvolution; the number of channels in the first 4 layers remains unchanged, and the number of channels in the subsequent layers is halved each time;
[0017] When inputting, the first downsampling path and the second downsampling path input data at the same time. The first path inputs the semantic segmentation map, and the second path inputs the defect-free fabric map. The number of channels doubles after each convolution, and the number of channels remains unchanged in the last 4 layers. In the last layer of downsampling, only the downsampling operation of the first downsampling path is performed.
[0018] The upsampling path deconvolves the final downsampling of the first downsampling path, and the obtained feature map is combined with the feature map of the same number of channels of the second downsampling path by concatenation, and then deconvolution is performed. After activation by the Relu function, it is combined with the feature map of the same number of channels of the second downsampling path again for deconvolution, and the above operation is repeated until the last two layers of deconvolution; no combination operation is performed in the last two layers of deconvolution operation; the last layer of deconvolution uses the tanh activation function, and the remaining layers are activated by the Relu function.
[0019] Furthermore, the Dropout function is added to the first three layers of deconvolution, and the parameter is set to 0.5.
[0020] Furthermore, the downsampling path has 8 layers and the upsampling path has 8 layers.
[0021] Furthermore, the objective function of the entire network in S3 is:
[0022]
[0023] In the figure: x represents the semantic segmentation image, y represents the original defective fabric image, z represents the original defect-free fabric image, G(x,z) represents the target domain image generated by the G network based on the semantic segmentation image and the original defect-free fabric image; D(x,y) represents the probability that the D network judges whether the real image is real, and D(x,G(x,z)) is the probability that the D network judges whether the image generated by the G network is real;
[0024] Here, x represents the semantic segmentation image, y represents the original defective fabric image, z represents the original defect-free fabric image, and G(x,z) represents the target domain image generated by the G network based on the semantic segmentation image and the original defect-free fabric image.
[0025] Furthermore, the optimization operations in S3 are:
[0026] The generator and discriminator are optimized using the loss function. The Adam optimizer is selected in the optimization process. The initial learning rate is 0.0001, and the random initialization method is selected for initializing the network parameters. The cycle is repeated 1000 times to obtain the optimized generator and discriminator.
[0027] Furthermore, the sizes of the three groups of images are normalized to 512×512 in S2.
[0028] Compared with the prior art, the present invention has the following beneficial effects:
[0029] The present invention proposes a defect fabric generation method based on an improved pix2pixGAN network model. Through the DPGAN fabric defect dataset generation method, defects can be generated on fabrics with different textures, and the dataset is enhanced. The size, position, and type of the defects are all manually set. The experimental results show that the generated defect dataset is very helpful to improve the accuracy of defect model detection. Compared with the unenhanced model, the model detection rate is increased from 73% to 84%, an increase of 11%. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 is a flow chart of an embodiment;
[0031] Figure 2 This is the structure diagram of the generator;
[0032] Figure 3 is a textured fabric yarn defect map, where: Figure 3 (a) Figure 3 (b) Figure 3 (c) shows three different textures;
[0033] Figure 4 is a defect-free image of textured fabric, where Figure 4 (a) Figure 4 (b) Figure 4 (c) shows three different textures;
[0034] Figure 5 This is a partial picture of Faster-Rcnn detection results before enhancing the data set, where: Figure 5 (a) Figure 5 (b) Figure 5 (c) Three representatives;
[0035] Figure 6 This is a partial diagram of the Faster-Rcnn detection results after enhancing the data set, where: Figure 6 (a) Figure 6 (b) Figure 6 (c) Three representatives. DETAILED DESCRIPTION
[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0038] It is very difficult to collect fabric defect images with different textures and defect morphologies, but it is relatively simple to collect defect-free fabric images containing various types of fabric backgrounds. If a generative model suitable for fabric defects can be constructed using a GAN network, and various defects can be synthesized on clean defect-free images, the training data set can be effectively enhanced, thereby improving the efficiency of subsequent target detection models. Based on the above analysis, the present invention proposes a method for enhancing a fabric yarn defect data set using an improved Pix2PixGAN network (referred to as DPGAN in the present invention), and uses a target detection network Faster R-CNN to verify the improvement effect of this method on the detection model. The results show that the yarn defect images generated by this method are more realistic and can effectively improve the accuracy of defect detection.
[0039] The present invention is further described in detail below in conjunction with the accompanying drawings:
[0040] See also Figure 1 , Figure 1 The flowchart of the embodiment includes the following steps:
[0041] Step 1: Extract the training set from the acquired texture 1 fabric defect data set, mainly the yarn defect data, and modify them to JPEG format, then modify the size to 600×600, and finally import them into PS software, use the lasso tool to circle the fabric defects, and modify them to white to produce the effect of semantic segmentation;
[0042] Step 2: Splice the semantically segmented fabric defect image with the original defective fabric image horizontally to make it 1200×600 in size, and then splice it horizontally with the original non-defective fabric image to make it 1800×600 in size, and manually remove the data set with large texture differences to ensure the neatness of the data set. Then, in the preprocessing stage of the model, use image segmentation to separate the three images, and resize them to (512×512) and normalize them. Use the map function to package the semantic segmentation map and the non-defective map as the training set of the generator and import the defect map into the discriminator.
[0043] Step 3: Train the DPGAN network model
[0044] The generator in step 3 is a U-net model composed of a downsampling path and an upsampling path, such as Figure 2 shown.
[0045] Add an extra path in the downsampling path. Input the training sets A and B into the two downsampling paths respectively, connect a 4×4 convolution kernel with a step size of 2, and use ReLU to correct after convolution. Repeat the above convolution operation, doubling the number of convolution channels each time, and keep the number of channels unchanged in the last 4 layers. Finally, in the bottom layer of the U-net model, that is, the last downsampling layer, only the downsampling operation of path A is required.
[0046] In the upsampling path, the deconvolution kernel size of the first layer connection is 4×4, with a step size of 2, and Relu is used for correction after deconvolution. Then, in the second upsampling operation, the penultimate feature map of the B path in the downsampling is first combined with the feature map to be upsampled for the second time, and then the deconvolution operation is performed; repeat the above operation, and the number of channels is halved each time, and the number of channels in the first 4 layers remains unchanged. No combination operation is required in the last 2 layers of convolution operation; the last layer of deconvolution needs to use the tanh activation function. Add the Dropout function to the first 3 layers of deconvolution, and set the parameter to 0.5.
[0047] The objective function of the DPGAN model is defined as:
[0048]
[0049] When optimizing, the training set is input into the generator to obtain G(x,z), and then G(x,z) and y are simultaneously sent to the discriminant model to calculate their cross entropy loss and L1 loss added as the loss function; the objective function of the entire network is written as:
[0050]
[0051] Where: x represents the semantic segmentation image, y represents the original defective fabric image, z represents the original defect-free fabric image, and G(x,z) represents the target domain image generated by the G network based on the semantic segmentation image and the original defect-free fabric image.
[0052] D(x,y) represents the probability that the D network judges whether the real image is real, and D(x,G(x,z)) represents the probability that the D network judges whether the image generated by the G network is real. L1 Loss will make the images in the source domain and the target domain as close as possible. ;
[0053] The low-frequency part of the image can be restored, while The high-frequency part of the image can be restored, making the source domain image closer to the target domain image.
[0054] The network model was optimized using the loss function. The Adam optimizer was selected during the optimization process. The initial learning rate during optimization was 0.0001. The random initialization method was selected for the initialization of network parameters. The loop was repeated 1000 times to obtain the optimized generator and discriminator models.
[0055] Step 4: Test the DPGAN model.
[0056] Using the texture 2 defect-free fabric image, modify the format to JPEG format, size 600×600, import it into PS software, use the lasso tool to randomly circle the yarn type blanks to produce the effect of semantic segmentation. It is spliced with the corresponding defect-free fabric image and imported into the DPGAN model to generate the texture 2 defect fabric image.
[0057] Step 5: Faster-Rcnn model recognition accuracy.
[0058] The defect category and the real box containing the defect are obtained by using LabelImg marking, and then the original PascalVOC dataset and the enhanced dataset are established respectively; the Faster-Rcnn network training model is imported to verify the improvement of the model detection rate by the enhanced dataset.
[0059] The data set of the tape yarn before enhancement is 1100 texture 1 fabrics, and the data set of the tape yarn after enhancement is 1100 texture 1 fabrics and 612 texture 2 fabrics, totaling 1712. The Faster-Rcnn model test set consists of 100 texture 1 defect fabrics and 100 texture 2 defect fabrics, and the AP value is used as the experimental evaluation indicator.
[0060] See also Figure 3-Figure 6 , Figure 3 (a) Figure 3 (b) Figure 3 (c) shows three different textured fabric yarn defect images. Figure 4 (a) Figure 4 (b) Figure 4 (c) Defect-free images of three different textured fabrics; Figure 5 (a) Figure 5 (b) Figure 5 (c) is the Faster-Rcnn detection result diagram before three enhanced data sets; Figure 6 (a) Figure 6 (b) Figure 6 (c) is the Faster-Rcnn detection result diagram after three enhanced data sets. From the above detection result diagrams, we can see that for the texture 1 data set, the model can detect the defect location and type before and after enhancement. For the texture 2 data set, the target location that was not detected before enhancement was successfully detected after enhancement, which proves the effectiveness of our method.
[0061] The above contents are only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.
Claims
1. A defective fabric generation method based on an improved pix2pixGAN network model, characterized in that: The following steps are involved: S1. Obtain a fabric yarn defect data set, mark the defect locations of the fabric, and obtain a semantic segmentation map of the fabric and the defects; S2, horizontally splicing the semantic segmentation map, the defect-free map and the defect map to obtain a spliced image; The spliced image is separated into three groups of images: semantic segmentation map, defect-free map and defect map by image segmentation, and the sizes of the three groups of images are normalized; the semantic segmentation map and defect-free map are packaged and imported as the training set of the generator by using the map function, and the defect map is imported into the discriminator at the same time; S3, input the training set into the generator, input the defect image into the discriminator, and use the sum of the cross entropy and L1Loss loss function of the generated image of the generator and the defect image of the discriminator as the objective function to optimize the generator and the discriminator. After the optimization is completed, the DPGAN model is obtained, and the generator outputs the target domain image; S4. Randomly perform semantic segmentation on the new textured fabric, concatenate the semantically segmented image with the original defect-free fabric image, input it into the DPGAN model, and generate a new texture defect fabric image.
2. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 1 is characterized in that: The specific process of generating the semantic segmentation map of fabric and defects in S1 is as follows: Modify the texture yarn defect data to JPEG format and change the size to 600×600; Circle the fabric defects and change them to white to obtain a semantic segmentation map.
3. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 1 is characterized in that: The generator in S3 is a U-net model consisting of a downsampling path and an upsampling path; The downsampling path consists of the first downsampling path and the second downsampling path, which are connected to 4×4 convolution kernels respectively, and ReLU is used for correction after convolution; The deconvolution kernel size of the upsampling path is 4×4, and ReLU is used for correction after deconvolution; the number of channels in the first 4 layers remains unchanged, and the number of channels in the subsequent layers is halved each time; When inputting, the first downsampling path and the second downsampling path input data at the same time. The first path inputs the semantic segmentation map, and the second path inputs the defect-free fabric map. The number of channels doubles after each convolution, and the number of channels remains unchanged in the last 4 layers. In the last layer of downsampling, only the downsampling operation of the first downsampling path is performed. The upsampling path deconvolves the final downsampling of the first downsampling path, and the obtained feature map is combined with the feature map of the same number of channels of the second downsampling path by concatenation, and then deconvolution is performed. After activation by the Relu function, it is combined with the feature map of the same number of channels of the second downsampling path again for deconvolution, and the above operation is repeated until the last two layers of deconvolution; no combination operation is performed in the last two layers of deconvolution operation; the last layer of deconvolution uses the tanh activation function, and the remaining layers are activated by the Relu function.
4. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 3 is characterized in that: The Dropout function is added to the first three layers of deconvolution, and the parameter is set to 0.
5.
5. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 3 is characterized in that: The downsampling path has 8 layers, and the upsampling path has 8 layers.
6. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 1 is characterized in that: The objective function of the entire network in S3 is: In the figure: x represents the semantic segmentation image, y represents the original defective fabric image, z represents the original defect-free fabric image, G(x,z) represents the target domain image generated by the G network based on the semantic segmentation image and the original defect-free fabric image; D(x,y) represents the probability that the D network judges whether the real image is real, and D(x,G(x,z)) is the probability that the D network judges whether the image generated by the G network is real; Here, x represents the semantic segmentation image, y represents the original defective fabric image, z represents the original defect-free fabric image, and G(x,z) represents the target domain image generated by the G network based on the semantic segmentation image and the original defect-free fabric image.
7. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 1 is characterized in that: The optimization operations in S3 are: The generator and discriminator are optimized using the loss function. The Adam optimizer is selected in the optimization process. The initial learning rate is 0.0001, and the random initialization method is selected for initializing the network parameters. The cycle is repeated 1000 times to obtain the optimized generator and discriminator.
8. The defective fabric generation method based on the improved pix2pixGAN network model according to claim 1 is characterized in that: In S2, the sizes of the three groups of images are normalized to 512×512.