Cloth defect image generation system and method based on improved variational autoencoder network
Through the improved variational autoencoder network, combined with the feature representation of GAN and VAE, high-quality cloth defect images are generated, which solves the problems of instability and blurred image generation in the prior art and provides better data support for deep learning Model training.
Patent Information
- Application Number
- CN202111198444.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-10-14
AI Technical Summary
Existing image generation methods, such as GAN and VAE, are difficult to generate high-quality fabric defect images with three types of holes, stains, and loose meridians, and there are problems of training instability and image blur, and lack sufficient data support depth. Learn model training.
The improved variational autoencoder network designs a kind of feature metric by using the learning feature representation in the GAN discriminator as the VAE reconstruction target, replacing the pixel similarity metric of the VAE as the feature metric learned by the discriminator, combining the resnet18 network and the discriminator network. New VAE-GAN network structure for generating higher quality cloth defect images.
The problem of lack of sufficient data to support deep learning model training is effectively solved, the generated image quality is significantly improved, the problems of training instability and image blur are solved, and higher quality cloth defect image generation is achieved.
Smart Images

Figure CN114067168B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image generation, and in particular to a cloth defect image generation system and method based on an improved variational autoencoder network. Background Art
[0002] At present, China's clothing industry has a huge market. In production, whether the cloth has defects is an important criterion for testing cloth quality. When we want to use deep learning technology to solve problems such as cloth defect detection and defect classification, we find that there is not enough image data with three types of cloth defects, holes, stains, and loose warps, to support model training.
[0003] At present, many image generation methods have been proposed, which use generative adversarial networks (GAN) (e.g. Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks [J]. arXiv: 1406.2661 [cs, stat], 2014.), variational autoencoders (VAE) (e.g. Kingma DP, Welling M. Auto-Encoding Variational Bayes [J]. arXiv: 1312.6114 [cs, stat], 2014.), pixelCNN, pixelRNN and other technologies. Among them, compared with the traditional model, GAN has two different networks instead of a single network, and the training method adopts adversarial training, and finally obtains a stable model that can generate sufficiently realistic images. However, the GAN network has problems such as unstable training, gradient disappearance, and mode collapse; VAE adds constraints to the encoder based on the standard autoencoder (AE), forcing it to generate latent variables that obey the unit Gaussian distribution, sample from the unit Gaussian distribution, and then pass it to the decoder to generate new images, but the image effect generated by VAE is often slightly blurred. Based on this, it is necessary to develop a method for stable image generation to generate high-quality images with three types of cloth defects: holes, stains, and loose warps. Summary of the invention
[0004] In order to solve the problem that there is not enough image data of three types of cloth defects, namely holes, stains and loose warps, to support the training of deep learning models, the present invention provides a cloth defect image generation system and method based on an improved variational autoencoder network, which improves the variational autoencoder network using the discriminator network in a generative adversarial network, uses the learned feature representation in the GAN discriminator as the basis for the VAE reconstruction target, replaces the pixel similarity metric of the VAE with the feature metric learned by the discriminator, and improves the quality of the generated image.
[0005] On the one hand, the present invention provides a cloth defect image generation system based on an improved variational autoencoder network, comprising: a variational autoencoder network and a discriminant network, wherein the variational autoencoder network is divided into an encoder network and a decoder network;
[0006] The encoder network is used to encode the real target image into a normal distribution q(z|x) of the latent space variable X;
[0007] The decoder network is used to generate new target images by sampling latent space variables ~X from the normal distribution q(z|x);
[0008] The discriminator network is used to measure the similarity between the generated target image and the real target image, calculate the adversarial loss, pass the adversarial loss to the encoding network and the decoding network, and replace the pixel-based reconstruction metric in the variational autoencoder network with the feature metric represented in the discriminator network.
[0009] Furthermore, the encoder network includes a resnet18 network; the resnet18 network includes, from shallow to deep, a Conv2d layer, a maximum pooling layer, 8 residual blocks and an average pooling layer connected in sequence; the 8 residual blocks are connected in series in sequence;
[0010] The Conv2d layer consists of a convolution layer with a convolution kernel size of 7×7 and a step size of 2, a normalization layer and an activation function ReLU; the convolution kernel size of the maximum pooling layer is 3×3 and the step size is 2; the convolution kernel size of the average pooling layer is 1×1; the 1st and 2nd residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 64 output channels; the 3rd and 4th residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 128 output channels; the 5th and 6th residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 256 output channels; the 7th and 8th residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 512 output channels.
[0011] Furthermore, the decoder network comprises 6 layers, the first layer comprises a deconvolution layer with a convolution kernel size of 4×4 and an activation function ReLU; the second to fifth layers each comprise a deconvolution layer with a convolution kernel size of 4×4, a normalization layer and an activation function ReLU; the sixth layer comprises a deconvolution layer with a convolution kernel size of 4×4 and an activation function Tanh; the number of output channels of the first to sixth layers are 512, 384, 192, 96, 64, and 3, respectively.
[0012] Furthermore, the discriminator network comprises 6 layers, the first layer includes a convolution layer with a convolution kernel size of 4×4 and a step size of 2, and an activation function LeakyReLU; the second to fifth layers are each composed of a convolution layer with a convolution kernel size of 4×4 and a step size of 2, a normalization layer, and an activation function ReLU; the sixth layer is composed of a convolution layer with a convolution kernel size of 4×4 and a step size of 1, and an activation function Sigmoid.
[0013] On the other hand, the present invention provides a method for generating a cloth defect image based on an improved variational autoencoder network, comprising:
[0014] Step 1: Construct the above-mentioned cloth defect image generation system based on the improved variational autoencoder network;
[0015] Step 2: Filter the cloth defect image data with holes, stains or loose warp defects as the training set, and scale and crop each image to 256×256 pixels;
[0016] Step 3: defining a loss function of the cloth defect image generation system;
[0017] Step 4: Initialize the cloth defect image generation system;
[0018] Step 5: Using the Adam algorithm to train the cloth defect image generation system using the training set;
[0019] Step 6: Use the trained cloth defect image generation system to generate cloth defect images with holes, stains or loose warp defects.
[0020] Furthermore, the loss function of the cloth defect image generation system includes the regularization prior loss L of the variational autoencoder network prior , the adversarial loss L between the variational autoencoder network and the discriminator network GAN , the feature loss of the lth layer of the discriminator network replacing the pixel loss
[0021] Among them, the regularized prior loss L of the variational autoencoder network prior It is defined as the KL divergence loss between the normal distribution q(z|x) that transforms the target image into the latent space variable X and the given normal distribution p(z) constraint:
[0022] L prior =D KL (q(z|x)||p(z))
[0023] Adversarial loss L GAN and the feature loss of the lth layer of the discriminator network They are defined as:
[0024] L GAN =log(Dis(x))+log(1-Dis(Gen(z)))
[0025]
[0026] Among them, D KL Represents the KL divergence loss function; Dis represents the discriminator function, which is used to judge the authenticity of the picture input to the discriminator. If it is true, the output is 1, otherwise it outputs 0; Gen represents the variational autoencoder. The encoder in the variational autoencoder encodes the input real picture into a latent space variable, and the decoder in the variational autoencoder decodes the latent space variable into a new picture as the input of the discriminator; E represents the expected value of the distribution function.
[0027] Furthermore, step 5 specifically includes: in the training process, updating the system parameters by iteration until the system converges; wherein, in each iteration process, the process of updating the system parameters is specifically as follows:
[0028] Calculate the loss of the encoder network and decoder network through the loss function;
[0029] According to the loss obtained, the increment ΔW of the convolutional layer weight in the convolutional layer is calculated using the principle of the back propagation algorithm and the gradient descent algorithm. (i) ,implement Update the convolution kernel parameters;
[0030] Among them, W (i) represents the convolution kernel parameters of the convolution layer after the i-th iteration, is the parameter update amount calculated by the back propagation algorithm and the gradient descent algorithm at the i-th iteration, is the step length.
[0031] Beneficial effects of the present invention:
[0032] The present invention uses a discriminant network in a generative adversarial network to improve a variational autoencoder network. Aiming at the problem that there is not enough image data of three types of cloth defects, namely, holes, stains and loose warps, to support the training of a deep learning model, a cloth defect image generation system based on an improved variational autoencoder network and its related parameter configuration are designed, which specifically includes: using GAN to improve VAE to form a new VAE-GAN network structure; designing an improved variational autoencoder network that can generate higher quality images of three types of cloth defects, namely, holes, stains and loose warps; and effectively solving the problem that there is not enough image data of three types of cloth defects, namely, holes, stains and loose warps, to support the training of a deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1The overall structure diagram of the cloth defect image generation system based on the improved variational autoencoder network provided by the embodiment of the present invention;
[0034] Figure 2 The overall structure diagram of the variational autoencoder network provided by the embodiment of the present invention;
[0035] Figure 3 The structure corresponding to the encoding network of the variational autoencoder network provided in the embodiment of the present invention
[0036] Figure 4 A structural diagram corresponding to a decoding network of a variational autoencoder network provided in an embodiment of the present invention;
[0037] Figure 5 A structural diagram corresponding to the discriminant network provided in an embodiment of the present invention;
[0038] Figure 6 A schematic flow chart of a method for generating cloth defect images based on an improved variational autoencoder network provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0040] like Figure 1 As shown, an embodiment of the present invention provides a cloth defect image generation system based on an improved variational autoencoder network, including a variational autoencoder network Generator and a discriminator network Discriminator, wherein the variational autoencoder network is divided into an encoder network Encoder and a decoder network Decoder;
[0041] The encoder network is used to encode the real target image into a normal distribution q(z|x) of the latent space variable X;
[0042] The decoder network is used to generate new target images by sampling latent space variables ~X from the normal distribution q(z|x);
[0043] The discriminator network is used to measure the similarity between the generated target image and the real target image, calculate the adversarial loss, pass the adversarial loss to the encoding network and the decoding network, and replace the pixel-based reconstruction metric in the variational autoencoder network with the feature metric represented in the discriminator network.
[0044] As an implementation method, Figure 2 As shown, the encoder network includes a resnet18 network; the resnet18 network includes a Conv2d layer, a maximum pooling layer, 8 residual blocks and an average pooling layer connected in sequence from shallow to deep; the 8 residual blocks are connected in series in sequence;
[0045] The Conv2d layer consists of a convolution layer with a convolution kernel size of 7×7 and a step size of 2, a normalization layer and an activation function ReLU; the convolution kernel size of the maximum pooling layer is 3×3 and the step size is 2; the convolution kernel size of the average pooling layer is 1×1; the 1st and 2nd residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 64 output channels; the 3rd and 4th residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 128 output channels; the 5th and 6th residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 256 output channels; the 7th and 8th residual blocks both contain 2 layers of convolution with a convolution kernel size of 3×3 and 512 output channels.
[0046] As an implementation method, Figure 3 As shown, the decoder network includes 6 layers, the first layer includes a deconvolution layer with a convolution kernel size of 4×4 and an activation function ReLU, and the structure is expressed as UpConv=[ConvTranspose 4×4 -ReLU]; the second to fifth layers all contain a deconvolution layer with a convolution kernel size of 4×4, a normalization layer, and an activation function ReLU, and the structure is expressed as UpConv=[ConvTranspose 4×4 -BN-ReLU]; the sixth layer contains a deconvolution layer with a convolution kernel size of 4×4 and an activation function Tanh, Up Conv = [ConvTranspose 4×4 -Tanh]; the number of output channels from the first to the sixth layers are 512, 384, 192, 96, 64, and 3 respectively.
[0047] The variational autoencoder network in the embodiment of the present invention uses the resnet18 network to extract the features of the image, and uses the idea of the residual block to solve the problem of gradient disappearance caused by the network being too deep. After the decoding network, ~X is sampled from the normal distribution q(z|x) to generate a new target image.
[0048] As an implementation method, Figure 4As shown in the figure, the discriminator network includes 6 layers, the first layer includes a convolution layer with a convolution kernel size of 4×4 and a step size of 2, and an activation function LeakyReLU; the second to fifth layers are each composed of a convolution layer with a convolution kernel size of 4×4 and a step size of 2, a normalization layer, and an activation function ReLU; the sixth layer is composed of a convolution layer with a convolution kernel size of 4×4 and a step size of 1, and an activation function Sigmoid.
[0049] In the embodiment of the present invention, the discriminator network uses the convolution layer to extract the features of the image, and uses the Sigmoid function to determine whether the input image is true or false. The feature loss of the lth layer of the discriminator network and the discrimination result are fed back to the variational autoencoder network. The variational autoencoder network is retrained to generate images according to the feedback results until the model is stable.
[0050] The present invention also provides a method for generating a cloth defect image based on an improved variational autoencoder network, which adopts the cloth defect image generation system in the above embodiment and specifically includes the following steps:
[0051] S101: constructing a cloth defect image generation system based on an improved variational autoencoder network, such as the cloth defect image generation system in the above embodiment;
[0052] S102: Screening cloth defect image data with holes, stains or loose warp defects as a training set, and scaling and cropping each image to 256×256 pixels;
[0053] S103: defining a loss function of the cloth defect image generation system;
[0054] Specifically, in the embodiment of the present invention, the loss function of the cloth defect image generation system includes the regularized prior loss L of the variational autoencoder network: prior , the adversarial loss L between the variational autoencoder network and the discriminator network GAN , the feature loss of the lth layer of the discriminator network replacing the pixel loss This is because the pixel-based loss is not well suited for image data, so a higher-level and sufficiently invariant representation of the image is used to measure the similarity of the images, and the feature loss learned by the l-th layer of the discriminator is extracted as the loss for generating images.
[0055] Among them, the regularized prior loss L of the variational autoencoder network prior It is defined as the KL divergence loss between the normal distribution q(z|x) that transforms the target image into the latent space variable X and the given normal distribution p(z) constraint:
[0056] L prior =D KL (q(z|x)||p(z))
[0057] Adversarial loss L GAN and the feature loss of the lth layer of the discriminator network They are defined as:
[0058] L GAN =log(Dis(x))+log(1-Dis(Gen(z)))
[0059]
[0060] Among them, D KL Represents the KL divergence loss function; Dis represents the discriminator function, which is used to judge the true or false of the picture that the function will input to the discriminator. If it is true, the output is 1, otherwise it outputs 0; Gen represents the variational autoencoder. The encoder in the variational autoencoder encodes the input real picture into a latent space variable, and the decoder in the variational autoencoder decodes the latent space variable into a new picture as the input of the discriminator; E represents the expected value of the distribution function. The larger the expected value, the smaller the loss of the function.
[0061] S104: Initializing the cloth defect image generation system, including: for the convolution layer in the convolution network, initializing the convolution kernel parameters using the Xavier method.
[0062] S105: using the Adam algorithm and the training set to train the cloth defect image generation system;
[0063] Specifically, the image generated by decoding the latent space variable ~X and the real image generated by decoding the latent space variable X are sent to the discriminator network to calculate the adversarial loss. The adversarial loss measures the quality of the image generated by decoding the latent space variable ~X, and the result is fed back to the variational autoencoder network;
[0064] During the training process, the system parameters are updated iteratively until the system converges. In each iteration, the process of updating the system parameters is as follows:
[0065] Calculate the loss of the encoder network and decoder network through the loss function;
[0066] According to the loss obtained, the increment ΔW of the convolutional layer weight in the convolutional layer is calculated using the principle of the back propagation algorithm and the gradient descent algorithm. (i) ,implement Update the convolution kernel parameters;
[0067] Among them, W (i) represents the convolution kernel parameters of the convolution layer after the i-th iteration, is the parameter update amount calculated by the back propagation algorithm and the gradient descent algorithm at the i-th iteration, is the step length. Also called the learning rate, The size of determines the speed and degree of convergence during network training. When it is relatively large, the network can converge quickly during training, but it may not reach the global optimal value. When it is relatively small, the network training converges slowly, but it can often reach the global optimal value. In this preferred solution, the iterative parameter update algorithm uses Adam, and the learning rate is initially set to 0.001.
[0068] S106: Generate a cloth defect image with holes, stains or loose warp defects using the trained cloth defect image generation system.
[0069] Specifically, the cloth defect images with holes, stains or loose warp defects generated by the cloth defect image generation system trained in the present invention can be compared with the images generated by VAE and GAN, and the Cosine similarity and Mean squared error values of the generated images can be used to evaluate the quality of the images generated by the method of the present invention, thereby measuring the performance of the cloth defect system provided by the present invention.
[0070] In order to verify the effectiveness of the cloth defect image generation system and method provided by the present invention, the present invention also provides the following experiment.
[0071] In this experiment, Pytorch-1.6 was used to implement the system network structure, and the LFW test set was used to test the system network performance. The Adam algorithm was used to train the network, where the hyperparameter learning rate of the Adam algorithm was initially set to 0.0002, and the learning rate was automatically adjusted using dynamic learning. All convolutional kernels were initialized using the Xavier initialization method in Pytorch, and the bias term was set to 0.0. In each iteration, any 4 images in the training set were grouped into a batch and input into the network for training. This training was repeated many times until the network converged. After the training, the evaluation indicators of image attribute similarity, Cosine similarity and Mean squared error, were used to express the performance of the networks such as VAE and GAN, which were divided using the same data set and tested according to their original configuration.
[0072] As shown in Table 1, the Cosine similarity and Mean squared error values of the generated cloth defect images were tested under VAE, GAN and the system and method of the present invention respectively. It can be seen from Table 1 that the cloth defect system and method proposed in the present invention perform better than VAE and GAN on the data set.
[0073] Table 1
[0074] Model Cosine similarity Mean squared error test_set 0.9193 14.1987 VAE 0.9030 27.59±1.42 GAN 0.8892 27.89±3.0 VAEGAN 0.9114 22.39±1.16
[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A cloth defect image generation system based on an improved variational autoencoder network, characterized in that: It includes a variational autoencoder network and a discriminant network, wherein the variational autoencoder network is divided into an encoder network and a decoder network; The encoder network is used to encode the real target image into a normal distribution q(z|x) of the latent space variable X; Decoder network, used to sample latent space variables from a normal distribution q(z|x) ~ X generates a new target image; The discriminator network is used to measure the similarity between the generated target image and the real target image, calculate the adversarial loss, pass the adversarial loss to the encoding network and the decoding network, and replace the pixel-based reconstruction metric in the variational autoencoder network with the feature metric represented in the discriminator network; The encoder network includes a resnet18 network; the resnet18 network includes a Conv2d layer, a maximum pooling layer, 8 residual blocks and an average pooling layer connected in sequence from shallow to deep; the 8 residual blocks are connected in series in sequence; The Conv2d layer consists of a convolution layer with a convolution kernel size of 7×7 and a stride of 2, a normalization layer, and an activation function ReLU; the convolution kernel size of the maximum pooling layer is 3×3 and the stride is 2; the convolution kernel size of the average pooling layer is 1×1; the first and second residual blocks each contain two layers of convolution with a convolution kernel size of 3×3 and an output channel of 64; the third and fourth residual blocks each contain two layers of convolution with a convolution kernel size of 3×3 and an output channel of 128; the fifth and sixth residual blocks each contain two layers of convolution with a convolution kernel size of 3×3 and an output channel of 256; the seventh and eighth residual blocks each contain two layers of convolution with a convolution kernel size of 3×3 and an output channel of 512; The decoder network comprises 6 layers, the first layer comprises a deconvolution layer with a convolution kernel size of 4×4 and an activation function ReLU; the second to fifth layers each comprise a deconvolution layer with a convolution kernel size of 4×4, a normalization layer and an activation function ReLU; the sixth layer comprises a deconvolution layer with a convolution kernel size of 4×4 and an activation function Tanh; the number of output channels of the first to sixth layers are 512, 384, 192, 96, 64, and 3 respectively; The discriminator network comprises 6 layers, the first layer includes a convolution layer with a convolution kernel size of 4×4 and a step size of 2 and an activation function LeakyReLU; the second to fifth layers are each composed of a convolution layer with a convolution kernel size of 4×4 and a step size of 2, a normalization layer and an activation function ReLU; the sixth layer is composed of a convolution layer with a convolution kernel size of 4×4 and a step size of 1 and an activation function Sigmoid.
2. A cloth defect image generation method based on an improved variational autoencoder network, characterized in that: include: Step 1: constructing a cloth defect image generation system based on an improved variational autoencoder network as described in claim 1; Step 2: Filter the cloth defect image data with holes, stains or loose warp defects as the training set, and scale and crop each image to 256×256 pixels; Step 3: defining a loss function of the cloth defect image generation system; Step 4: Initialize the cloth defect image generation system; Step 5: Using the Adam algorithm to train the cloth defect image generation system using the training set; Step 6: Use the trained cloth defect image generation system to generate cloth defect images with holes, stains or loose warp defects.
3. The cloth defect image generation method based on the improved variational autoencoder network according to claim 2 is characterized in that: The loss function of the cloth defect image generation system includes the regularized prior loss L of the variational autoencoder network prior , the adversarial loss L between the variational autoencoder network and the discriminator network GAN , the feature loss of the lth layer of the discriminator network replacing the pixel loss Among them, the regularized prior loss L of the variational autoencoder network prior It is defined as the KL divergence loss between the normal distribution q(z|x) that transforms the target image into the latent space variable X and the given normal distribution p(z) constraint: L prior =D KL (q(z|x)||p(z)) Adversarial loss L GAN and the feature loss of the discriminator network layer l They are defined as: L GAN =log(Dis(x))+log(1-Dis(Gen(z))) Among them, D KL Represents the KL divergence loss function; Dis represents the discriminator function, which is used to judge the authenticity of the picture input to the discriminator. If it is true, it outputs 1, otherwise it outputs 0; Gen represents the variational autoencoder. The encoder in the variational autoencoder encodes the input real picture into a latent space variable, and the decoder in the variational autoencoder decodes the latent space variable into a new picture as the input of the discriminator; E represents the expected value of the distribution function.
4. The cloth defect image generation method based on the improved variational autoencoder network according to claim 2 is characterized in that: Step 5 specifically includes: during the training process, updating the system parameters through iteration until the system converges; wherein, during each iteration, the process of updating the system parameters is specifically as follows: Calculate the loss of the encoder network and decoder network through the loss function; According to the loss obtained, the increment ΔW of the convolutional layer weight in the convolutional layer is calculated using the principle of the back propagation algorithm and the gradient descent algorithm. (i) ,implement Update the convolution kernel parameters; Among them, W (i) represents the convolution kernel parameters of the convolution layer after the i-th iteration, is the parameter update amount calculated by the back propagation algorithm and the gradient descent algorithm at the i-th iteration, is the step length.
