Fabric defect image generation method based on improved StyleGAN model

Abstract

The invention discloses a fabric defect image generation method based on an improved StyleGAN model. The method comprises the following steps: acquiring an original fabric defect image by using a camera system, and preprocessing to obtain a preprocessed image data set; and establishing an improved StyleGAN model, inputting the preprocessed image data set into the model for training, carrying out iteration until the model reaches an over-fitting peak value, obtaining quality optimal image parameters of a to-be-generated fabric defect image, carrying out image synthesis, and outputting a final fabric defect image. According to the method, intelligent generation of the fabric defect image based on the improved StyleGAN is realized, the network model is simple, the number of parameters is small, the operability is high, and the quality and the precision of the generated image are excellent.

Description

Technical Field

[0001] The present invention relates to a method for generating fabric defect images, and particularly to the field of deep learning, and particularly to a method for generating fabric defect images based on an improved StyleGAN model. Background Art

[0002] Existing equipment and labor in textile companies are facing severe challenges due to the following issues. Manual fabric inspection is inefficient, labor-intensive, and has a high rate of missed inspections. Furthermore, data storage and management are difficult, with no standardized format. This creates a disconnect between the upstream and downstream supply chains, making defect detection unreliable and resource-intensive. Traditional machine vision technology is widely used in fields like 3C and automotive production. However, automated fabric defect detection using machine vision is difficult in the textile industry due to the diverse range of fabric materials, sizes, and defects. However, with the rapid development and application of artificial intelligence (AI), intelligent visual inspection of fabrics has become an inevitable trend.

[0003] High-quality images are the foundation and key to achieving high-precision visual inspection and analysis. Affected by the actual factory production environment, different production processes, textile fabrics, and yield rates, the distribution of the types and quantities of apparent defects on textiles is extremely uneven. From micron-scale threads, plying into submillimeter-scale threads, winding into centimeter-scale packages, and finally weaving into meter-sized fabrics, different types of defects can randomly appear in different locations on textiles with different geometric scales. Existing datasets fail to truly reflect actual industrial production conditions. Detection algorithms trained using existing datasets have poor generalization capabilities, limiting improvements in defect detection accuracy.

[0004] To address the above problems, there is an urgent need to develop computer image enhancement technology and intelligent synthesis technology of basic data sets of textile defect samples to ensure that there are sufficient base data samples for network training and improve the adaptability and generalization ability of the detection model under different working conditions. Summary of the Invention

[0005] In order to solve the problems existing in the background technology, the present invention provides a method for generating fabric defect images based on an improved StyleGAN model.

[0006] The technical solution adopted in the present invention is:

[0007] The fabric defect image generation method based on the improved StyleGAN model of the present invention includes:

[0008] Step 1) using a multi-light source unstructured multi-scale array camera system to collect a number of original fabric defect images and construct an original image dataset, and preprocessing the original image dataset to obtain a preprocessed image dataset.

[0009] Step 2) establishing an improved StyleGAN model, inputting the preprocessed image dataset as a training set into the improved StyleGAN model for training to obtain a newly generated fabric defect image, and continuously iterating until the improved StyleGAN model reaches an overfitting peak to obtain the optimal quality image parameters of the fabric defect image to be generated, and synthesizing the image according to the optimal quality image parameters to output the final fabric defect image.

[0010] In the step 1), the multi-light source unstructured multi-scale array camera system includes eight industrial cameras and a surface light source. The fabric with defects is rolled on the upper end of a support roller, the support roller is arranged vertically, and the lower end of the support roller is installed on a base. Four of the industrial cameras are symmetrically arranged in pairs on the upper and lower sides of the fabric, and the lenses are all vertically facing the fabric. The other four industrial cameras are symmetrically arranged in pairs on the upper and lower outer sides of the fabric, and the lenses are all facing the fabric at a 45-degree angle. The distance between each industrial camera and the fabric is the same; the surface light source is located on the upper side of the fabric, and the light covers the entire imaging range of the multi-light source unstructured multi-scale array camera system.

[0011] In step 1), the original image dataset is preprocessed. For each original fabric defect image, the original fabric defect image is unified into a preset image format and resolution, and then background noise is removed to obtain a preprocessed image. The preset image format is a format suitable for the improved StyleGAN model.

[0012] In the step 2), the improved StyleGAN model includes a mapping network and an improved synthesis network, the mapping network includes a potential space Z, a normalization operation and eight fully connected layers connected in sequence; the improved synthesis network includes a first adaptive instance normalization operation, a first 3×3 convolution layer, a second adaptive instance normalization operation, a first mirror module, an upsampling layer, a second 3×3 convolution layer, a third adaptive instance normalization operation, a third 3×3 convolution layer, a fourth adaptive instance normalization operation and a second mirror module connected in sequence, the input of the improved StyleGAN model is first processed by the mapping network and then an intermediate vector W is output; noise is added to the preset initial constant matrix and then the noise is processed together with the intermediate vector W by the first Adaptive instance normalization is performed, and the processed output is then fed into the first 3×3 convolutional layer to obtain the first convolution result. Noise is added to the first convolution result, and the result is then processed together with the intermediate vector W through a second adaptive instance normalization operation, a first mirroring module, an upsampling layer, and a second 3×3 convolutional layer, resulting in a second convolution result. Noise is added to the second convolution result, and the result is then processed together with the intermediate vector W through a third adaptive instance normalization operation and a third 3×3 convolutional layer, resulting in a third convolution result. Noise is added to the third convolution result, and the result is then processed together with the intermediate vector W through a fourth adaptive instance normalization operation and a second mirroring module, resulting in the output of the improved StyleGAN model. The added noise can be random Gaussian noise.

[0013] Improved StyleGAN model: The learning rate of the generator and discriminator is adjusted to 1e-4, the gamma value of R1 regularization is changed to 10, and the adaptive discriminator augmentation option in the network is enabled by default to perform data augmentation on the collected dataset. The adaptive discriminator augmentation (ADA) module is used to prevent the generator and discriminator from simultaneously learning the data features after the data augmentation operation, ensuring that data augmentation is only effective for the discriminator and that the subsequent training set latent vectors are not disturbed. Finally, a mirror operation module is added to the synthetic network structure of the generator.

[0014] The mirror module performs a linear transformation on the pixel values of the image generated after each iteration.

[0015] For example, a, b, x, y, u, and v in the following formula all have general meanings and have no special meaning. is (x,y) T , then the vector that is symmetric about a specific axis of symmetry is (u,ν) T , the functional relationship between b and a can be expressed as:

[0016]

[0017] Bringing in specific vector coordinates can be expressed as:

[0018]

[0019] The matrix for linear transformation φ of the above vector coordinates is:

[0020]

[0021] In the step 2), the mirror module of the improved StyleGAN model is continuously iterated, and the image similarity evaluation index FID (Fréchet Inception Distance) value between the newly generated fabric defect image and the preprocessed image of the original fabric defect image is continuously measured until the improved StyleGAN model reaches the overfitting peak, and the newly generated fabric defect image and its image parameters with the highest image similarity evaluation index FID between the preprocessed image of the original fabric defect image are obtained. The finally obtained image parameters are input into the generator as the image parameters with the best quality for image synthesis to obtain the final fabric defect image.

[0022] The beneficial effects of the present invention are:

[0023] This method realizes the intelligent generation of fabric defect images based on the improved StyleGAN model. The network model is simple, with few parameters, strong operability, and excellent generated image quality and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flowchart of the fabric defect image algorithm based on the improved StyleGAN in the embodiment;

[0025] Figure 2 This is an imaging position diagram of an industrial camera for collecting fabric defect image data in an embodiment, wherein: Figure 2 (a) is a top view of the industrial camera imaging position. Figure 2 (b) is a side view of the imaging position of the industrial camera;

[0026] Figure 3 The fabric defect image data with uniform resolution and size in the embodiment;

[0027] Figure 4 This is a fabric defect image generated based on the unimproved StyleGAN in the embodiment;

[0028] Figure 5 This is a fabric defect image generated based on the improved StyleGAN in the embodiment. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] The fabric defect image generation method based on the improved StyleGAN model of the present invention includes:

[0031] Step 1) using a multi-light source unstructured multi-scale array camera system to collect a number of original fabric defect images and construct an original image dataset, and preprocessing the original image dataset to obtain a preprocessed image dataset.

[0032] In step 1), the multi-light source unstructured multi-scale array camera system includes eight industrial cameras and a surface light source. The fabric with defects is rolled on the upper end of a support roller, the support roller is arranged vertically, and the lower end of the support roller is installed on a base. Four of the industrial cameras are symmetrically arranged in pairs on the upper and lower sides of the fabric, and the lenses are all facing vertically towards the fabric. The other four industrial cameras are symmetrically arranged in pairs on the upper and lower outer sides of the fabric, and the lenses are all facing the fabric at a 45-degree angle. The distance between each industrial camera and the fabric is the same; the surface light source is located on the upper side of the fabric, and the light covers the entire imaging range of the multi-light source unstructured multi-scale array camera system.

[0033] In step 1), the original image dataset is preprocessed. For each original fabric defect image, the original fabric defect image is unified into a preset image format and resolution, and then background noise is removed to obtain a preprocessed image. The preset image format is a format suitable for the improved StyleGAN model.

[0034] Step 2) establishing an improved StyleGAN model, inputting the preprocessed image dataset as a training set into the improved StyleGAN model for training to obtain a newly generated fabric defect image, and continuously iterating until the improved StyleGAN model reaches an overfitting peak to obtain the optimal quality image parameters of the fabric defect image to be generated, and synthesizing the image according to the optimal quality image parameters to output the final fabric defect image.

[0035] In step 2), the improved StyleGAN model includes a mapping network and an improved synthesis network, the mapping network includes a potential space Z, a normalization operation and eight fully connected layers connected in sequence; the improved synthesis network includes a first adaptive instance normalization operation, a first 3×3 convolution layer, a second adaptive instance normalization operation, a first mirror module, an upsampling layer, a second 3×3 convolution layer, a third adaptive instance normalization operation, a third 3×3 convolution layer, a fourth adaptive instance normalization operation and a second mirror module connected in sequence, the input of the improved StyleGAN model is first processed by the mapping network and then an intermediate vector W is output; noise is added to the preset initial constant matrix and then the noise is processed together with the intermediate vector W by the first adaptive The output of the first convolutional layer is processed by instance normalization, and then fed into the first 3×3 convolutional layer to obtain the first convolution result. Noise is added to the first convolutional result, and then the result and the intermediate vector W undergo a second adaptive instance normalization operation, a first mirroring module, an upsampling layer, and a second 3×3 convolutional layer, to produce the second convolution result. Noise is added to the second convolutional result, and then the result and the intermediate vector W undergo a third adaptive instance normalization operation and a third 3×3 convolutional layer, to produce the third convolution result. Noise is added to the third convolutional result, and then the result and the intermediate vector W undergo a fourth adaptive instance normalization operation and a second mirroring module, to produce the output of the improved StyleGAN model. The added noise can be random Gaussian noise.

[0036] Improved StyleGAN model: The learning rate of the generator and discriminator is adjusted to 1e-4, the gamma value of R1 regularization is changed to 10, and the adaptive discriminator enhancement option in the network is turned on by default to perform data augmentation on the collected dataset. The adaptive discriminator enhancement ADA module is used to prevent the generator and discriminator from simultaneously learning the data features after data augmentation operations, ensuring that data augmentation is only effective for the discriminator and that the subsequent training set latent vectors are not disturbed. Finally, a mirror operation module is added to the synthetic network structure of the generator.

[0037] The mirror module linearly transforms the pixel values of the image generated after each iteration.

[0038] For example, a, b, x, y, u, and v in the following formula all have general meanings and have no special meaning. is (x,y) T , then the vector that is symmetric about a specific axis of symmetry is (u,v) T , the functional relationship between b and a can be expressed as:

[0039]

[0040] Bringing in specific vector coordinates can be expressed as:

[0041]

[0042] The matrix for linear transformation φ of the above vector coordinates is:

[0043]

[0044] In step 2), the mirror module of the improved StyleGAN model is continuously iterated, and the image similarity evaluation index FID (Fréchet Inception Distance) value between the newly generated fabric defect image and the preprocessed image of the original fabric defect image is continuously measured until the improved StyleGAN model reaches the overfitting peak, and the newly generated fabric defect image and its image parameters with the highest image similarity evaluation index FID between the preprocessed image of the original fabric defect image are obtained. The finally obtained image parameters are input into the generator as the image parameters with the optimal quality for image synthesis to obtain the final fabric defect image.

[0045] like Figure 1 As shown in the figure, the working process of the improved StyleGAN network is as follows: the latent vector z of the fabric defect image dataset belonging to the latent space Z is normalized and uniformly converted into a vector with a certain size, and then enters a full mapping network composed of 8 fully connected layers for nonlinear transformation. The role of this network is to de-entangle the latent vector. The main functions are: 1. The latent vector z objectively obeys a certain probability density distribution, preventing the vectors in the low probability density area from being unable to express deep image features in the network; 2. The vectors expressing different image features themselves have a certain connection, preventing the vectors expressing different features from being entangled. The synthesis network initially has a 4*4*512 constant matrix. By performing instance normalization, upsampling, and convolution kernel operations on the image, and inputting the generated image into the mirror operation module, its pixel values are coordinate transformed, and then it is continuously iterated to gradually improve the image resolution. The final image resolution reaches 1024*1024*3; before each upsampling, the two vectors y are obtained by affine transformation using the intermediate variable W. (s,i) and y (b,i) The adaptive instance normalization operation is inserted into the modules of different resolution levels, and then the style value pairs (y (s,i) ,y (b,i) ) as the scaling and translation values of w, so that the style features can affect images of different resolutions according to the different levels of expression of image features.

[0046] The specific process of training the improved StyleGAN network to obtain new fabric defect images is as follows: the image training set first enters the mapping network, and through a series of affine transformations, the latent vector z is transformed into an intermediate vector w, and w is transformed into a style vector style, as follows:

[0047] style=(y (s,i) ,y (b,i) )

[0048] Among them, y (s,i) and y (b,i) It represents the scale and shift required for normalization of each resolution level obtained by the box transform of the intermediate vector w. The corresponding number of them is the same as the number of channels of each layer of feature map. i is the number of identifiers of the training data.

[0049] The Adaptive Instance Normalization (AdaIN) operation is as follows:

[0050]

[0051] Among them, y(s,i) and y(b,i) represent the standard deviation and mean of the feature vector of the feature map respectively; μ(x i ) and σ(x i ) represent the content graph x i The mean and standard deviation of the eigenvector.

[0052] The training set then enters the synthesis network, where each sub-network layer receives the affine transformation vectors y(s,i) and y(b,i) obtained by transforming w. Based on the StyleGAN network's division of synthetic image resolution into different levels, random Gaussian noise vectors are added or removed in stages to images at different resolution levels [4, 8], [16, 32], and [64…1024], respectively. The factors that determine the background and detail features of the final synthetic images are verified, with the affine transformation vector added first and the noise vector added later. A comprehensive comparison of the quality metrics of the generated images during training and testing reveals that adding random noise vectors to lower-resolution synthetic images (e.g., [4…32]) in the StyleGAN network is meaningless and may interfere with background generation. However, adding random noise vectors to higher-resolution synthetic images ([64…1024]) promotes the expression of detailed features and improves similarity with the original input data.

[0053] The convolutional layer dynamically adjusts feature maps of different resolutions by controlling the different levels of image features based on the affine transformation obtained by transforming the input w. The discriminator sets the number of generated samples learned at each iteration, measures the FID value of the generated image to determine the quality of the generated image, and takes the parameters of the optimal image quality stage and inputs them into the generator for image generation. The details are as follows:

[0054]

[0055] Among them, FID represents the degree of similarity between two images, x represents the real image, g represents the generated image, Tr represents the sum of the elements on the diagonal of the matrix, μ represents the mean, and σ represents the covariance.

[0056] like Figure 2 (a) and Figure 2 (b) shows the industrial camera imaging position of the multi-light source unstructured multi-scale array camera system.

[0057] Optical lens parameter configuration for a multi-light source, unstructured, multi-scale array camera system: The appropriate lens is also crucial for capturing clear, detailed, and high-quality images. The lens acts like the aperture in pinhole imaging, allowing the camera to receive more light in a shorter period of time. This light is then focused on the camera sensor to produce a clearer image. The lens features two rings: focus and aperture, which adjust the focal length and brightness to ensure a clear image. Lens selection should reference the relevant dimensional parameters of industrial cameras and meet requirements for interfaces and mounting dimensions. Industrial lens parameters primarily focus on focal length. The lens's imaging performance is related to the working distance, field of view, and sensor size.

[0058] For rolls, the sensor size of the commonly used industrial camera is 2 / 3 inches, the corresponding target surface size is 8.8mm×6.6mm, and the diagonal is 11mm, so Sw is 6.6mm. In the actual acquisition process, the distance between the industrial lens and the end face of the chemical fiber cake is required to be 50cm, so W is 500mm. The width of the roll field of view FOVw is 440mm. Combined with the focal length size of the industrial lens, the focal length is ultimately set to 8mm. Finally, considering the compatibility of the lens and camera, the lens size must be greater than or equal to the size of the camera sensor chip, etc., and the model of the industrial lens is determined.

[0059] For cloth, since its distance from the camera is fixed, the distance from the camera target plane to the cloth is about 430mm, and the camera resolution DPI is between 160-180, a 35mm manual fixed-focus lens is selected for the camera lens.

[0060] Light source design for a multi-light source, unstructured, multi-scale array camera system: Most industrial image acquisition sites suffer from low lighting conditions. Textile processing workshops are large, relatively dim, and often feature relatively few lights. Consequently, captured textile images are blurry and subject to significant noise interference, making it difficult to clearly discern surface details. This significantly impacts the accuracy of subsequent detection algorithms. Using appropriate light sources to illuminate the textile image acquisition process is a crucial component of the overall acquisition process. Therefore, light source design focuses on two key aspects: the luminous medium and the shape of the light source.

[0061] (1) Luminescent medium: Since LED light sources have the characteristics of long service life, diverse light source shapes, diverse wavelength selection, and stable luminescence, and have little impact on image quality during long-term use, the present invention uses white LED light sources as imaging light sources.

[0062] (2) Light source shape: The shape of the light source is mainly determined by the shape or distribution of the object being inspected. For yarns and fabrics with large aspect ratios and high width-to-thickness ratios, choosing a linear light source can better cover the entire imaging range and ensure the consistency of image sampling.

[0063] (3) Lighting method: Lighting design also includes establishing a reasonable lighting method to avoid the influence of ambient light and maximize the use of light sources. Considering the low light transmittance of textiles, the present invention mainly adopts forward bright field lighting to overcome the disadvantages of the coaxial light source being too large, which will affect the shooting effect, and the reflection problem caused by the strong light of forward dark field lighting.

[0064] The fabric defect image dataset is a 512*512 square image with equal length and width, such as Figure 3 As shown, there are several fabric defect images of uniform resolution and size in the fabric defect image dataset.

[0065] like Figure 4 and Figure 5 The following figures show fabric defect images generated using the unmodified StyleGAN model and the improved StyleGAN model, respectively. The improved network model, for the fabric defect dataset studied in this project, can retain more of the defect image background and maximize the preservation of fabric defect detail, resulting in images with a similar overall style and greater fidelity. Overall, the improved network model is simple, has a small number of parameters, is highly operational, and produces images of excellent quality and accuracy.

[0066] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating fabric defect images based on an improved StyleGAN model, characterized in that: include: Step 1) using a multi-light source unstructured multi-scale array camera system to collect a number of original fabric defect images and construct an original image dataset, and preprocessing the original image dataset to obtain a preprocessed image dataset; Step 2) establishing an improved StyleGAN model, inputting the preprocessed image dataset as a training set into the improved StyleGAN model for training to obtain a newly generated fabric defect image, and continuously iterating until the improved StyleGAN model reaches an overfitting peak to obtain the optimal quality image parameters of the fabric defect image to be generated, and synthesizing the image according to the optimal quality image parameters to output the final fabric defect image.

2. The method for generating fabric defect images based on the improved StyleGAN model according to claim 1, characterized in that: In the step 1), the multi-light source unstructured multi-scale array camera system includes eight industrial cameras and a surface light source. The fabric with defects is rolled on the upper end of a support roller, the support roller is arranged vertically, and the lower end of the support roller is installed on a base. Four of the industrial cameras are symmetrically arranged in pairs on the upper and lower sides of the fabric, and the lenses are all vertically facing the fabric. The other four industrial cameras are symmetrically arranged in pairs on the upper and lower outer sides of the fabric, and the lenses are all facing the fabric at a 45-degree angle. The distance between each industrial camera and the fabric is the same; the surface light source is located on the upper side of the fabric, and the light covers the entire imaging range of the multi-light source unstructured multi-scale array camera system.

3. The method for generating fabric defect images based on the improved StyleGAN model according to claim 1, characterized in that: In the step 1), the original image data set is preprocessed. For each original fabric defect image, the original fabric defect image is unified into a preset image format and resolution, and then background noise is removed to obtain a preprocessed image.

4. The method for generating fabric defect images based on the improved StyleGAN model according to claim 1, characterized in that: In the step 2), the improved StyleGAN model includes a mapping network and an improved synthesis network, the mapping network includes a potential space Z, a normalization operation, and eight fully connected layers connected in sequence; the improved synthesis network includes a first adaptive instance normalization operation, a first convolutional layer, a second adaptive instance normalization operation, a first mirror module, an upsampling layer, a second convolutional layer, a third adaptive instance normalization operation, a third convolutional layer, a fourth adaptive instance normalization operation, and a second mirror module connected in sequence, and the input of the improved StyleGAN model is first processed by the mapping network and then outputs an intermediate vector W; After adding noise to the preset initial constant matrix, the matrix is processed together with the intermediate vector W through the first adaptive instance normalization operation. The processed output is input into the first convolution layer to obtain the first convolution result. After adding noise to the first convolution result, the matrix is processed together with the intermediate vector W through the second adaptive instance normalization operation, the first mirror module, the upsampling layer, and the second convolution layer to output the second convolution result. After adding noise to the second convolution result, the matrix is processed together with the intermediate vector W through the third adaptive instance normalization operation and the third convolution layer to output the third convolution result. After adding noise to the third convolution result, the matrix is processed together with the intermediate vector W through the fourth adaptive instance normalization operation and the second mirror module to output as the output of the improved StyleGAN model.

5. The method for generating fabric defect images based on the improved StyleGAN model according to claim 4, characterized in that: The mirror module performs a linear transformation on the pixel values of the image generated after each iteration.

6. The method for generating fabric defect images based on the improved StyleGAN model according to claim 4, characterized in that: In the step 2), the mirror module of the improved StyleGAN model is continuously iterated, and the image similarity evaluation index FID value between the newly generated fabric defect image and the preprocessed image of the original fabric defect image is continuously measured until the improved StyleGAN model reaches an overfitting peak, and a newly generated fabric defect image and its image parameters with the highest image similarity evaluation index FID between the preprocessed image of the original fabric defect image and the improved StyleGAN model are obtained, and the image parameters finally obtained are used as the image parameters with the optimal quality.

Images