ACGAN based on multi-class attention mechanism for generating car sewing line images

Abstract

The application discloses an ACGAN sewing line image generation method based on a multi-category attention mechanism, which adopts an ACGAN improved model to generate sewing line defect image samples to solve the imbalance problem of the sewing line defect data set, and the improvement of the original ACGAN lies in that: the multi-category attention mechanism is introduced so that the ACGAN can pay attention to the features of the sewing line defect image samples from multiple angles, and the effect of the ACGAN improved model is enhanced; the residual block is introduced to solve the gradient disappearance problem, improve the learning ability of the ACGAN improved model, and accelerate the convergence speed of the ACGAN improved model; the loss function of the ACGAN improved model is improved, the loss function of the sample category and the loss function of the sample true and false are improved, the convergence of the ACGAN improved model is accelerated, and the performance of the ACGAN improved model is improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning, and more particularly to an ACGAN-based method for generating stitch images based on a multi-class attention mechanism. Background Technology

[0002] After the plastic particles are produced, they need to be packaged in bags, which are then sealed with stitching. The stitching thread is a thick, white thread sewn by a sewing machine, similar to the thread used for sealing rice bags. After the stitching is complete, a layer of binding paper is glued on with hot melt adhesive to improve the sealing quality.

[0003] The sewing process is done by sewing machines on an assembly line, which inevitably leads to some quality problems. Serious quality problems can cause packaging damage and leakage of plastic particles. This not only causes inconvenience but also lowers consumers' perception of the product's quality.

[0004] To address the defect identification problem, industrial cameras were used to capture images of the stitching lines, and deep learning algorithms were used for defect identification and classification. After collecting a large number of stitching line defect samples and classifying them, the factory categorized the defects into 14 classes according to actual business needs. However, the number of defect samples in each class varied significantly, resulting in a dataset imbalance problem. For example, creases (without damaged packaging) accounted for 30% of the total defect samples, while uncut binding paper accounted for less than 2%. This dataset imbalance caused the classifier to favor the majority class, resulting in poor prediction performance for the minority class. The model overfitted to the majority class and underfitted to the minority samples.

[0005] Traditional approaches to addressing imbalanced datasets include undersampling, oversampling, and traditional image data augmentation. Undersampling removes a large number of samples from the majority class, making the number of samples in each class similar. This reduces the data the classifier learns, resulting in missing information. Oversampling repeatedly samples samples from the minority class, making the number of samples in each class similar. This causes the classifier to repeatedly learn from the minority samples, leading to overfitting and difficulty in applying learned classification experience to unseen samples. Traditional image data augmentation uses geometric and color transformations to increase the number of samples in the minority class by cropping, rotating, scaling, changing colors, and adding noise. However, these transformations are based on certain rules and may fail to capture complex patterns and variations in a specific dataset. Furthermore, rotation and cropping may result in partial loss of image information.

[0006] In recent years, with the development of deep learning, Auxiliary Classifier Generative Adversarial Networks (ACGANs) can generate more diverse and realistic images. However, when applied to generate samples of seam defects, problems arise such as gradient discrepancies, making it difficult to capture key features.

[0007] Therefore, a suitable method is needed to solve the problem of imbalanced datasets. This invention focuses on sample generation for images with imbalanced seam defect images. Summary of the Invention

[0008] The purpose of this invention is to address the problem of imbalanced datasets of seam defect image samples in existing technologies by providing an ACGAN-based method for generating seam defect images based on a multi-class attention mechanism.

[0009] The objective of this invention is achieved through the following technical solution: a method for generating stitch images based on a multi-class attention mechanism using ACGAN, comprising the following steps:

[0010] (1) Collect images of the sewing defect samples and crop the collected sewing defect images to the same size;

[0011] (2) Construct a dataset based on the cropped sewing defect images obtained in step (1), and randomly divide the dataset into a training set and a test set according to the proportion; wherein, the dataset includes sewing defect images and their corresponding category labels, and the proportion of sewing defect images of each category in the training set and the test set is the same.

[0012] (3) Improve the ACGAN model by using a multi-class attention mechanism and residual module for optimization to obtain the improved ACGAN model and its loss function; the improved ACGAN model includes a generator and a discriminator.

[0013] (4) Use the training set constructed in step (2) to iteratively train the ACGAN improved model obtained in step (3), and adjust the parameters of the generator and discriminator according to the loss function of the ACGAN improved model to obtain the trained ACGAN improved model.

[0014] (5) Design various experiments to test the quality of the ACGAN improved model trained in step (4) in order to fine-tune the parameters of the ACGAN improved model and obtain the final ACGAN improved model.

[0015] Furthermore, the input to the generator is a vector formed by concatenating noise and category labels. This vector passes through a fully connected layer, two upsampling residual modules, and a multi-class attention module in sequence. The multi-class attention module includes a compression and activation network module and a non-local neural network module. The output of the multi-class attention module is concatenated according to the channel dimension and then passed through a fully connected layer, a deconvolution layer, and a tanh activation function in sequence to output the generated stitching defect image.

[0016] The upsampling residual module includes batch normalization, a linear rectified function, a convolutional layer, residual connections, and an upsampling layer.

[0017] Furthermore, the input to the discriminator is a real image of a stitching defect and an image of a stitching defect generated by the generator. The discriminator includes two downsampling residual modules, a multi-class attention module, and two parallel fully connected layers. The output of one fully connected layer is then processed by a sigmoid activation function to output the true / false probability, and the output of the other fully connected layer is then processed by a softmax activation function to output the class probability. The multi-class attention module includes a compression and activation network module and a non-local neural network module.

[0018] The downsampling residual module includes a convolutional layer, batch normalization, a linear rectified function, residual connections, and a downsampling layer.

[0019] Furthermore, the loss function of the improved ACGAN model includes the log-likelihood loss LC′ for classification prediction and the loss function L for data authenticity discrimination. S ′;

[0020] Log-likelihood loss L for classification prediction C The expression for ′ is:

[0021] L C ′=E[w(c)·log P(C=c|X real )]+E[w(c)·log P(C=c|X fake )]

[0022] Where E represents the expected value; log represents the logarithmic function; X represents the image sample of the stitching defect, X real X represents a sample image of a real seam defect. fake This represents a sample image of a stitching defect generated by the generator; P represents the probability, P(C=c|X real P(C = c|X) represents the probability that a real image sample with a defective stitching is classified as the corresponding true category label. fake ) represents the probability that the generated image sample of the stitching defect is classified as the corresponding category label generated by the generator; w(c) represents the degree of influence of category c on the bag breakage and leakage.

[0023] The loss function L for judging the truth or falsehood of data S The expression for ′ is:

[0024]

[0025] Where S represents the discrimination state of the stitching defect image sample, S = real means the stitching defect image sample is judged as a real sample, S = fake means the stitching defect image sample is judged as a sample generated by the generator, and λ is the penalty coefficient. Linear interpolation between real and generated stitch defect image samples. This represents the distribution of linear interpolation between real and generated image samples of stitching defects. It is the gradient, D is the discriminator, and ||·||2 is the 2-norm.

[0026] Furthermore, step (4) includes the following sub-steps:

[0027] (4.1) Randomly extract real sewing defect image samples x and their corresponding real category labels c1 from the training set, randomly sample Gaussian noise z from the normal distribution, and randomly generate corresponding category labels c2 for each Gaussian noise.

[0028] (4.2) The noise z and the class label c2 are concatenated and input into the generator of the ACGAN improved model to obtain the generated stitch defect image sample;

[0029] (4.3) Input the generated sewing defect image samples and the real sewing defect image samples into the discriminator of the ACGAN improved model to obtain the discrimination probability and class probability;

[0030] (4.4) Calculate the discriminator's loss function Loss1 = L based on the loss function of the ACGAN improved model. C ′+L S ′, L C ′ is a measure of the probability that data belongs to a specific category, L S ′ is a measure of the likelihood that the data is true.

[0031] (4.5) The training objective of the discriminator is to maximize Loss1. The parameters of the discriminator are updated by backpropagation based on the discriminator's loss function.

[0032] (4.6) Randomly sample Gaussian noise z with batch size n from a normal distribution, and randomly generate class labels c2 with batch size n;

[0033] (4.7) After concatenating the noise z and the category label c2, input them into the generator to obtain the generated sewing defect image sample. Input the generated sewing defect image sample into the discriminator updated in step (4.5) to obtain the discrimination probability and the category probability.

[0034] (4.8) Calculate the generator's loss function Loss2 = L based on the loss function of the ACGAN improved model. C '-L S ', L C ' is a measure of the probability that data belongs to a specific category, L S It measures the likelihood that the data is true.

[0035] (4.9) The training objective of the generator is to maximize the Loss2, and the generator parameters are updated by backpropagation based on the generator's loss function;

[0036] (4.10) Repeat steps (4.1) to (4.9) until the preset number of iterations is reached to obtain the trained ACGAN improved model.

[0037] The beneficial effects of this invention are as follows: by introducing a multi-class attention mechanism, this invention enables ACGAN to focus on the features of seam defect samples from multiple angles, which is beneficial to enhancing the model's performance; by introducing residual blocks, this invention can solve the gradient vanishing problem, effectively improving the model's learning ability and accelerating the model's convergence speed; this invention improves the loss function for sample categories and the loss function for true and false samples, which is beneficial to accelerating model convergence and helping to improve model performance. Attached Figure Description

[0038] Figure 1 This is an overview of the different defects;

[0039] Figure 2 This is a graph showing the number of defects in each category;

[0040] Figure 3 This is a flowchart of the image generation process;

[0041] Figure 4 It is the generator upsampling residual module;

[0042] Figure 5 This is a flowchart of the discriminator's image discrimination process;

[0043] Figure 6 It is the discriminator downsampling residual module;

[0044] Figure 7 This is a flowchart of the improved ACGAN model. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit this application.

[0046] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0047] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to determination," or "includes." Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process or method. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0048] The ACGAN-based stitch image generation method of the present invention, based on a multi-class attention mechanism, specifically includes the following steps:

[0049] (1) Collect images of the sewing defect samples and crop the collected sewing defect images to the same size.

[0050] In this embodiment, an industrial camera is used to capture images of samples with stitching defects on packaging bags. The number of defective samples of different categories is unevenly distributed. When labeling these defective bags, the defects are divided into 14 categories according to actual business needs, such as... Figure 1The image shows the characteristics of different defects in the stitching and bags. These 14 categories specifically include: ① broken thread (sewing thread not slanted); ② slanted sewing thread, incomplete sewing; ③ no sewing thread; ④ wrinkles + broken thread; ⑤ uncut binding paper; ⑥ severe wrinkles; ⑦ double-bag sewing; ⑧ dirty binding paper; ⑨ wrinkles (no damaged bag); ⑩ slanted sewing thread + wrinkles; The binding paper is not properly attached and is wrinkled; The binding paper was severely misattached; No binding paper; The sewing thread is skewed (no damage to the bag). Of course, defects can also be categorized into other types according to actual needs. In this embodiment, 7174 images were collected, and the specific number for each category is as follows: Figure 2 As shown. The collected images of stitching defects were then cropped and adjusted to obtain the same image size of 440×120.

[0051] (2) Construct a dataset based on the cropped sewing defect images obtained in step (1), and randomly divide the dataset into a training set and a test set according to the proportions; wherein, the proportions of sewing defect images of each category are the same in the training set and the test set.

[0052] It should be understood that, while ensuring that the proportion of stitch defect images of each category is consistent between the training set and the test set, the constructed dataset is randomly divided into training set and test set according to a certain ratio, such as 8:2.

[0053] (3) Improve the ACGAN model by using a multi-class attention mechanism and residual module to obtain the improved ACGAN model and its loss function.

[0054] In this embodiment, the Generative Adversarial Network (GAN) is a deep learning model composed of a generator and a discriminator. The principle of GAN is to generate realistic data samples through adversarial training of the two networks. The generator receives a random noise vector (latent space) as input and generates generated data samples similar to real data. The discriminator receives real data samples and the generated samples as input and outputs the probability that they are real data. The generator's goal is to generate samples as realistic as possible to fool the discriminator, while the discriminator's goal is to distinguish between real and generated data as accurately as possible. During training, the generator and discriminator optimize against each other. The generator deceives the discriminator by generating samples, while the discriminator improves its discrimination ability by distinguishing between real and generated samples. Through this adversarial training, the generator gradually learns to generate realistic data samples, while the discriminator gradually learns to distinguish between real and generated samples more accurately.

[0055] The uncontrollable generation direction of GANs leads to a certain waste of time and space, and may result in pattern collapse. To address these issues, Auxiliary Classifier Generative Adversarial Network (ACGAN) adds label constraints to the generator network of GANs, requiring the generator to generate samples of specific categories. At the same time, the discriminator network adds multi-classification functionality, distinguishing between true and false samples while achieving multi-classification tasks through the auxiliary classifier.

[0056] The loss function of ACGAN consists of two parts: the log-likelihood loss Ls for classifying data as true or false and the log-likelihood loss Lc for classifying data as false, which are expressed as follows:

[0057] L S =E[logP(S=real|X real )]+E[logP(S=fake|X fake )]

[0058] L C =E[logP(C=c|X real )]+E[logP(C=c|X fake )]

[0059] Where E represents the expected value, log represents the logarithmic function, and X represents the image sample of the stitching defect. real X represents a sample image of a real seam defect. fake The image sample with stitch defects generated by the generator is represented by P, which represents the probability. S represents the discrimination state of the image sample with stitch defects. S = real means that the image sample with stitch defects is judged as a real sample, S = fake means that the image sample with stitch defects is judged as a sample generated by the generator, and C = c means that the sample is judged as category c.

[0060] Both the discriminator and the classifier aim to generate images belonging to a specified category, but the generator expects the discriminator to be unable to distinguish between real and fake samples, while the discriminator aims to correctly distinguish between them. The generator is trained to maximize Lc-Ls. The discriminator is trained to maximize Lc+Ls.

[0061] In this embodiment, the residual concept is a key idea used in deep learning to solve the network degradation problem. In traditional deep neural networks, as the number of network layers increases, network learning becomes more difficult, and problems such as vanishing and exploding gradients become more pronounced. To address this issue, residual networks introduce residual shortening.

[0062] The idea behind residual shortcuts is to capture residual information by directly connecting the input to the output. Specifically, the difference between the output and input of a residual block is called the residual. Residual shortcuts add the residual to the input, meaning the output equals the sum of the residual block's output and the input. Mathematically, this is represented as H(x) = F(x) + x, where F(x) represents the output of the residual block and x represents the input. Even if F(x) hasn't learned any features, the output of the residual block will not be zero, thus avoiding gradient degradation.

[0063] The residual concept, through the introduction of residual shortening, overcomes the degradation problem in deep networks, improving network performance and convergence speed. Residual shortening allows for deeper networks, easier optimization, and better learning of data features. This idea has been widely applied in residual networks, making significant contributions to the development of deep learning.

[0064] In this embodiment, the basic principle of the attention mechanism is to focus on and learn important information of the object through weight coefficients. The attention mechanism in image recognition is very similar to the human visual mechanism, prioritizing key information rather than all information. Attention modules in images are mainly divided into channel attention modules and location-pixel attention modules. This invention uses a multi-class attention mechanism: the channel attention module is a Squeeze-and-Excitation Network (SENet) module, and the location-pixel attention module is a Non-local Neural Network (Non-local) module.

[0065] The SENet module is an attention module for image recognition. Its goal is to enhance the network's expressive power by adaptively reweighting channels in feature maps, strengthening important channels while suppressing less important ones. The core idea of ​​SENet is to introduce an attention module consisting of two key steps: Squeeze and Excitation. In the Squeeze step, SENet uses global average pooling to compress each channel of the feature map into a single value, transforming each channel's feature into a global descriptor. In the Excitation step, SENet uses a small fully connected neural network to learn the channel weights. By multiplying each channel's weight by the corresponding feature map, SENet enhances the responses of important channels while suppressing the responses of less important ones. Finally, the weighted feature maps are fed into the next layer for further processing and classification.

[0066] Non-local modules are attention modules used in image recognition. Their goal is to enhance the network's perceptual capabilities by establishing global long-range dependencies, thereby improving the performance of image recognition tasks. The core idea of ​​non-local modules is to perform self-attention computation on feature maps to calculate the correlation between each location and all other locations. Specifically, non-local modules include three main steps: calculating similarity, calculating weights, and generating weighted features. Calculating similarity involves quantifying the relationship between each location in the feature map by calculating the similarity between each location and all other locations; a similarity matrix is ​​obtained by measuring the similarity between features. Calculating weights involves normalizing the similarity matrix to obtain the weights between each location and all other locations; typically, Softmax is used for normalization. Generating weighted features involves multiplying the features of each location by their corresponding weights and summing the weighted features of all locations to generate the final weighted features. In this way, each location can utilize information from other locations for feature enhancement.

[0067] In summary, the ACGAN-based stitching image generation method of the present invention generates stitching defect images through an improved ACGAN model, which includes a generator and a discriminator.

[0068] In this embodiment, the network structure of the generator is as follows: Figure 3 As shown, the generator's input is a vector concatenated from noise z and class labels c. This vector passes through a fully connected layer, two upsampling residual modules, and a multi-class attention module. The multi-class attention module includes a SENet module and a non-local module. The output of the multi-class attention module is concatenated along the channel dimension and then passed through a fully connected layer, a deconvolution layer, and a tanh activation function to output the generated stitching defect image. The upsampling residual module includes batch normalization (BatchNorm), rectified linear unit (ReLU), a convolutional layer, residual connections, and an upsampling layer, as shown below. Figure 4 As shown, in order to ensure that the channel dimensions of the input vector x and the output vector are consistent, the number of channels of the input vector x needs to be changed through the identity mapping layer, and then the input vector x and the output vector are added together.

[0069] In this embodiment, the network structure of the discriminator is as follows: Figure 5As shown, the discriminator's input consists of real images of stitching defects and images of stitching defects generated by the generator. The discriminator includes two downsampling residual modules, a multi-class attention module, and two parallel fully connected layers. The output of one fully connected layer is then processed by a sigmoid activation function to output the true / false probability, and the output of the other fully connected layer is then processed by a softmax activation function to output the class probability. The multi-class attention module includes an SENet module and a non-local module. The downsampling residual modules include convolutional layers, batch normalization (BatchNorm), rectified linear unit (ReLU), residual connections, and downsampling layers, as shown below. Figure 6 As shown, in order to ensure that the channel dimensions of the input vector x and the output vector are consistent, the number of channels of the input vector x needs to be changed through the identity mapping layer, and then the input vector x and the output vector are added together.

[0070] In this embodiment, the loss function of ACGAN consists of two parts: the log-likelihood loss Ls for judging whether the data is true or false and the log-likelihood loss Lc for classification prediction. By optimizing these two parts, the loss function of the improved ACGAN model can be obtained.

[0071] Furthermore, among the 14 defect categories, each defect has a different impact on bag breakage and leakage. Some defect categories will cause bag breakage and leakage, including ① broken thread (sewing thread not slanted), ② slanted sewing thread and incomplete sewing, ③ no sewing thread, ④ wrinkles + broken thread, ⑦ double-bag sewing, and ⑩ slanted sewing thread + wrinkles. The remaining categories will not cause bag breakage. To make the improved ACGAN model pay more attention to defect categories that lead to bag breakage and leakage, the log-likelihood loss Lc for classification prediction is modified, resulting in the improved ACGAN model's log-likelihood loss Lc for classification prediction. C ':

[0072] L C =E[w(c)·logP(C=c|X real )]+E[w(c)·logP(C=c|X fake )]

[0073] Where E represents the expected value; log represents the logarithmic function; X represents the image sample of the stitching defect, X real X represents a sample image of a real seam defect. fake This represents a sample image of a stitching defect generated by the generator; P represents the probability, P(C=c|X real P(C = c|X) represents the probability that a real image sample with a defective stitching is classified as the corresponding true category label. fake) represents the probability that the generated sewing defect image sample is the corresponding category label generated by the generator; w(c) represents the degree of influence of category c on bag breakage and leakage. The larger the value, the easier it is for the bag to break and leak. For the categories that are easy to break and leak, w(c) = 10, and w(c) = 1 for other categories.

[0074] Furthermore, the loss function Ls for distinguishing between real and fake data can be expressed as the JS divergence between the real image distribution and the generated image distribution. However, when the two distributions have no overlap, the generator's gradient is approximately zero. Using the Wasserstein distance to measure the approximation between the two distributions allows for the calculation of their distance even without overlap. Unlike the abrupt gradient of the JS divergence, the gradient of the Wasserstein distance changes more smoothly. However, due to the Lipshitz continuity condition, the derivative of the Wasserstein distance cannot exceed the Lipshitz constant K. To address this issue, this invention incorporates the concept of gradient penalty (GP) into Ls to overcome the Lipshitz continuity condition limitation. The gradient penalty is shown in the following equation:

[0075]

[0076] Where λ is the penalty coefficient and E is the expected value. Linear interpolation between real and generated samples, i.e. x is a real sample. It is a fake sample, ε ~ U[0,1], where U is a uniform distribution. The distribution is the linear interpolation between the real sample distribution and the generated sample distribution. Here, is the gradient, D is the discriminator, ||·||2 is the L2 norm, and K is the Lipschitz constant.

[0077] To keep the gradient as close to 1 as possible, the Lipschitz constant K is set to 1. After adding a gradient penalty term, the loss function L for the improved ACGAN model to distinguish between true and false data is obtained. S ':

[0078]

[0079] Where E represents the expected value, log represents the logarithmic function; X represents the image sample of the stitching defect, X real X represents a sample image of a real seam defect. fakeThis represents a sample image of a stitching defect generated by the generator; P represents the probability, S represents the discrimination state of the sample image, S = real means the sample image is judged as a real sample, S = fake means the sample image is judged as a sample generated by the generator, and λ is the penalty coefficient. Linear interpolation between real and generated stitch defect image samples. This represents the distribution of linear interpolation between real and generated image samples of stitching defects. It is the gradient, D is the discriminator, and ||·||2 is the 2-norm.

[0080] (4) Use the training set constructed in step (2) to iteratively train the improved ACGAN model obtained in step (3), and adjust the parameters of the generator and discriminator according to the loss function of the improved ACGAN model to obtain the trained improved ACGAN model. The flowchart of the improved ACGAN model is as follows. Figure 7 As shown.

[0081] (4.1) Randomly extract real sewing defect image samples x and their corresponding real class labels c1 from the training set, randomly sample Gaussian noise z from the normal distribution, and randomly generate corresponding class labels c2 for each Gaussian noise.

[0082] It should be understood that the Gaussian noise follows a normal (Gaussian) distribution. A Gaussian noise z is randomly sampled from the normal distribution, and a class label c2 is randomly drawn from the list [1,2,3... total number of class labels]. The Gaussian noise z and the class label c2 are input into the generator to generate the image corresponding to class c2.

[0083] (4.2) The noise z and the class label c2 are concatenated and input into the generator of the ACGAN improved model to obtain the generated stitch defect image sample.

[0084] (4.3) Input the generated sewing defect image samples and the real sewing defect image samples into the discriminator of the ACGAN improved model to obtain the discrimination probability and class probability.

[0085] (4.4) Calculate the discriminator's loss function Loss1 = L based on the loss function of the ACGAN improved model. C '+L S ', L C ' is a measure of the probability that data belongs to a specific category, L S It measures the likelihood that the data is true.

[0086] (4.5) The training objective of the discriminator is to maximize Loss1, and the parameters of the discriminator are updated by backpropagation based on the discriminator's loss function.

[0087] (4.6) Randomly sample Gaussian noise z with batch size n from a normal distribution, and randomly generate category labels c2 with batch size n.

[0088] (4.7) After concatenating the noise z and the category label c2, input them into the generator to obtain the generated sewing defect image sample. Input the generated sewing defect image sample into the discriminator updated in step (4.5) to obtain the discrimination probability and the category probability.

[0089] (4.8) Calculate the generator's loss function Loss2 = L based on the loss function of the ACGAN improved model. C '-L S ', L C ' is a measure of the probability that data belongs to a specific category, L S It measures the likelihood that the data is true.

[0090] (4.9) The training objective of the generator is to maximize the loss function and update the generator parameters by backpropagation based on the generator's loss function.

[0091] (4.10) Repeat steps (4.1)-(4.9) until the preset number of iterations is reached to obtain the trained ACGAN improved model.

[0092] (5) Design various experiments to test the quality of the ACGAN improved model trained in step (4) in order to fine-tune the parameters of the ACGAN improved model and obtain the final ACGAN improved model.

[0093] In this embodiment, three experiments were designed: quality assessment of generated images, comparison experiment, and ablation experiment. These three experiments were used to test the quality of the trained ACGAN improved model. Based on the test results, the parameters of the generator and discriminator in the ACGAN improved model were fine-tuned to obtain the final ACGAN improved model.

[0094] The first experiment: quality assessment of the generated images.

[0095] In evaluating the quality of image generation models, Mode Score (MS) and Frechet Inception Distance (FID) are two commonly used metrics. While both MS and FID are used to assess generative models, they have different focuses. MS primarily focuses on the diversity and quality of the generative model, while FID focuses on the distributional differences between generated samples and real data. A higher Mode Score indicates a generative model that can generate diverse images, each with unique styles and features. FID calculates the Frechet distance between the multivariate Gaussian distributions of generated and real samples. A lower FID indicates a very close approximation of the real sample distribution, while a higher MS indicates a higher quality generative model.

[0096] This experiment compared GAN, DCGAN, CGAN, WGAN-GP, ACGAN, and a modified ACGAN. MS (Maximum Score) and FID (Failure ID) were used to compare the image quality generated by each model. Table 1 shows the average MS and FID for each model. The table shows that ACGAN has a higher MS and lower FID than the other models. This result indicates that the modified ACGAN model can learn the features of real data better than other models, generating high-quality images of stitching defects.

[0097] Table 1 Comparison of MS and FID values ​​for different models

[0098] GAN model 1.201 0.434 DCGAN model 1.296 0.403 CGAN model 1.266 0.429 WGAN-GP model 1.301 0.389 ACGAN model 1.363 0.378 ACGAN Improved Model 1.423 0.363

[0099] The second type of experiment: a comparative experiment.

[0100] To test the impact of different GAN-generated stitching defect samples as data augmentation schemes on classifier accuracy, the number of stitching defect samples of each category in the training set was augmented to the same number N using DCGAN, CGAN, WGAN-GP, and ACGAN, with the same training and test sets. After several hyperparameter tests, N=5000 was selected. AlexNet, VGGNet, GoogLeNet, and ResNet image recognition classifiers were trained using the stitching defect training set augmented to N for each category. The image recognition classifiers were tested using the test set without data augmentation. The test results of the image recognition classifier trained on the training set without any data augmentation were used as the benchmark. Specific results are shown in Table 2.

[0101] Table 2. Accuracy of different sample augmentation methods on deep networks.

[0102]

[0103]

[0104] As can be seen from the table, the improved ACGAN model outperforms other data augmentation schemes in the test results, and its accuracy is improved by more than 10% compared with the baseline. The classification accuracy of each category under different classifiers is shown in Table 3 below.

[0105] Table 3. Classification accuracy of each category of defect samples under different classifiers.

[0106]

[0107] The third type of experiment: ablation experiment.

[0108] This invention introduces three improvements to ACGAN: a multi-class attention mechanism, residual blocks, and an improved loss function. To verify the effectiveness of these improvements, different improvement strategies were added to ACGAN, increasing the number of samples for each class of stitching defects to N=5000. AlexNet, VGGNet, GoogLeNet, and ResNet image recognition classifiers were trained using a data-augmented training set, and the performance of the image recognition classifiers was tested using a test set without data augmentation. The experimental results are shown in Table 4. Here, F represents the improved loss function, R represents the residual block, and A represents the multi-class attention mechanism. The improved ACGAN is thus ACGAN+A+R+F.

[0109] Table 4 Ablation Experiment

[0110] ACGAN 76.62 74.65 79.44 80.23 +A 81.34 82.25 83.52 82.63 +R+A 83.76 83.32 84.67 84.89 ACGAN Improved Model 85.43 83.40 86.53 87.72

[0111] As shown in Table 4, the accuracy of each image recognition classifier improved after introducing the multi-class attention mechanism. This is because the multi-class attention mechanism includes channel attention and position pixel attention, which can capture important features of seam defects. The residual block also helps improve accuracy while avoiding gradient vanishing. The improved loss function speeds up module convergence and also improves accuracy. Therefore, the three improvements proposed in this invention all contribute to improving the accuracy of the image recognition classifier.

[0112] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating stitch image based on a multi-class attention mechanism using ACGAN, characterized in that, Includes the following steps: (1) Collect images of the sewing defect samples and crop the collected sewing defect images to the same size; (2) Construct a dataset based on the cropped sewing defect images obtained in step (1), and randomly divide the dataset into a training set and a test set according to the proportion; wherein, the dataset includes sewing defect images and their corresponding category labels, and the proportion of sewing defect images of each category in the training set and the test set is the same. (3) Improve the ACGAN model by using a multi-class attention mechanism and residual module for optimization to obtain the improved ACGAN model and its loss function; the improved ACGAN model includes a generator and a discriminator. The generator's input is a vector concatenated from noise and class labels. This vector passes sequentially through a fully connected layer, two upsampling residual modules, and a multi-class attention module. The multi-class attention module includes a compression and activation network module and a non-local neural network module. The output of the multi-class attention module is concatenated according to the channel dimension and then passed sequentially through a fully connected layer, a deconvolution layer, and a tanh activation function to output the generated stitching defect image. The upsampling residual module includes batch normalization, a linear rectified function, a convolutional layer, residual connections, and an upsampling layer. The input to the discriminator is a real image of a stitching defect and an image of a stitching defect generated by the generator. The discriminator includes two downsampling residual modules, a multi-class attention module, and two parallel fully connected layers. The output of one fully connected layer is then processed by a sigmoid activation function to output the true / false probability, and the output of the other fully connected layer is then processed by a softmax activation function to output the class probability. The multi-class attention module includes a compression and activation network module and a nonlocal neural network module. The downsampling residual module includes a convolutional layer, batch normalization, a linear rectified function, residual connections, and a downsampling layer. The loss function of the improved ACGAN model includes the log-likelihood loss for classification prediction. and the loss function for judging the authenticity of data. Log-likelihood loss for classification prediction The expression is: Where E represents the expected value; log represents the logarithmic function; and X represents a sample image of a stitching defect. Images representing real-world seam defects. This represents a sample image of stitching defects generated by the generator; P represents the probability. This represents the probability that a real image sample with a defective stitching will be classified as the corresponding true category label. This represents the probability that the generated image sample of the stitching defect is classified as the corresponding category label when it was generated by the generator; This represents the degree of influence of category c on bag breakage and leakage; (4) Use the training set constructed in step (2) to iteratively train the ACGAN improved model obtained in step (3), and adjust the parameters of the generator and discriminator according to the loss function of the ACGAN improved model to obtain the trained ACGAN improved model. (5) Design various experiments to test the quality of the ACGAN improved model trained in step (4) in order to fine-tune the parameters of the ACGAN improved model and obtain the final ACGAN improved model.

2. The ACGAN stitching image generation method based on multi-class attention mechanism according to claim 1, characterized in that, Loss function for judging the authenticity of data The expression is: Where S represents the discrimination state of the stitching defect image sample, S=real means that the stitching defect image sample is judged as a real sample, and S=fake means that the stitching defect image sample is judged as a sample generated by the generator. It is the penalty coefficient. Linear interpolation between real and generated stitch defect image samples. This represents the distribution of linear interpolation between real and generated image samples of stitching defects. D is the gradient, and D is the discriminator. It is a 2-norm.

3. The ACGAN stitch line image generation method based on multi-class attention mechanism according to claim 1, characterized in that, Step (4) includes the following sub-steps: (4.1) Randomly extract real sewing defect image samples x and their corresponding real class labels c1 from the training set, randomly sample Gaussian noise z from the normal distribution, and randomly generate corresponding class labels c2 for each Gaussian noise. (4.2) The noise z and the class label c2 are concatenated and input into the generator of the ACGAN improved model to obtain the generated stitch defect image samples; (4.3) Input the generated sewing defect image samples and the real sewing defect image samples into the discriminator of the ACGAN improved model to obtain the discrimination probability and class probability; (4.4) Calculate the loss function of the discriminator based on the loss function of the improved ACGAN model. , It measures the probability that data belongs to a specific category. It measures the likelihood that the data is true. (4.5) The training objective of the discriminator is to maximize Loss1, and the parameters of the discriminator are updated by backpropagation based on the discriminator's loss function; (4.6) Randomly sample Gaussian noise z with batch size n from a normal distribution, and randomly generate class labels c2 with batch size n; (4.7) After concatenating the noise z and the category label c2, input them into the generator to obtain the generated sewing defect image sample. Input the generated sewing defect image sample into the discriminator updated in step (4.5) to obtain the discrimination probability and the category probability. (4.8) Calculate the generator's loss function based on the loss function of the ACGAN improved model. , It measures the probability that data belongs to a specific category. It measures the likelihood that the data is true. (4.9) The training objective of the generator is to maximize the Loss2, and the generator parameters are updated by backpropagation based on the generator's loss function; (4.10) Repeat steps (4.1) to (4.9) until the preset number of iterations is reached to obtain the trained ACGAN improved model.

Images