A cigarette case surface defect image generation method based on defect injection network

Abstract

The application discloses a cigarette case surface defect image generation method based on a defect injection network, and specifically comprises the following steps: firstly, constructing a cigarette case defect image dataset; then, constructing a pre-training defect positioning network and a conditional adversarial main network; then, adopting a two-stage strategy, in the first stage, training the defect positioning network to enable the defect positioning network to learn defect detailed features to generate a modulation signal; in the second stage, training the conditional adversarial main network to learn defect basic contour features; finally, injecting the defect modulation signal generated by the defect positioning network into an intermediate feature layer of the conditional adversarial main network, and outputting a cigarette case image of a specified defect by the conditional adversarial main network. The method can generate defect samples with high quality and accurate defect features, rapidly expand a defect sample library, and provide a reliable data basis for subsequent training of a cigarette case defect detection model.

Description

Technical Field

[0001] This invention belongs to the field of cigarette packaging quality inspection and computer vision technology, specifically relating to a method for generating images of surface defects in cigarette boxes based on a defect injection network. Background Technology

[0002] On automated cigarette packaging production lines, machine vision-based surface defect detection is crucial for ensuring product quality. The performance of such detection models relies on a large number of accurately labeled defect images. However, in actual production, the vast majority of products are qualified, and surface defects on cigarette boxes (such as label printing errors, packaging damage, stains, and curling edges) occur infrequently and vary in shape, making it costly and time-consuming to collect a sufficient number of diverse real defect images.

[0003] Data augmentation using generative adversarial networks (GANs) is a cutting-edge approach to solving this problem. However, existing solutions, such as the StyleGAN series, have significant limitations when applied to images of industrial products with complex and detailed textures (such as text, logos, and holographic patterns), like cigarette boxes. First, as unsupervised generative models, their generation process is random, making it difficult to precisely control the type, size, and location of defects, thus failing to meet the requirements of detection tasks for specific defect samples. Second, directly generating high-resolution, high-detail-fidelity structured images is challenging, often resulting in texture blurring or local distortion. Furthermore, the models have a large number of parameters, require high training stability, and consume significant computational resources.

[0004] Some improved solutions attempt to introduce conditional information or use divide-and-conquer strategies (such as segmenting first, generating, and then stitching), but they face challenges such as inaccurate segmentation, obvious stitching marks, and poor spatial consistency when dealing with defects that blend well with the background (such as large-area warping and wrinkles). Therefore, there is a need for a defect image generation method that can achieve precise spatial control, high detail fidelity, and high efficiency and stability, and directly synthesize high-quality data that can be used to train downstream detection models. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for generating cigarette box surface defect images based on a defect injection network. This method constructs and co-trains a defect localization network and a conditional adversarial master network, achieving decoupled learning and controllable injection of defect features. This enables the generation of high-quality, accurately located cigarette box surface defect images based on the input defect dataset.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Step S1: Construct a training dataset of defective cigarette box images and corresponding annotation files; Step S2: Construct a defect generation model that includes a pre-trained defect localization network and a conditional adversarial main network; Step S3: The defect localization network and the conditional adversarial main network are trained using a two-stage training strategy. First, the defect localization network is trained to learn the input defect feature parameters to generate a modulation signal. Second, the conditional adversarial main network is trained to learn the basic contours and approximate features of the defect image only under the defect dataset. Step S4: Using the trained defect localization network and the conditional adversarial main network, the modulated signal with defect details generated by the defect localization network is injected into the intermediate feature layer of the conditional adversarial main network, and finally the conditional adversarial main network outputs an image of the cigarette box with the specified defect.

[0007] The defect localization network includes: a defect feature encoder, which adopts a convolutional neural network structure to convert the description parameters of the defect into a multi-dimensional feature vector; a spatial mapping module, which generates a spatial modulation mask based on the defect location information to indicate the addition of the defect in a specific region of the image; and a feature modulator, which performs feature modulation using an adaptive weighted fusion method based on the spatial modulation mask, and performs position-aware feature fusion on the intermediate feature layer of the conditional adversarial main network.

[0008] The conditional adversarial main network includes a mapping network and a conditional-aware synthesis network. The conditional-aware synthesis network is composed of multiple cascaded feature modulation blocks with increasing resolution. Each feature modulation block receives a style vector from the mapping network and spatial conditional features from the defect localization network. The spatial conditional features are adaptively weighted and fused with intermediate features in the feature modulation blocks through a dedicated conditional fusion module to control the generation of defects on the corresponding resolution feature map.

[0009] Furthermore, the adaptive weighted fusion method is as follows: the defect modulation features and the original features are fused according to the region indicated by the spatial modulation mask, and the degree of fusion is controlled by a learnable modulation intensity parameter, so as to finally obtain the modulated feature map.

[0010] Furthermore, the two-stage training strategy is as follows: in the first stage, the defect localization network is trained using the defect image dataset to learn to generate modulation signals; in the second stage, the parameters of the defect localization network are kept unchanged, and the conditional adversarial master network is trained using the defect image dataset to learn to generate defect images using the modulation signals.

[0011] Furthermore, the fusion process of the conditional fusion module is as follows: the intermediate feature map of the feature modulation block is concatenated with the spatial conditional features after upsampling and alignment in the channel dimension; the concatenated features are input into an attention generation network composed of convolutional layers and activation functions to calculate a spatially adaptive attention weight map; this attention weight map is multiplied element-wise with the original intermediate feature map, and then added to the original features through a residual connection to complete the modulation fusion.

[0012] Furthermore, the loss functions used during the training of the defect localization network and the generative adversarial main network include: defect region reconstruction loss: calculating the difference between the generated image and the real defect image only for the defect region; background preservation loss: calculating the difference between the generated image and the real image for the non-defect region (background) to ensure that the background remains unchanged; feature alignment loss: constraining the consistency of the generated image and the real image in visual content and structure by comparing the high-level semantic features of the generated image and the real image; and adversarial loss: used to introduce adversarial game in the training of the conditional adversarial main network to improve the overall realism and diversity of the generated images.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention creatively employs a decoupled two-stage training paradigm and defect injection mechanism. An independent defect localization network learns the abstract representation and spatial mapping of defects, while a conditional adversarial main network learns the basic image generation. The two networks collaborate during the generation stage through feature modulation. This design makes the control over defect location and type explicit and precise, fundamentally solving the problem of high randomness in unsupervised generation and achieving controllable image synthesis. 2. This invention, by designing a feature modulator incorporating adaptive weighted fusion and a conditional fusion module with a spatial attention mechanism, can smoothly and accurately inject defect features into the appropriate levels and spatial locations of the generation process. This fine-grained feature manipulation mechanism enables the model to perfectly preserve the complex texture details of the cigarette box background (such as finely printed text and patterns) while generating defects, achieving high-quality fusion of defects and background and generating high-fidelity images; 3. This invention comprehensively utilizes defect region reconstruction, background preservation, feature alignment, and adversarial loss during training. This multi-objective and targeted combination of loss functions ensures that the model achieves an optimal balance across multiple key dimensions: guaranteeing the accuracy of defect generation, maintaining the realism of the background, and promoting the diversity of generated samples, thereby enabling the construction of a comprehensive, high-quality defect image library that can be used for practical model training. 4. The two-stage training strategy described in this invention is not only conceptually clear but also contributes to training stability. First, one part of the network is fixed while another part is trained, reducing the difficulty of simultaneously optimizing multiple complex modules. Furthermore, the modular design allows each part of the network to be optimized or replaced independently; for example, the conditional adversarial main network can be implemented using different generator architectures, improving the flexibility and scalability of the method. 5. The method provided by this invention forms a complete technical closed loop, from data preparation to final image generation. Users can generate high-quality images of specified defects in batches through simple parameter input, greatly reducing the cost and time required to acquire defect data, and providing an efficient and reliable data support solution for the research and development and performance improvement of deep learning-based cigarette box appearance inspection systems. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the overall workflow of the method described in this invention.

[0015] Figure 2 This is a schematic diagram of the overall network architecture of the defect injection generation model described in this invention.

[0016] Figure 3 A detailed diagram of the specific structure of the defect localization network is provided.

[0017] Figure 4 Image of small box warping defect generated using the original generative model.

[0018] Figure 5 The image shows a small box warping defect generated using the improved generation model of this invention.

[0019] Figure 6 This is a true image of the warped edge defect on the small box.

[0020] Figure 7 The graph shows the quantitative assessment results of FID score and SSIM. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment uses the generation of an image of a "small box warping edge" defect as an example for illustration, but the application of this invention is not limited to this specific defect.

[0022] The hardware platform for implementing this invention includes a computing server equipped with a high-performance CPU and an NVIDIA GPU. The software environment is based on an RTX4090 cloud server and uses Python 3.7 and the PyTorch 1.13.1 deep learning framework.

[0023] The overall workflow of the method of this invention is as follows: Figure 1As shown, firstly, real image data containing target defects are collected and preprocessed to construct a training set; secondly, a defect generation model is constructed, including a pre-trained defect localization network and a conditional adversarial main network, and then a two-stage training strategy is adopted to train the defect localization network and the conditional adversarial main network: in the first stage, the defect localization network is trained independently, and in the second stage, the trained defect localization network and the conditional adversarial main network are integrated for end-to-end joint training; finally, the trained model is used to generate controllable defect images based on the input defect dataset.

[0024] In the data preparation and preprocessing stage, real images of cigarette boxes containing the "small box curling edge" target defect need to be collected, and the bounding box location information of the defect needs to be labeled by professionals. Subsequently, the images are sized (in this embodiment, they are uniformly adjusted to 512×512 pixels) and color normalized, thereby constructing a structured training dataset containing the original images and their corresponding defect annotation files.

[0025] The construction and training of the defect injection generation model is the core of this invention. The overall network architecture of the model is as follows: Figure 2 As shown, the conditional adversarial main network is based on the StyleGAN2-ADA model (corresponding to...). Figure 2 The Generator and Discriminator parts of the main conditional adversarial network are the generator and discriminator, respectively. The generator comprises a mapping network and a conditional-aware synthesis network. The mapping network is responsible for transforming random noise vectors into a series of style vectors (the random noise vectors...). z The samples are taken from a standard normal distribution, and their dimensions are consistent with the StyleGAN2-ADA model on which they are based. In this embodiment, a 512-dimensional noise vector is used, i.e. z (∈ R^512). The conditional synthesis network is the core generator of StyleGAN2-ADA, consisting of multiple cascaded synthesis blocks with increasing resolution. Its key innovation lies in the integration of a conditional fusion module designed in this invention within each synthesis block (connected to the Positioning head of the defect localization network, located between Upsample and AdaIN). This module receives the modulated signal (upsampled and aligned) from the defect localization network and the intermediate feature map of the current synthesis block of the generator. Through concatenation, convolution, and activation functions (such as Sigmoid), it generates a spatial attention weight map, thereby precisely modulating the intermediate features to guide the generation of defects in the specified region. The discriminator of this network adopts the original convolutional neural network structure of StyleGAN2-ADA.

[0026] The structure of the defect localization network is detailed in [link to details]. Figure 3 This network employs an encoder structure as a defect feature encoder, used to map parameter vectors containing information such as defect type and bounding box coordinates into high-dimensional feature vectors. Subsequently, a spatial mapping module decodes these feature vectors into a set of spatial modulation masks, the resolution of which matches the feature map resolution of the corresponding layer in the conditional adversarial main network. Figure 1 In the first stage of training shown, the defect localization network is trained independently using the training dataset. The training objective is to optimize the loss function so that its output modulation mask accurately covers the labeled area of ​​the real defect, thereby enabling the network to predict the spatial distribution of defects based on parameters.

[0027] In such Figure 1 In the second stage of training, the parameters of the previously trained defect localization network are first frozen. Random noise vectors and real images are input into the mapping network and discriminator of the conditional adversarial master network, respectively. The goal of this stage is to train the conditional adversarial master network to generate an image that accurately captures defect features at a specified location while maintaining a realistic overall background, based on the modulation signal provided by the defect localization network.

[0028] In each training iteration, the system receives a batch of training data. Defect annotation information is input into the frozen defect localization network to generate corresponding spatial modulation signals. Simultaneously, random noise vectors and real images are input into the mapping network and discriminator of the conditional adversarial main network, respectively. The conditional adversarial main network synthesizes a defect image based on these inputs. The generated image is optimized in each iteration. This stage employs a carefully designed joint loss function for optimization, which is a weighted combination of multiple sub-losses, specifically including: 1. Defect region reconstruction loss: Calculate the pixel-level difference between the generated image and the real image in the defect-annotated area (such as L1 loss or MSE loss), forcing the generator to accurately reconstruct the shape and details of the defect at the specified location; 2. Background Preservation Loss: Calculate the pixel-level differences (such as L1 loss) between the generated image and the real image in non-defect areas (background) to ensure that the generation process does not destroy the original, intact texture and structure of the cigarette box; 3. Feature alignment loss: Using a pre-trained VGG network, multi-level features of the generated image and the real image are extracted, and the perceptual distance between their feature maps (such as LPIPS loss) is calculated to constrain the generated image to maintain consistency with the real image at the semantic feature level, thereby improving the naturalness of the generated defects and the degree of integration with the background. 4. Adversarial Loss: The non-saturating adversarial loss from the StyleGAN2-ADA framework is adopted. The discriminator of the conditional adversarial main network not only needs to distinguish between real and fake images, but also needs to combine defect localization information to encourage the generator to produce high-quality results that are difficult to distinguish from real images both globally and locally, while promoting the diversity of generated samples.

[0029] The aforementioned loss functions work together to guide the optimization direction of the generator. Through the backpropagation algorithm and iterative updates of the optimizer (such as Adam), the conditional adversarial main network gradually learns to generate defects that meet the specified conditions at the correct locations in the image, guided by the modulation signal provided by the defect localization network and in conjunction with its own image generation capabilities, while maintaining the high realism and naturalness of other areas of the image (background).

[0030] Once training is complete, image generation can begin. The parameters are input into the trained defect localization network to generate a modulated signal. Simultaneously, a random noise vector is generated in the manner described above. z The modulated signal and noise vector are input into the trained conditional adversarial main network, and after being processed layer by layer by the conditional sensing synthesis network, a high-quality cigarette box image with the "small box edge curling" defect at the specified location is finally output.

[0031] To verify the effect, the image generated by this invention ( Figure 5 ) and the image generated by the original generative model ( Figure 4 ) and real images ( Figure 6 A comparison was made. It is evident that the defects generated by this invention are highly consistent with the specified conditions in terms of location and shape, and the background texture is clear and natural, significantly superior to the original model. Further quantitative evaluation results are as follows: Figure 7 As shown, the original model has an FID score of 85.67 and an SSIM score of 0.621, indicating poor similarity to real images. In contrast, the optimized defect generation model has an FID score of 49.28 (a decrease of 42.5%) and an SSIM score of 0.864 (an improvement of 39.1%), confirming that the images generated by the defect generation model have higher realism and structural similarity.

[0032] The above embodiments are merely preferred embodiments of the present invention, used to specifically illustrate the technical solutions of the present invention and not to limit them. Those skilled in the art should understand that various modifications and substitutions can be made to the network structure, training details, loss function combinations, etc., in the described method without departing from the principles and spirit of the present invention, and all such modifications and substitutions should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for generating surface defect images of cigarette boxes based on a defect injection network, characterized in that, Includes the following steps: Step S1: Construct a training dataset of cigarette box defect images and corresponding labeled files; S2: Construct a defect generation model including a pre-trained defect localization network and a conditional adversarial main network; S3: Train the defect localization network and the conditional adversarial main network using a two-stage training strategy. First, train the defect localization network to learn the input defect feature parameters and generate a modulation signal; second, train the conditional adversarial main network to learn the basic contours and approximate features of the defect image only under the defect dataset; S4: Using the trained defect localization network and the conditional adversarial main network, inject the modulation signal with defect details generated by the defect localization network into the intermediate feature layer of the conditional adversarial main network, and finally, the conditional adversarial main network outputs a cigarette box image with the specified defect. The defect localization network includes: a defect feature encoder, which adopts a convolutional neural network structure to convert the description parameters of the defect into a multi-dimensional feature vector; and a spatial mapping module, which generates a spatial modulation mask based on the defect location information to indicate the addition of defects in specific regions of the image. The feature modulator performs feature modulation using an adaptive weighted fusion method based on the spatial modulation mask, and performs position-aware feature fusion on the intermediate feature layer of the conditional adversarial main network; the conditional adversarial main network includes a mapping network and a conditional-aware synthesis network. The condition-aware synthesis network is composed of multiple cascaded feature modulation blocks with increasing resolution. Each feature modulation block receives a style vector from the mapping network and spatial condition features from the defect localization network. The spatial condition features are adaptively weighted and fused with intermediate features in the feature modulation blocks through a dedicated conditional fusion module to control the generation of defects on the corresponding resolution feature map.

2. The method for generating cigarette box surface defect images based on defect injection networks according to claim 1, characterized in that, The adaptive weighted fusion method is as follows: the defect modulation features and the original features are fused according to the region indicated by the spatial modulation mask, and the degree of fusion is controlled by a learnable modulation intensity parameter, so as to finally obtain the modulated feature map.

3. The method for generating cigarette box surface defect images based on defect injection networks according to claim 1, characterized in that, The two-stage training strategy is as follows: In the first stage, the defect localization network is trained using the defect image dataset to learn how to generate modulated signals; in the second stage, the parameters of the defect localization network are kept unchanged, and the conditional adversarial master network is trained using the defect image dataset to learn how to generate defect images using the modulated signals.

4. The method for generating cigarette box surface defect images based on defect injection networks according to claim 1, characterized in that, The fusion process of the conditional fusion module is as follows: the intermediate feature map of the feature modulation block is concatenated with the spatial conditional features after upsampling and alignment in the channel dimension; the concatenated features are input into an attention generation network composed of convolutional layers and activation functions to calculate a spatially adaptive attention weight map; this attention weight map is multiplied element-wise with the original intermediate feature map, and then added to the original features through a residual connection to complete the modulation fusion.

5. The method for generating cigarette box surface defect images based on defect injection networks according to claim 1, characterized in that, The loss functions used during the training of the defect localization network and the generative adversarial main network include: defect region reconstruction loss: calculating the difference between the generated image and the real defect image only for the defect region; background preservation loss: calculating the difference between the generated image and the real image for the non-defect region, i.e., the background, to ensure that the background remains unchanged; feature alignment loss: constraining the consistency of the generated image and the real image in visual content and structure by comparing the high-level semantic features of the generated image and the real image; and adversarial loss: used to introduce adversarial game in the training of the conditional adversarial main network to improve the overall realism and diversity of the generated images.

Images