A method and device for generating defect sample images based on CGNET

Generating new defective cigarette box sample images through the CGNET network solves the problem of insufficient number of defective cigarette box sample images in the prior art, and improves the accuracy and recognition accuracy of the cigarette box defect detection model.

CN117409281BActive Publication Date: 2024-11-19北京远舢智能科技有限公司
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202311396843.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-26
Publication Date
2024-11-19
Estimated Expiration
2043-10-26

AI Technical Summary

Technical Problem

In the prior art, the number of defective cigarette box sample images is small, which leads to inaccurate enough in the cigarette box defect detection model, thereby reducing the recognition accuracy in the cigarette box production process.

Method used

Using CGNET-based image generation method, new defective cigarette box sample images are generated through GAN network and CGNET network, and the noise generation network, visual adjustment network and multi-task network are used for semantic segmentation and image conversion, and generate causal relationships and geometric structures. Defective cigarette case sample image.

Benefits of technology

The accuracy of the cigarette box defect detection model and the accuracy of identifying the defective cigarette box are improved, and the problem of insufficient number of samples of defective cigarette box is solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117409281B_ABST
    Figure CN117409281B_ABST
Patent Text Reader

Abstract

The present application provides a method and device for generating defective sample images based on CGNET, the generating method comprising: obtaining an original defective cigarette box sample image; inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image; and determining a defective cigarette box sample image for training a cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image. The generating method can solve the problem of a small number of defective cigarette box sample images in the prior art, improve the accuracy of the trained cigarette box defect detection model, and the accuracy of defective cigarette boxes identified during the cigarette box production process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for generating defect sample images based on CGNET. Background Art

[0002] In the production process of cigarette boxes, image processing technology is generally required to identify cigarette boxes with defects in the production process.

[0003] At present, due to the small number of defective cigarette boxes in reality, the number of defective cigarette box sample images is small, which leads to the cigarette box defect detection model trained using the small number of defective cigarette box sample images is not accurate enough, resulting in low accuracy in identifying defective cigarette boxes in the cigarette box production process. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method and device for generating defective sample images based on CGNET, so as to generate more defective cigarette box sample images.

[0005] In a first aspect, an embodiment of the present application provides a method for generating a defect sample image based on CGNET, the generating method comprising:

[0006] Obtaining an original defective cigarette box sample image;

[0007] Inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image;

[0008] Determine a defective cigarette box sample image for training a cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image.

[0009] Optionally, the image generation model includes: a GAN network and a CGNET network; the step of inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image includes:

[0010] Inputting the original defective cigarette box sample image into the GAN network to perform feature extraction on the original defective cigarette box sample image to obtain a first feature vector of the original defective cigarette box sample image;

[0011] Inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image;

[0012] Performing image transformation on the second feature vector of the original defective cigarette box sample image to generate a new defective cigarette box sample image.

[0013] Optionally, the CGNET network includes a noise generation network, a visual adjustment network and a multi-task network;

[0014] The step of inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image includes:

[0015] Inputting the first eigenvector into the noise generation network for noise generation processing to generate a noise vector;

[0016] Inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain an adjusted feature vector;

[0017] Inputting the adjusted feature vector into the multi-task network to obtain the feature vector to be processed output by each sub-task network included in the multi-task network;

[0018] The feature vectors to be processed output by each subtask network are determined as the second feature vectors of the original defective cigarette box sample image.

[0019] Optionally, the noise generation network includes a BN layer, a hidden layer, a noise layer and a fusion layer; the step of inputting the first feature vector into the noise generation network for noise generation processing to generate a noise vector includes:

[0020] Inputting the first feature vector into the BN layer to perform vector normalization integration to obtain an integrated vector;

[0021] Inputting the integration vector into the hidden layer to obtain a split vector;

[0022] Inputting the split vector into the noise layer for vector enhancement to obtain a noise enhancement vector;

[0023] The noise enhancement vector and the split vector are input into the fusion layer for vector fusion to obtain a noise vector.

[0024] Optionally, the visual accommodation network includes a first visual neural network group, a second visual neural network group, and a third visual neural network group; inputting the noise vector and the first feature vector into the visual accommodation network for vector accommodation to obtain an adjusted feature vector includes:

[0025] Inputting the noise vector and the first feature vector into the first visual neural network group to obtain a first adjusted feature vector;

[0026] Inputting the noise vector and the first feature vector into the second visual neural network group to obtain a second adjusted feature vector;

[0027] Inputting the noise vector and the first feature vector into the third visual neural network group to obtain a third adjusted feature vector;

[0028] The first adjusted feature vector, the second adjusted feature vector and the third adjusted feature vector are fused to obtain an adjusted feature vector.

[0029] In a second aspect, an embodiment of the present application provides a device for generating a defect sample image based on CGNET, the device comprising:

[0030] An acquisition module, used for acquiring an original defective cigarette box sample image;

[0031] A first image generation module, used for inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image;

[0032] The second image generation module is used to determine the defective cigarette box sample image used for training the cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image.

[0033] Optionally, the image generation model includes: a GAN network and a CGNET network; the first image generation module is specifically used to:

[0034] Inputting the original defective cigarette box sample image into the GAN network to perform feature extraction on the original defective cigarette box sample image to obtain a first feature vector of the original defective cigarette box sample image;

[0035] Inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image;

[0036] Performing image transformation on the second feature vector of the original defective cigarette box sample image to generate a new defective cigarette box sample image.

[0037] Optionally, the CGNET network includes a noise generation network, a visual adjustment network and a multi-task network; the first image generation module is specifically used to:

[0038] Inputting the first eigenvector into the noise generation network for noise generation processing to generate a noise vector;

[0039] Inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain an adjusted feature vector;

[0040] Inputting the adjusted feature vector into the multi-task network to obtain the feature vector to be processed output by each sub-task network included in the multi-task network;

[0041] The feature vectors to be processed output by each subtask network are determined as the second feature vectors of the original defective cigarette box sample image.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the generation method as described above are performed.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the generation method as described above are executed.

[0044] The embodiment of the present application provides a method and device for generating defective sample images based on CGNET, comprising: obtaining an original defective cigarette box sample image; inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image; and determining a defective cigarette box sample image for training a cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image. The embodiment of the present application provides a method for generating defective sample images based on CGNET, which generates new defective cigarette box sample images using an image generation model, thereby solving the problem of a small number of defective cigarette box sample images in the prior art, and improving the accuracy of the trained cigarette box defect detection model, as well as the accuracy of defective cigarette boxes identified during the cigarette box production process.

[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flowchart of a method for generating a defect sample image based on CGNET provided by an exemplary embodiment of the present application is shown;

[0048] Figure 2 A schematic diagram of the structure of a device for generating defect sample images based on CGNET provided by an exemplary embodiment of the present application is shown;

[0049] Figure 3 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0051] At present, due to the small number of defective cigarette boxes in reality, the number of defective cigarette box sample images is small, which leads to the cigarette box defect detection model trained using the small number of defective cigarette box sample images is not accurate enough, resulting in low accuracy in identifying defective cigarette boxes in the cigarette box production process.

[0052] Based on this, an embodiment of the present application provides a method for generating defective sample images based on CGNET, which uses an image generation model to generate new defective cigarette box sample images, thereby solving the problem of the small number of defective cigarette box sample images in the prior art, and improving the accuracy of the trained cigarette box defect detection model, as well as the accuracy of defective cigarette boxes identified during the cigarette box production process.

[0053] See also Figure 1 , Figure 1 A flowchart of a method for generating a defect sample image based on CGNET provided by an exemplary embodiment of the present application is shown.

[0054] like Figure 1 As shown in , a method for generating a defect sample image based on CGNET provided in an embodiment of the present application includes the following steps:

[0055] S1, obtaining an original defective cigarette box sample image;

[0056] Here, the number of original defective cigarette box sample images may be determined according to actual conditions. For example, the number of original defective cigarette box sample images may be 1,000.

[0057] S2, inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image;

[0058] S3. Determine a defective cigarette box sample image for training a cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image.

[0059] The embodiment of the present application provides a method for generating defective sample images based on CGNET, which uses an image generation model to generate new defective cigarette box sample images, thereby solving the problem of the small number of defective cigarette box sample images in the prior art, and improving the accuracy of the trained cigarette box defect detection model and the accuracy of defective cigarette boxes identified in the cigarette box production process.

[0060] Next, the specific implementation of steps S1 to S3 will be introduced.

[0061] As an example, the image generation model may include: a GAN network and a CGNET network;

[0062] Here, the GAN network is a network that generates new samples by learning the statistical characteristics of training data. However, in the processing of defective images of cigarette boxes, the specific reasons or conditions for their formation cannot be directly deduced by simply observing the statistical characteristics of training data. Therefore, if only the GAN network is used, it may be difficult to generate defective sample images of cigarette boxes with geometric structure logic and rationality.

[0063] Therefore, in this application, on the basis of the GAN network, a CGNET network is added to introduce more prior knowledge and auxiliary information, so that the image generation model can generate more reasonable and geometrically structured cigarette box defect sample images under the constraints of causality.

[0064] Here, the GAN network is a generation model composed of two competing neural networks. It consists of a generator network (Generator) and a discriminator network (Discriminator).

[0065] The generator network is a neural network used to generate new samples. It takes random vectors or noise as input and gradually generates samples through a series of conversion layers. The goal of the generator is to generate data similar to real samples to confuse the discriminator network.

[0066] The discriminator network is a neural network used to judge the authenticity of samples. It receives samples from real samples and samples generated by the generator network, and predicts the probability value of each sample, indicating the possibility of it being a real sample. The goal of the discriminator is to accurately distinguish between real samples and generated samples.

[0067] Specifically, when the image generation model includes: a GAN network and a CGNET network, regarding step S2, when it is specifically implemented, the following steps may be included:

[0068] S21, inputting the original defective cigarette box sample image into the GAN network to perform feature extraction on the original defective cigarette box sample image to obtain a first feature vector of the original defective cigarette box sample image;

[0069] S22, inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image;

[0070] As an example, the CGNET network may include a noise generation network, a visual adjustment network, and a multi-task network;

[0071] Here, the multi-task network may include multiple sub-task networks, including but not limited to at least one of a picture segmentation task network, a grayscale image generation task network, a picture blurring task network, and an image restoration task network. The picture segmentation task network is used to segment the defective cigarette box image into the outline of the entire cigarette box and the outline of the defective part; the grayscale image generation task network is used to form a grayscale image of the defective cigarette box image; the picture blurring task network is used to generate a blurred defective cigarette box image from a clear defective cigarette box image; and the image restoration task network is used to generate a clearer image from the defective cigarette box image, which also includes color restoration, etc.

[0072] Specifically, when the CGNET network includes a noise generation network, a visual adjustment network, and a multi-task network, regarding step S22, when it is specifically implemented, the following steps may be included:

[0073] S221, inputting the first feature vector into the noise generation network for noise generation processing to generate a noise vector;

[0074] As an example, the noise generation network may include a BN layer, a hidden layer, a noise layer, and a fusion layer;

[0075] Here, the BN (Batch Normalization) layer is a commonly used neural network layer, which is used to normalize the distribution of input data in deep learning. The main purpose of the BN layer is to solve the problem of internal covariate shift in deep neural networks. Internal covariate shift refers to the fact that the input distribution of each layer changes as the parameters are updated during the training process, resulting in problems such as difficulty in network training and slow convergence. The BN layer normalizes the input data by performing mean normalization and variance normalization on each mini-batch sample. Specifically, for an input data with d features, the BN layer first calculates the mean and variance of each feature. Then, for each feature, the mean and variance of the batch samples are used for normalization, so that the mean is close to 0 and the variance is close to 1. Finally, the normalized results are linearly transformed by learnable scaling factors and translation factors to restore the expressiveness of the data.

[0076] NoisyNet is a technique for introducing randomness into neural networks, aiming to enhance the exploration and generalization capabilities of the model. Traditional neural networks usually generate outputs by weighted summing the inputs and applying nonlinear activation functions during forward propagation. The noise layer adds random noise terms to this process to introduce randomness. The introduction of the noise layer has the following effects:

[0077] Enhanced exploration capability: By introducing noise terms, the noise layer can introduce randomness in the forward propagation process, thereby enhancing the model's exploration capability. This enables the model to more fully explore different paths and solutions during training and avoid falling into local optimal points.

[0078] Improved robustness: The randomness of the noise layer can help the model better adapt to noise and changes in the data. It makes the model more robust to small perturbations in the input, thereby improving the model's generalization ability and ability to handle unknown data.

[0079] Reduce the risk of overfitting: By introducing noise, the noise layer has a certain regularization effect, which can reduce the risk of overfitting of the model. The randomness of the noise makes the model insensitive to slight changes in the training data, thereby reducing the possibility of overfitting.

[0080] In the case where the noise generation network includes a BN layer, a hidden layer, a noise layer and a fusion layer, regarding step S221, in specific implementation, the following steps may be included:

[0081] First, the first feature vector is input into the BN layer for vector normalization and integration to obtain an integrated vector; then, the integrated vector is input into the hidden layer to obtain a split vector; then, the split vector is input into the noise layer for vector enhancement to obtain a noise enhanced vector; finally, the noise enhanced vector and the split vector are input into the fusion layer for vector fusion to obtain a noise vector.

[0082] S222, inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain an adjusted feature vector;

[0083] As an example, the visual accommodation network may include a first visual neural network group, a second visual neural network group, and a third visual neural network group;

[0084] As an example, the first visual neural network group may include 1 BN neural network layer, 5 ConvLSTM neural network layers, 1 Flatten neural network layer and 2 Dense neural network layers, the second visual neural network group may include 1 BN neural network layer, 5 ConvLSTM neural network layers, 1 Flatten neural network layer and 2 Dense neural network layers, and the third visual neural network group may include 1 Flatten neural network layer, 5 LSTM neural network layers, and 2 Dense neural network layers.

[0085] Here, ConvLSTM is a neural network architecture that combines convolutional neural network (CNN) and long short-term memory network (LSTM). It extends the traditional LSTM structure to make it suitable for processing sequence data with spatial structural characteristics, such as image sequences or video sequences. Among them, LSTM is a variant of recurrent neural network (RNN) that can effectively process sequence data. It captures and memorizes long-term dependencies in the input sequence through a gating mechanism, which can effectively solve the gradient vanishing and gradient exploding problems of traditional RNN in long sequence training. The main components of LSTM include input gate, forget gate, output gate and cell state. Each gated unit contains weighted operations and nonlinear activation functions to gate and adjust the input and hidden states. CNNLSTM is a neural network model that combines convolutional neural network (CNN) and long short-term memory network (LSTM). It is designed to process data with spatial structural characteristics and time series relationships. In CNNLSTM, CNN is first used to extract the spatial features of the input data. CNN consists of convolutional layers, pooling layers, activation functions, etc., which can effectively learn and extract local features in the input data. For image data, CNN is usually used to extract feature representations of images. Subsequently, the output of CNN is used as the input sequence of LSTM. LSTM is a variant of recurrent neural network (RNN) suitable for processing sequence data. LSTM captures and memorizes long-term dependencies in sequence data through a gating mechanism. In CNNLSTM, the input of LSTM is the feature sequence extracted by CNN, and the memory unit of LSTM is used to model and process the time series relationship of the input data. By combining CNN and LSTM, CNNLSTM is able to consider both the spatial information and the temporal relationship of the input data. This is very useful for tasks such as processing video data, action recognition, and behavior prediction. CNNLSTM can extract spatial features from video sequences and capture the temporal dependencies of video sequences through LSTM, thereby realizing the analysis and processing of video data.

[0086] In the case where the visual accommodation network includes a first visual neural network group, a second visual neural network group, and a third visual neural network group, regarding step S222, in specific implementation, the following steps may be included:

[0087] First, the noise vector and the first feature vector are input into the first visual neural network group to obtain a first adjusted feature vector; then, the noise vector and the first feature vector are input into the second visual neural network group to obtain a second adjusted feature vector; then, the noise vector and the first feature vector are input into the third visual neural network group to obtain a third adjusted feature vector; then, the first adjusted feature vector, the second adjusted feature vector and the third adjusted feature vector are fused to obtain an adjusted feature vector.

[0088] S223, inputting the adjusted feature vector into the multi-task network to obtain the feature vector to be processed output by each subtask network included in the multi-task network;

[0089] For example, when the subtask network includes a picture segmentation task network, a grayscale image generation task network, and a picture blurring task network, the feature vectors to be processed output by the multi-task network include the feature vectors to be processed output by the picture segmentation task network, the feature vectors to be processed output by the grayscale image generation task network, and the feature vectors to be processed output by the picture blurring task network.

[0090] Here, by inputting the adjusted feature vector into the multi-task network, the network can learn richer features and knowledge from different tasks.

[0091] The image generation model in this application can generate images under the constraints of causal relationships. This means that the network can understand the causal relationship between different factors and generate images according to logic. By considering the causal relationship, the generated defective cigarette box images are more reasonable and have geometric structures, which conform to the laws of the real world.

[0092] S224: Determine the to-be-processed feature vector output by each subtask network as the second feature vector of the original defective cigarette box sample image.

[0093] S23, performing image conversion on the second feature vector of the original defective cigarette box sample image to generate a new defective cigarette box sample image.

[0094] Based on the same inventive concept, an embodiment of the present application also provides a device for generating a defective cigarette box sample image corresponding to the method for generating a defective cigarette box sample image. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0095] See also Figure 2 , Figure 2 A schematic structural diagram of a device for generating defect sample images based on CGNET provided by an exemplary embodiment of the present application is shown.

[0096] like Figure 2 As shown in , the generating device 200 includes:

[0097] An acquisition module 210 is used to acquire an original defective cigarette box sample image;

[0098] A first image generation module 220, configured to input the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image;

[0099] The second image generating module 230 is used to determine the defective cigarette box sample image for training the cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image.

[0100] Optionally, the image generation model includes: a GAN network and a CGNET network; the first image generation module 220 is specifically used for:

[0101] Inputting the original defective cigarette box sample image into the GAN network to perform feature extraction on the original defective cigarette box sample image to obtain a first feature vector of the original defective cigarette box sample image;

[0102] Inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image;

[0103] Performing image transformation on the second feature vector of the original defective cigarette box sample image to generate a new defective cigarette box sample image.

[0104] Optionally, the CGNET network includes a noise generation network, a visual adjustment network and a multi-task network; the first image generation module 220 is specifically used for:

[0105] Inputting the first eigenvector into the noise generation network for noise generation processing to generate a noise vector;

[0106] Inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain an adjusted feature vector;

[0107] Inputting the adjusted feature vector into the multi-task network to obtain the feature vector to be processed output by each sub-task network included in the multi-task network;

[0108] The feature vectors to be processed output by each subtask network are determined as the second feature vectors of the original defective cigarette box sample image.

[0109] Optionally, the noise generation network includes a BN layer, a hidden layer, a noise layer and a fusion layer; the first image generation module 220 is specifically used to:

[0110] Inputting the first feature vector into the BN layer to perform vector normalization integration to obtain an integrated vector;

[0111] Inputting the integration vector into the hidden layer to obtain a split vector;

[0112] Inputting the split vector into the noise layer for vector enhancement to obtain a noise enhancement vector;

[0113] The noise enhancement vector and the split vector are input into the fusion layer for vector fusion to obtain a noise vector.

[0114] Optionally, the visual accommodation network includes a first visual neural network group, a second visual neural network group and a third visual neural network group; the first image generation module 220 is specifically used to:

[0115] Inputting the noise vector and the first feature vector into the first visual neural network group to obtain a first adjusted feature vector;

[0116] Inputting the noise vector and the first feature vector into the second visual neural network group to obtain a second adjusted feature vector;

[0117] Inputting the noise vector and the first feature vector into the third visual neural network group to obtain a third adjusted feature vector;

[0118] The first adjusted feature vector, the second adjusted feature vector and the third adjusted feature vector are fused to obtain an adjusted feature vector.

[0119] The embodiment of the present application provides a device for generating defective sample images based on CGNET, which uses an image generation model to generate new defective cigarette box sample images, thereby solving the problem of the small number of defective cigarette box sample images in the prior art, and improving the accuracy of the trained cigarette box defect detection model and the accuracy of defective cigarette boxes identified in the cigarette box production process.

[0120] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown in , the electronic device 300 includes a processor 310 , a memory 320 and a bus 330 .

[0121] The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 through the bus 330. When the machine-readable instructions are executed by the processor 310, the steps of the generation method in the above method embodiment can be executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0122] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the generation method in the above method embodiment can be executed. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0123] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0127] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0128] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for generating defect sample images based on CGNET, characterized in that: The generation method comprises: Obtaining an original defective cigarette box sample image; Inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image; Determine a defective cigarette box sample image for training a cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image; The image generation model includes: a GAN network and a CGNET network; the inputting the original defective cigarette box sample image into the pre-trained image generation model to obtain a new defective cigarette box sample image includes: Inputting the original defective cigarette box sample image into the GAN network to perform feature extraction on the original defective cigarette box sample image to obtain a first feature vector of the original defective cigarette box sample image; Inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image; Performing image conversion on the second eigenvector of the original defective cigarette box sample image to generate a new defective cigarette box sample image; The CGNET network includes a noise generation network, a visual adjustment network and a multi-task network; The step of inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image includes: Inputting the first eigenvector into the noise generation network for noise generation processing to generate a noise vector; Inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain an adjusted feature vector; Inputting the adjusted feature vector into the multi-task network to obtain the feature vector to be processed output by each sub-task network included in the multi-task network; Determine the to-be-processed feature vector output by each subtask network as the second feature vector of the original defective cigarette box sample image; The visual adjustment network includes a first visual neural network group, a second visual neural network group, and a third visual neural network group; the step of inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain the adjusted feature vector includes: Inputting the noise vector and the first feature vector into the first visual neural network group to obtain a first adjusted feature vector; Inputting the noise vector and the first feature vector into the second visual neural network group to obtain a second adjusted feature vector; Inputting the noise vector and the first feature vector into the third visual neural network group to obtain a third adjusted feature vector; The first adjusted feature vector, the second adjusted feature vector and the third adjusted feature vector are merged to obtain an adjusted feature vector; Among them, the multi-task network includes multiple sub-task networks, and the sub-task networks include at least one of a picture segmentation task network, a grayscale image generation task network, a picture blurring task network, and an image restoration task network; the picture segmentation task network is used to segment the defective cigarette box picture into the overall outline of the cigarette box and the outline of the defective part; the grayscale image generation task network is used to form a grayscale image of the defective cigarette box picture; the picture blurring task network is used to generate a blurred defective cigarette box picture from a clear defective cigarette box picture; and the image restoration task network is used to generate a clearer picture from the defective cigarette box picture.

2. The generation method according to claim 1, characterized in that: The noise generation network includes a BN layer, a hidden layer, a noise layer and a fusion layer; the first feature vector is input into the noise generation network for noise generation processing to generate a noise vector, including: Inputting the first feature vector into the BN layer to perform vector normalization integration to obtain an integrated vector; Inputting the integration vector into the hidden layer to obtain a split vector; Inputting the split vector into the noise layer for vector enhancement to obtain a noise enhancement vector; The noise enhancement vector and the split vector are input into the fusion layer for vector fusion to obtain a noise vector.

3. A device for generating defect sample images based on CGNET, characterized in that: The generating device comprises: An acquisition module, used for acquiring an original defective cigarette box sample image; A first image generation module, used for inputting the original defective cigarette box sample image into a pre-trained image generation model to obtain a new defective cigarette box sample image; A second image generation module, configured to determine a defective cigarette box sample image for training a cigarette box defect detection model based on the original defective cigarette box sample image and the new defective cigarette box sample image; The image generation model includes: a GAN network and a CGNET network; the first image generation module is specifically used for: Inputting the original defective cigarette box sample image into the GAN network to perform feature extraction on the original defective cigarette box sample image to obtain a first feature vector of the original defective cigarette box sample image; Inputting the first feature vector into the CGNET network for semantic segmentation to obtain a second feature vector of the original defective cigarette box sample image; Performing image conversion on the second eigenvector of the original defective cigarette box sample image to generate a new defective cigarette box sample image; The CGNET network includes a noise generation network, a visual adjustment network and a multi-task network; the first image generation module is specifically used for: Inputting the first eigenvector into the noise generation network for noise generation processing to generate a noise vector; Inputting the noise vector and the first feature vector into the visual adjustment network for vector adjustment to obtain an adjusted feature vector; Inputting the adjusted feature vector into the multi-task network to obtain the feature vector to be processed output by each sub-task network included in the multi-task network; Determine the to-be-processed feature vector output by each subtask network as the second feature vector of the original defective cigarette box sample image; The visual adjustment network includes a first visual neural network group, a second visual neural network group and a third visual neural network group; the first image generation module is specifically used to: Inputting the noise vector and the first feature vector into the first visual neural network group to obtain a first adjusted feature vector; Inputting the noise vector and the first feature vector into the second visual neural network group to obtain a second adjusted feature vector; Inputting the noise vector and the first feature vector into the third visual neural network group to obtain a third adjusted feature vector; The first adjusted feature vector, the second adjusted feature vector and the third adjusted feature vector are merged to obtain an adjusted feature vector; Among them, the multi-task network includes multiple sub-task networks, and the sub-task networks include at least one of a picture segmentation task network, a grayscale image generation task network, a picture blurring task network, and an image restoration task network; the picture segmentation task network is used to segment the defective cigarette box picture into the overall outline of the cigarette box and the outline of the defective part; the grayscale image generation task network is used to form a grayscale image of the defective cigarette box picture; the picture blurring task network is used to generate a blurred defective cigarette box picture from a clear defective cigarette box picture; and the image restoration task network is used to generate a clearer picture from the defective cigarette box picture.

4. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the generation method as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the generation method according to any one of claims 1 to 2 are executed.

Citation Information

Patent Citations

  • Image generation method of improved GAN model

    CN111027603A

  • Image defect detection method and device and electronic equipment

    CN112037223A

  • Neural network training method and device, defect detection method and device and storage medium

    CN114663337A

  • System and method for classifying products manufactured by a manufacturing process

    CN115375614A

  • Earphone cover defect detection method, device and equipment and storage medium

    CN116152209A