Glass bottle defect generation method and device, electronic device, and readable storage medium

Abstract

The present disclosure provides a method and device for generating defects in glass bottles, an electronic device, and a readable storage medium, and belongs to the field of glass bottle detection technology. The method includes: training an image generation backbone network based on a first data set. Training a defect perception attention generation network based on a second data set, and generating a third data set. The second and third data sets respectively include different numbers of glass bottle defect images. A first defect generation network model is obtained based on the trained defect perception attention generation network and the image generation backbone network. The first defect generation network model is trained based on the first data set, the second data set, and the third data set to obtain a target defect generation network model. The method and device for generating defects in glass bottles, the electronic device, and the readable storage medium provided by the present disclosure can generate more realistic, complex, and diverse glass bottle defect images using a large number of normal images and a small number of defective images, optimize unbalanced data sets, and improve the accuracy of defect detection.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of glass bottle detection, and more specifically, relates to a glass bottle defect generation method and device, electronic equipment, and a readable storage medium. Background Art

[0002] The production process of pharmaceutical glass bottles includes heating glass tubes, drawing and other processes. The types of defects produced are complex and varied. As the production process continues to improve, the number of defects in some products is relatively small, while the pharmaceutical industry's requirements for the quality of packaging materials are gradually increasing. It is easy to obtain a large number of defect-free image sets in the production line of pharmaceutical glass products, but defective images are scarce, and it is challenging to obtain a large number of defective images. The essence of deep learning-based product inspection methods is based on big data. The lack of defective samples will lead to problems such as overfitting of the training process, poor detection accuracy, and difficulty in launching the detection model. Therefore, how to generate defective samples is an extremely important part of the glass bottle defect generation process, and the defective samples generated by existing defective sample generation technologies lack diversity. Summary of the invention

[0003] The purpose of the present disclosure is to provide a method and device for generating defects in glass bottles, an electronic device, and a readable storage medium to solve the problem that defect samples generated by existing defect sample generation technologies lack diversity.

[0004] A first aspect of an embodiment of the present disclosure provides a method for generating defects in a glass bottle, comprising:

[0005] The image generation backbone network is trained based on the first data set. The first data set includes defect-free images of glass bottles. The image generation backbone network includes a genuine mapping network and a synthesis network.

[0006] The defect-aware attention generation network is trained based on the second data set, and a third data set is generated. The second data set includes a first number of glass bottle defect images. The third data set includes a second number of glass bottle defect images. The second number is greater than the first number. The defect-aware attention generation network includes a defect mapping network and a defect-aware generation network.

[0007] The first defect generation network model is obtained based on the trained defect-aware attention generation network and image generation backbone network.

[0008] The first defect generation network model is trained based on the first data set, the second data set, and the third data set to obtain a target defect generation network model.

[0009] A second aspect of the embodiments of the present disclosure provides a glass bottle defect generating device, comprising:

[0010] The first training module is used to train an image generation backbone network based on a first data set. The first data set includes defect-free images of glass bottles. The image generation backbone network includes a genuine mapping network and a synthesis network.

[0011] The first computing module is used to train a defect-aware attention generation network based on the second data set and generate a third data set. The second data set includes a first number of glass bottle defect images. The third data set includes a second number of glass bottle defect images. The second number is greater than the first number. The defect-aware attention generation network includes a defect mapping network and a defect-aware generation network.

[0012] The second computing module is used to obtain a first defect generation network model based on the trained defect perception attention generation network and image generation backbone network.

[0013] The second training module is used to train the first defect generation network model based on the first data set, the second data set and the third data set to obtain a target defect generation network model.

[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned glass bottle defect generation method when executing the computer program.

[0015] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned glass bottle defect generation method are implemented.

[0016] The beneficial effects of the glass bottle defect generation method and device, electronic device, and readable storage medium provided by the embodiments of the present disclosure are: the present application can simulate and generate defective images of glass bottles by training the image generation backbone network and the defect perception attention generation network. The introduction of the defect perception attention generation network can enhance the perception and capture of defect features, making the generated defective images more realistic, diverse, and in line with actual needs. Data enhancement is achieved by training the defect perception attention generation network with a limited number of glass bottle defect images and generating a larger number of glass bottle defect images in the third data set. This can not only solve the problem of insufficient defective image samples in practice, but also improve the robustness and generalization ability of subsequent model training.

[0017] The first defect generation network model is trained by combining defect-free images, real defect images and generated defect images to obtain the target defect generation network model. This multi-source data-driven training method can continuously optimize and improve the performance of the model, making the generated defect images more in line with the requirements of actual quality inspection. This method is not only suitable for defect generation of glass bottles, but can also be extended to defect simulation and generation of other similar products. By adjusting the network structure and training data, it can adapt to the defect characteristics and generation requirements of different products. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a process for generating defects in a glass bottle provided in an embodiment of the present disclosure;

[0020] Figure 2 A structural framework diagram of a glass bottle defect generation and discrimination network provided in an embodiment of the present disclosure;

[0021] Figure 3 A diagram of a defect perception generation network structure framework provided by an embodiment of the present disclosure;

[0022] Figure 4 A structural block diagram of a glass bottle defect generating device provided by an embodiment of the present disclosure;

[0023] Figure 5 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0025] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0026] Please refer to Figure 1 , Figure 1This is a flow chart of a method for generating defects in a glass bottle provided in one embodiment of the present disclosure. The method may include S101 to S104.

[0027] S101: training an image generation backbone network based on a first data set. The first data set includes defect-free images of glass bottles. The image generation backbone network includes a genuine mapping network and a synthesis network.

[0028] In this embodiment, the process of generating a backbone network based on the training image of the first data set may include:

[0029] The first data set is input into the image generation backbone network to obtain a first feature. The first feature includes a defect-free image feature.

[0030] The random sampling code is input into the genuine mapping network to obtain the first vector. The first vector is used to control the visual features.

[0031] Random noise is input into the genuine mapping network to obtain the scale factor.

[0032] A sample image of a glass bottle without defects is generated based on a first parameter, a first feature, a first vector, a scale factor and a synthesis network. The first parameter includes a constant feature map.

[0033] For example, Figure 2 As shown, the image generation backbone network can be composed of a genuine mapping network and a synthetic network. The genuine mapping network is applied to the generation process of defect-free sample images. Hundreds of defect-free images of glass bottles are collected, and these images are trained on the image generation backbone network. According to the fourth formula, defect-free image features (i.e., the first features) are extracted from the defect-free images of glass bottles.

[0034] The fourth formula is:

[0035] =M( )

[0036] in, represents random sampling encoding, M() represents feature extraction function, Indicates defect-free image features.

[0037] In this embodiment, the random sampling code is input into the genuine mapping network to obtain the first vector, which may include: the genuine mapping network may input the random sampling code into Mapped to a defect-free image of a glass bottle. The first vector w* is obtained according to the random sampling encoding, and the first vector w* is added to the image generation backbone network G to control different visual features.

[0038] For example, a set of random samples that conform to the Gaussian distribution can be encoded The input is sent to an N (8)-layer perceptron (a genuine mapping network composed of fully connected layers), which is encoded into an intermediate vector of the same size. The intermediate vector is copied 18 times and then mapped to the w+ space through an affine transformation A to generate the first vector w* that controls the change.

[0039] Exemplarily, the synthesis network can be composed of multiple (8) residual blocks with skip structures, and the feature map size is gradually upsampled from 4 to 1024. The constant feature map is input, and the first vector w* obtained by the genuine mapping network is received in the adaptive instance regularization part of each residual block, and the scale factor B whose size matches each layer of the synthesis network is output after each convolution layer. The scale factor B can be obtained by inputting random noise n into the mapping network, and finally generating a defect-free image of the glass bottle of the target scale.

[0040] S102: training a defect-aware attention generation network based on the second data set and generating a third data set. The second data set includes a first number of glass bottle defect images. The third data set includes a second number of glass bottle defect images. The second number is greater than the first number. The defect-aware attention generation network includes a defect mapping network and a defect-aware generation network.

[0041] In this embodiment, training a defect-aware attention generation network based on the second data set and generating a third data set may include: inputting the second data set into the defect-aware attention generation network, determining a latent vector set, a defect mask, and a second feature corresponding to the second data set. The second feature includes a plurality of multi-level defect features of the glass bottle defect image.

[0042] A third data set is generated based on the set of latent vectors, the defect mask, and the second feature.

[0043] Exemplarily, a defect mapping network and a defect perception generation network may be constructed, and the defect perception generation network may include a defect location attention guidance control module and a defect perception attention generation control module. The second data set (including the glass bottle defect image) is input into the defect perception attention generation control module. The defect perception attention generation control module gradually extracts multi-level defect features in the image through its multi-layer structure, such as convolutional layers and pooling layers. These features are encoded as second features, which describe in detail the properties of the defect, such as shape, size, location, and texture.

[0044] At the same time, the attention mechanism in the defect position attention guidance control module is activated. A corresponding defect mask is generated based on the second data set, and the defect mask information can be used to guide the defect perception generation network to generate defects at specific locations in the normal sample image.

[0045] Based on the defect mapping network, the latent vector set corresponding to the second data set can be obtained. The latent vector set, defect mask and second feature are used as input, reconstructed through the generator network, and the encoded information is decoded back into the form of an image. On this basis, the defect-aware generation network also uses the attention mechanism and multi-layer features to enhance the details and clarity of the defect area. Based on these input information, the defect-aware attention generation network can generate a third data set, which is a set of glass bottle defect images with more diverse and prominent defect features. Such a data set can be used for subsequent model training and testing to improve the model's defect recognition ability and generalization performance.

[0046] Exemplarily, in the process of constructing the defect location attention guidance control module, the defect location attention guidance control module can use mask information to guide the defect detection model to generate defects at specific locations of the defect-free image of the glass bottle. The mask M can be a binary image, where the white area value is 1, indicating the location where the defect should appear, and the black area value is 0, indicating the location without the defect. The mask is applied at each layer or key stage of the defect detection model to ensure that the defect only appears at the specified location.

[0047] S103: Obtain a first defect generation network model based on the trained defect perception attention generation network and image generation backbone network.

[0048] In this embodiment, the trained defect-aware attention generation network can be added to the image generation backbone network to obtain a first defect generation network model.

[0049] For example, a company is developing a network model for automatically detecting surface defects of industrial products. A defect-aware attention generation network can be trained, which can learn the key features of potential defects in the image and generate corresponding attention weights on the feature map. These weights can guide the network to focus on areas in the image that may contain defects. The trained defect-aware attention generation network is integrated into an image generation backbone network to form a first defect generation network model. This backbone network can be a generative adversarial network (GAN), which can include a generator for generating new images.

[0050] S104: Training the first defect generation network model based on the first data set, the second data set, and the third data set to obtain a target defect generation network model.

[0051] In this embodiment, the first defect generation network model may include a generator, a first discriminator, and a second discriminator.

[0052] The generator and the first discriminator are trained based on the first training set to obtain a first difference value. The first training set includes defect-free images of glass bottles.

[0053] The generator and the second discriminator are trained based on the first data set, the second data set and the third data set respectively to obtain a second difference value.

[0054] The generator, the first discriminator, and the second discriminator are updated based on the loss function until the first difference value is less than a first threshold value and the second difference value is less than a second threshold value.

[0055] A target defect generation network model is obtained based on the updated generator, the first discriminator, the second discriminator and the first defect generation network model.

[0056] In this embodiment, the generator and the first discriminator can be trained using a first training set containing only defect-free images of glass bottles. This step allows the generator to learn how to generate images similar to defect-free images, while the first discriminator can learn how to distinguish between real defect-free images and images generated by the generator, thereby obtaining a first difference value. The generator and the second discriminator are trained using a first data set (containing defect-free and defective images), a second data set (which may be enhanced defective images), and a third data set (which may be defective images under different angles or lighting conditions). This step allows the generator to learn how to generate or modify images to highlight defects, while the second discriminator learns how to distinguish between real defective images and images generated by the generator, thereby obtaining a second difference value. During the entire training process, the loss function can be used to guide the parameter update of the generator, the first discriminator, and the second discriminator until the first difference value is less than the set first threshold value, and the second difference value is less than the set second threshold value. Finally, based on the updated generator, the discriminator, and the first defect generation network model, a target defect generation network model can be obtained, which can be used for actual defect generation tasks.

[0057] For example, Figure 2 As shown in FIG. 1 , the generator and the discriminant network in the image generation backbone network can be jointly trained based on a small number of glass bottle defect images, defect mask images, and glass bottle defect-free images. The discriminant network includes a first discriminator D and a second discriminator The first discriminator D uses the same discriminator network structure as styleGAN2-ADA, which is used to generate and judge defect-free images of glass bottles. It is used to determine whether the generated glass bottle defect image matches the mask. The specific training process includes: fixing the constant c and the parameters of the image generation backbone network, and training along the defect perception generation network module (first defect generation network model). In this embodiment, the first defect generation network model is a model that has been trained to detect defects in glass bottles. The generator in the trained first detection model can be used to generate defect images of the corresponding category.

[0058] =S( )

[0059] Among them, the image generation backbone network consists of a mapping network and a synthesis network, S represents the synthesis network, represents the set of generated latent vectors.

[0060] It can be concluded from the above that this embodiment achieves the generation of more realistic and diverse glass bottle defect images based on a large number of normal images and a small number of defect images, optimizes the unbalanced data set, enhances the stability of deep learning detection model training, and improves product detection accuracy. By using multi-level defect features, this embodiment can more comprehensively capture the complexity and diversity of glass bottle defect images, thereby improving the accuracy and efficiency of defect detection. Expanding defect images not only increases the number of training samples, but also simulates various changes in the real world through data enhancement technology, improving the generalization ability of the model, and enabling it to handle a wider range of defect types and scenarios.

[0061] In one embodiment of the present disclosure, determining a potential vector set corresponding to the second data set includes:

[0062] Defect potential codes are obtained based on the second data set.

[0063] The defect latent code is augmented to obtain an extended vector.

[0064] The main direction vector is obtained based on the extended vector and the first formula. The main direction vector is the direction vector corresponding to the defect potential code.

[0065] A set of latent vectors is generated based on the principal direction vector and the second formula.

[0066] The first formula is:

[0067]

[0068] in, represents the main direction vector of the potential code of the k-th defect, represents the mean vector of the extended vector, Represents the mean vector Model.

[0069] The second formula is:

[0070]

[0071] in, represents the generated latent vector set, represents the initial latent vector set, Represents the parameter that controls the intensity of the defect.

[0072] Exemplarily, a defect mapping network can be constructed, and a latent vector set corresponding to the second data set can be obtained through the defect mapping network. Sample the k-th defect sample image, and map the random noise into an initial latent vector set of the latent space through the defect mapping network. The defect mapping network can use an 8-layer fully connected layer. Sample the k-th defect, input it into the defect mapping network, obtain the defect latent code of the latent space of the defect vector, and expand it to obtain an extended vector. Calculate the mean vector of these extended vectors and standardize them to obtain the main direction vector of the k-th defect latent code. The calculation formula of the main direction vector is:

[0073]

[0074] Based on the direction vector of the defect potential code, the potential vector of the k-th defect is generated. The calculation formula for generating the potential vector is:

[0075]

[0076] This embodiment can expand the potential vector set. By expanding the data of the defect potential code to obtain the extended vector, the diversity of the data can be increased, so that the model can learn more abundant defect features during the training process, thereby improving the generalization ability of the model. The method based on the main direction vector can more effectively capture the key features of the defect and improve the accuracy of defect detection.

[0077] In one embodiment of the present disclosure, determining a second feature corresponding to the second data set includes:

[0078] Based on the second data set, multi-scale features are acquired through multiple convolution kernels;

[0079] Obtaining defect perception features based on the multi-scale features;

[0080] Obtaining the second feature based on a third formula, the defect mask, and the defect perception feature;

[0081] The third formula is:

[0082] = *(1-M)+ *M

[0083] in, represents the defect feature of the i-th layer of multi-level defect features, represents the output feature of the image generation backbone network at the i-th layer, M represents the defect mask, represents the defect-aware features of the i-th layer.

[0084] Exemplarily, the defect perception generation control network can be composed of 5 residual feature enhancement modules. Different from ordinary residual blocks, the branch residual structure in this network adopts multi-scale convolution and integrates the attention mechanism to construct a defect feature generation network branch to extract multi-scale and multi-level defect features of defect features and enhance feature expression capabilities. In addition, deformable convolution is integrated to extract multi-level and rich defect feature detail information. The output feature expression of the defect feature synthesis network at an image resolution of k*k is:

[0085]

[0086] in, Represents multi-level defect characteristics, represents the output feature of the current layer, M represents the defect mask, Represents the defect perception feature of the current layer. Among them, It is the defect perception feature obtained by multi-scale feature fusion. The defect perception generation network structure is as follows Figure 3 As shown in the figure, the network can perform feature fusion on each residual block feature. First, three convolution kernels of different scales (3*3, 7*7, 13*13) can be used to jointly capture multi-scale feature information. A 1*1 convolution kernel is used to integrate features between different scales, and the channel attention mechanism is used to enhance the focus on key features, injecting more defect feature expressiveness into parameterization. In order to better handle defect target features of different shapes and sizes, the second 3*3 convolution in the residual block structure is designed as a deformable convolution, thereby further enhancing the model's ability to express defect target features.

[0087] In one embodiment of the present disclosure, updating the generator, the first discriminator and the second discriminator based on the loss function includes:

[0088] The loss function is:

[0089]

[0090] Among them, L represents the loss value, represents the first loss function, G represents the generator, D represents the first discriminator, represents the second loss function, represents the second discriminator, (G) represents the defect mask loss function, represents the weight adjustment parameter, represents the perceptual loss function, Represents the weight adjustment parameter.

[0091] The calculation formula of the first loss function is:

[0092]

[0093] in, (G,D) represents adversarial loss, (G) represents the path length regularization function, represents the regularization function.

[0094] The calculation formula of the perceptual loss function is:

[0095] =

[0096] in, represents the vector inner product, and denote the jth latent vector and the j+1th latent vector respectively, and m denotes the number of latent vectors.

[0097] For example, in order to measure the difference between the generated glass bottle defect image and the real defect image, and the difference between the real defect mask and the glass bottle defect image and the generated defect image mask, the generator and discriminator networks can be updated alternately by the following formula:

[0098]

[0099]

[0100] in, represents the loss function in StyleGAN2-ADA, represents the adversarial loss in styleGAN2-ADA, represents the path length regularization function for disentangling the latent space vector, Represents the regularization function, which is used to stabilize the training process. The loss function structure is similar to the loss function Ls of styleGAN2. The second discriminator acts on the defect mask and defect feature generation part. represents the defect mask loss, which is used to control the distance between different defect potential vectors and make different potential codes produce different defect features. represents the weight adjustment parameter, represents the perceptual loss, which is used to control the latent vector in the latent space to generate similar defect features. The coefficient Represents the weight adjustment parameter.

[0101]

[0102] in, represents the defect mask loss, and Represents two random defect codes respectively and The corresponding latent vector output after the defect mapping network, and Represent the latent vectors and the latent vector The corresponding defect mask is obtained. Represents the 1-norm of a vector.

[0103] =

[0104] During the fine-tuning process, the parameters of the image generation backbone network are fixed, and the defect-aware attention generation network part is fine-tuned. During the training process, ensure that the parameters of the deformable convolution layer and the attention module are optimized at the same time. The weights of the defect-aware attention generation network are updated through the back-propagation algorithm and appropriate Adam. Finally, the target defect generation network model is obtained.

[0105] The loss function used in this embodiment comprehensively considers the adversarial loss, path length regularization function and defect mask loss function, which helps the model to optimize the image generation quality and defect detection accuracy during the training process. Adversarial loss can make the generated images more realistic, the path length regularization function helps to maintain the smoothness of the generated images in the latent space, and the defect mask loss function ensures that the model can accurately locate and generate defects. The introduction of the perceptual loss function further enhances the performance of the model. It captures the similarity and change trend between vectors by calculating the inner product between latent vectors, thereby helping the model to generate more continuous and natural defect images. This helps to improve the robustness and generalization ability of the model in defect generation and detection tasks.

[0106] Corresponding to the method for generating defects in glass bottles in the above embodiment, Figure 4 This is a structural block diagram of a glass bottle defect generation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 4 , the glass bottle defect generating device 20 comprises:

[0107] The first training module 21 is used to train the image generation backbone network based on the first data set. The first data set includes defect-free images of glass bottles. The image generation backbone network includes a genuine mapping network and a synthesis network.

[0108] The first computing module 22 is used to train the defect perception attention generation network based on the second data set and generate a third data set. The second data set includes a first number of glass bottle defect images. The third data set includes a second number of glass bottle defect images. The second number is greater than the first number. The defect perception attention generation network includes a defect mapping network and a defect perception generation network.

[0109] The second computing module 23 is used to obtain a first defect generation network model based on the trained defect perception attention generation network and image generation backbone network.

[0110] The second training module 24 is used to train the first defect generation network model based on the first data set, the second data set and the third data set to obtain a target defect generation network model.

[0111] In one embodiment of the present disclosure, the first training module 21 is specifically used to input the first data set into the image generation backbone network to obtain the first feature. The first feature includes a defect-free image feature.

[0112] The random sampling code is input into the genuine mapping network to obtain the first vector. The first vector is used to control the visual features.

[0113] Random noise is input into the genuine mapping network to obtain the scale factor.

[0114] A sample image of a glass bottle without defects is generated based on a first parameter, a first feature, a first vector, a scale factor and a synthesis network. The first parameter includes a constant feature map.

[0115] In one embodiment of the present disclosure, the first calculation module 22 includes:

[0116] The first computing unit is used to input the second data set into the defect-aware attention generation network to determine a latent vector set, a defect mask, and a second feature corresponding to the second data set. The second feature includes multiple multi-level defect features of the glass bottle defect image.

[0117] A third data set is generated based on the set of latent vectors, the defect mask, and the second feature.

[0118] In one embodiment of the present disclosure, the second training module 24 includes:

[0119] The second computing unit is used for the first defect generation network model including a generator, a first discriminator and a second discriminator.

[0120] The generator and the first discriminator are trained based on the first training set to obtain a first difference value. The first training set includes defect-free images of glass bottles.

[0121] The generator and the second discriminator are trained based on the first data set, the second data set and the third data set respectively to obtain a second difference value.

[0122] The generator, the first discriminator, and the second discriminator are updated based on the loss function until the first difference value is less than a first threshold value and the second difference value is less than a second threshold value.

[0123] A target defect generation network model is obtained based on the updated generator, the first discriminator, the second discriminator and the first defect generation network model.

[0124] In an embodiment of the present disclosure, the first computing unit is specifically configured to obtain a defect potential code based on the second data set.

[0125] The defect latent code is augmented to obtain an extended vector.

[0126] The main direction vector is obtained based on the extended vector and the first formula. The main direction vector is the direction vector corresponding to the defect potential code.

[0127] A set of latent vectors is generated based on the principal direction vector and the second formula.

[0128] The first formula is:

[0129]

[0130] in, represents the main direction vector of the potential code of the k-th defect, represents the mean vector of the extended vector, Represents the mean vector Model.

[0131] The second formula is:

[0132]

[0133] in, represents the generated latent vector set, represents the initial latent vector set, Represents the parameter that controls the intensity of the defect.

[0134] In an embodiment of the present disclosure, the first computing unit is further configured to acquire multi-scale features through multiple convolution kernels based on the second data set.

[0135] Defect perception features are obtained based on multi-scale features.

[0136] A second feature is obtained based on the first formula, the defect mask and the defect perception feature.

[0137] The first formula is:

[0138] = *(1-M)+ *M

[0139] in, represents the defect feature of the i-th layer of multi-level defect features, represents the output feature of the image generation backbone network at layer i, M represents the defect mask, represents the defect-aware features of the i-th layer.

[0140] In one embodiment of the present disclosure, the second computing unit is specifically used for a loss function:

[0141]

[0142] Among them, L represents the loss value, represents the first loss function, G represents the generator, D represents the first discriminator, represents the second loss function, represents the second discriminator, (G) represents the defect mask loss function, represents the weight adjustment parameter, represents the perceptual loss function, Represents the weight adjustment parameter.

[0143] The calculation formula of the first loss function is:

[0144]

[0145] in, (G,D) represents adversarial loss, (G) represents the path length regularization function, represents the regularization function.

[0146] The calculation formula of the perceptual loss function is:

[0147] =

[0148] in, represents the vector inner product, and They represent the j-th latent vector and the j+1-th latent vector respectively, and m represents the number of latent vectors.

[0149] See also Figure 5 , Figure 5 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 5The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of each module / unit in the above-mentioned device embodiments, such as Figure 4 The functions of modules 21 to 24 are shown.

[0150] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0151] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0152] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0153] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the glass bottle defect generation method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device described in the embodiments of the present disclosure, which will not be repeated here.

[0154] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0155] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0156] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0157] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0158] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0160] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0161] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for generating defects in a glass bottle, characterized in that: include: An image generation backbone network is trained based on a first data set; the first data set includes defect-free images of glass bottles; the image generation backbone network includes a genuine mapping network and a synthesis network; the genuine mapping network is applied to the generation process of defect-free sample images; the synthesis network is composed of a plurality of residual blocks with a jump structure; Training a defect-aware attention generation network based on the second data set and generating a third data set; the second data set includes a first number of glass bottle defect images; the third data set includes a second number of glass bottle defect images; the second number is greater than the first number; The defect-aware attention generation network includes a defect mapping network and a defect-aware generation network; The method of training a defect-aware attention generation network based on the second data set and generating a third data set includes: inputting the second data set into the defect-aware attention generation network, determining a latent vector set, a defect mask, and a second feature corresponding to the second data set; the second feature includes a plurality of multi-level defect features of a defect image of a glass bottle; generating a third data set based on the set of latent vectors, the defect mask, and the second feature; The defect perception generation network includes a defect location attention guidance control module and a defect perception attention generation control module, and the defect perception attention generation control module includes a convolution layer and a pooling layer; The step of inputting the second data set into the defect-aware attention generation network and determining a potential vector set, a defect mask, and a second feature corresponding to the second data set comprises: Inputting the second data set into the defect perception attention generation control module, extracting multi-level defect features of the glass bottle defect image through the convolution layer and the pooling layer, and using the multi-level defect features as the second features; the multi-level defect features include defect shape, defect size, defect position and defect texture; activating the attention mechanism in the defect position attention guidance control module, generating a defect mask corresponding to the second data set based on the second data set; guiding the defect perception generation network to generate defects on the defect-free sample image based on the defect mask; Determine a latent vector set corresponding to the second data set based on the defect mapping network; input the latent vector set, the defect mask and the second feature into a generator network for reconstruction, and decode the encoded information to generate a third data set; Obtaining a first defect generation network model based on the trained defect perception attention generation network and the image generation backbone network, including: adding the trained defect perception attention generation network to the image generation backbone network to obtain a first defect generation network model; the first defect generation network model includes a generator, a first discriminator and a second discriminator; The first defect generation network model is trained based on the first data set, the second data set, and the third data set to obtain a target defect generation network model, including: The generator and the first discriminator are trained respectively based on a first training set to obtain a first difference value; the first training set includes defect-free images of glass bottles; Based on the first data set, the second data set and the third data set, the generator and the second discriminator are trained respectively to obtain a second difference value; updating the generator, the first discriminator, and the second discriminator based on the loss function until the first difference value is less than a first threshold value and the second difference value is less than a second threshold value; The target defect generation network model is obtained based on the updated generator, the first discriminator, the second discriminator and the first defect generation network model.

2. A method for generating defects in a glass bottle according to claim 1, characterized in that: The generating a backbone network based on the training image of the first data set comprises: Inputting the first data set into the image generation backbone network to obtain a first feature; the first feature includes a defect-free image feature; Inputting the random sampling code into the genuine mapping network to obtain a first vector; the first vector is used to control the visual feature; Input random noise into the genuine mapping network to obtain the scale factor; A sample image of a glass bottle without defects is generated based on a first parameter, the first feature, the first vector, the scale factor and the synthesis network; the first parameter includes a constant feature map.

3. A method for generating defects in a glass bottle according to claim 1, characterized in that: Determining a potential vector set corresponding to the second data set includes: Obtaining a defect potential code based on the second data set; Performing data expansion on the defect potential code to obtain an extended vector; A main direction vector is obtained based on the extended vector and the first formula; the main direction vector is a direction vector corresponding to the defect potential code; generating the potential vector set based on the main direction vector and a second formula; The first formula is: in, represents the main direction vector of the potential code of the k-th defect, represents the mean vector of the extended vector, Represents the mean vector Model; The second formula is: in, represents the generated latent vector set, represents the initial latent vector set, Represents the parameter that controls the magnitude of the defect intensity.

4. A method for generating defects in a glass bottle according to claim 1, characterized in that: Determining a second feature corresponding to the second data set includes: Based on the second data set, multi-scale features are acquired through multiple convolution kernels; Obtaining defect perception features based on the multi-scale features; Obtaining the second feature based on a third formula, the defect mask, and the defect perception feature; The third formula is: = *(1-M)+ *M in, represents the defect feature of the i-th layer of multi-level defect features, represents the output feature of the image generation backbone network at the i-th layer, M represents the defect mask, represents the defect-aware features of the i-th layer.

5. A method for generating defects in a glass bottle as claimed in claim 1, characterized in that: The updating of the generator, the first discriminator and the second discriminator based on the loss function comprises: The loss function is: Among them, L represents the loss value, represents the first loss function, G represents the generator, D represents the first discriminator, represents the second loss function, represents the second discriminator, (G) represents the defect mask loss function, represents the weight adjustment parameter, represents the perceptual loss function, represents the weight adjustment parameter; The calculation formula of the first loss function is: in, (G,D) represents adversarial loss, (G) represents the path length regularization function, represents the regularization function; The calculation formula of the perceptual loss function is: = in, represents the vector inner product, and denote the jth latent vector and the j+1th latent vector respectively, and m denotes the number of latent vectors.

6. A glass bottle defect generating device, characterized in that: include: A first training module is used to train an image generation backbone network based on a first data set; the first data set includes defect-free images of glass bottles; the image generation backbone network includes a genuine mapping network and a synthesis network; the genuine mapping network is applied to the generation process of defect-free sample images; the synthesis network is composed of a plurality of residual blocks with a jump structure; A first computing module is used to train a defect-aware attention generation network based on a second data set and generate a third data set; the second data set includes a first number of glass bottle defect images; the third data set includes a second number of glass bottle defect images; the second number is greater than the first number; The defect-aware attention generation network includes a defect mapping network and a defect-aware generation network; The first computing module is specifically used to input the second data set into the defect perception attention generation network to determine a potential vector set, a defect mask and a second feature corresponding to the second data set; the second feature includes multiple multi-level defect features of the glass bottle defect image; generating a third data set based on the set of latent vectors, the defect mask, and the second feature; The defect perception generation network includes a defect location attention guidance control module and a defect perception attention generation control module, and the defect perception attention generation control module includes a convolution layer and a pooling layer; The first calculation module is further specifically used to input the second data set into the defect perception attention generation control module, extract multi-level defect features of the glass bottle defect image through the convolution layer and the pooling layer, and use the multi-level defect features as the second features; the multi-level defect features include defect shape, defect size, defect position and defect texture; activating an attention mechanism in the defect position attention guidance control module, and generating a defect mask corresponding to the second data set based on the second data set; Instructing the defect-aware generation network to generate defects on a defect-free sample image based on the defect mask; Determining a potential vector set corresponding to the second data set based on the defect mapping network; Inputting the latent vector set, the defect mask and the second feature into a generator network for reconstruction, and decoding the encoded information to generate a third data set; A second computing module, for obtaining a first defect generation network model based on the trained defect perception attention generation network and the image generation backbone network, comprising: adding the trained defect perception attention generation network to the image generation backbone network to obtain a first defect generation network model; the first defect generation network model includes a generator, a first discriminator and a second discriminator; A second training module, configured to train the first defect generation network model based on the first data set, the second data set, and the third data set to obtain a target defect generation network model; The second training module is specifically used to train the generator and the first discriminator respectively based on the first training set to obtain a first difference value; the first training set includes defect-free images of glass bottles; Based on the first data set, the second data set and the third data set, the generator and the second discriminator are trained respectively to obtain a second difference value; updating the generator, the first discriminator, and the second discriminator based on the loss function until the first difference value is less than a first threshold value and the second difference value is less than a second threshold value; The target defect generation network model is obtained based on the updated generator, the first discriminator, the second discriminator and the first defect generation network model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images