Image augmentation method and device based on deep learning, storage medium and computer

Abstract

This invention provides an image augmentation method, apparatus, storage medium, and computer. The image augmentation method includes: providing a first image set comprising multiple original images and a generative adversarial network (GAN); processing the original images to obtain processed images; the processed images constitute a second image set; a third image set is formed based on the first and second image sets; training the GAN using images from the third image set to obtain a final GAN; the final GAN ​​generating multiple generated images based on the third image set, and determining the authenticity of the generated images; the generated images judged as authentic form a fourth image set; and a fifth image set is formed based on the third and fourth image sets. This effectively augments the image set with high-quality images, providing a superior image source for training convolutional neural network models for subsequent image classification or object detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep learning, in particular to an image expansion method and device based on deep learning, a storage medium and a computer. BACKGROUND

[0002] Deep learning has high prediction accuracy in recognition applications, and performs well in image classification and image semantic segmentation, mainly using convolutional neural networks (CNN) to find features in images, or to separate targets from backgrounds, or to distinguish different categories of targets.

[0003] However, the premise of establishing a high-quality neural network model is to have a high-quality image training set, which not only has a certain quantity requirement, but also needs to have a certain accommodation for environmental differences, which is particularly important in industrial applications, because there are many uncontrollable factors that cause picture changes, and the decline of hardware performance such as light source, camera, lens, and external environmental changes and interference will affect the robustness of the model, but cannot be fully reflected in the existing image data.

[0004] Therefore, an effective image expansion method is an important way to achieve high adaptability in industrial environments. SUMMARY

[0005] The purpose of the present application is to provide an image expansion method and device, a storage medium and a computer, which can effectively expand a high-quality image set to provide a high-quality image source for subsequent convolutional neural network model training for image classification or target detection.

[0006] To achieve the purpose of the application, according to one aspect of the present application, an image expansion method is provided, which includes: providing a first image set and a generative adversarial network, wherein the first image set includes a plurality of original images, and the generative adversarial network includes a generative model and a discriminative model; performing image processing on the original images to obtain processed images, and a plurality of processed images constitute a second image set, wherein the image processing includes one or more of graphic transformation, light effect processing, color and detail processing, and special processing, and a third image set is formed based on the first image set and the second image set; training the generative adversarial network using images in the third image set to obtain a final generative adversarial network; the generative model of the final generative adversarial network generates a plurality of generated images based on the third image set, and the discriminative model of the final generative adversarial network judges the authenticity of the plurality of generated images, and a plurality of generated images judged to be true form a fourth image set, and a fifth image set is formed based on the third image set and the fourth image set.

[0007] According to another aspect of the present application, the present application provides an image expansion device, comprising: a pre-expansion module, which performs image processing on a plurality of original images in a first image set to obtain a plurality of processed images, the plurality of processed images constituting a second image set, and a third image set is formed based on the first image set and the second image set, wherein the image processing comprises one or more of graphic transformation, light effect processing, color and detail processing, and special processing; a generative adversarial network comprising a generative model and a discriminative model, the generative adversarial network being trained using images in the third image set to obtain a final generative adversarial network, the generative model of the final generative adversarial network generating a plurality of generated images based on the third image set, the discriminative model of the final generative adversarial network determining the authenticity of the plurality of generated images, and a fourth image set being formed by the plurality of generated images determined to be true, wherein a fifth image set is formed based on the third image set and the fourth image set.

[0008] According to another aspect of the present application, the present application provides a storage medium storing program instructions, the program being executed to perform the above-mentioned image expansion method. The image expansion method comprises: providing a first image set and a generative adversarial network, wherein the first image set comprises a plurality of original images, and the generative adversarial network comprises a generative model and a discriminative model; performing image processing on the original images to obtain a plurality of processed images, the plurality of processed images constituting a second image set, wherein the image processing comprises one or more of graphic transformation, light effect processing, color and detail processing, and special processing, and a third image set is formed based on the first image set and the second image set; training the generative adversarial network using images in the third image set to obtain a final generative adversarial network; the generative model of the final generative adversarial network generating a plurality of generated images based on the third image set, the discriminative model of the final generative adversarial network determining the authenticity of the plurality of generated images, and a fourth image set being formed by the plurality of generated images determined to be true, and a fifth image set being formed based on the third image set and the fourth image set.

[0009] According to another aspect of the present invention, a computer is provided, comprising a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to perform the image augmentation method described above. The image augmentation method includes: providing a first image set and a generative adversarial network (GAN), wherein the first image set includes multiple original images, and the GAN includes a generative model and a discriminative model; performing image processing on the original images to obtain processed images, wherein the multiple processed images constitute a second image set, wherein the image processing includes one or more of graphic transformation, lighting effect processing, color and detail processing, and special processing; forming a third image set based on the first and second image sets; training the GAN using images from the third image set to obtain a final GAN; the generative model of the final GAN ​​generates multiple generated images based on the third image set; the discriminative model of the final GAN ​​determines the authenticity of the multiple generated images; the multiple generated images determined to be authentic form a fourth image set; and a fifth image set is formed based on the third and fourth image sets.

[0010] Compared with existing technologies, this invention uses a combination of traditional image processing and generative adversarial networks (GANs) to effectively augment images, providing high-quality image sources for training convolutional neural network models for subsequent image classification or object detection. This makes the models compatible with multiple interference factors, resulting in higher accuracy in industrial environments with multiple interferences. Attached Figure Description

[0011] Figure 1 This is a schematic flowchart of one embodiment of the image augmentation method of the present invention;

[0012] Figure 2 This is a schematic flowchart of one embodiment of the image augmentation device of the present invention;

[0013] Figure 3 This is a schematic diagram illustrating the expansion principle of the image augmentation scheme in this invention;

[0014] Figure 4 This is a schematic diagram of the generative model of the generative adversarial network in this invention in one embodiment;

[0015] Figure 5 This is a schematic diagram of the discriminative model of the generative adversarial network in one embodiment of the present invention. Detailed Implementation

[0016] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of the specific implementation methods, structure, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0017] Figure 1 This is a schematic flowchart of one embodiment of the image augmentation method 100 of the present invention. Figure 1 As shown, the image augmentation method 100 includes the following steps or operations.

[0018] Step 110: Provide a first image set and a Generative Adversarial Network (GAN), wherein the first image set includes multiple original images.

[0019] In this paper, an image is used for subsequent traditional image processing and training of the generative adversarial network. The original image can be acquired by a camera through capture. For example... Figure 3 As shown, it only illustrates one original image, but there can actually be many. The content of this original image is a chick, but obviously, the content of the original image can be any other content that needs to be classified or detected.

[0020] The generative adversarial network (GAN) is a deep learning model that includes a generative model and a discriminative model, where the game-like learning between the generative and discriminative models produces fairly good outputs. Clearly, the first image set and the GAN can be provided in the same step or in different steps, depending on the needs and choices.

[0021] As mentioned in the background, the first image set is usually small and needs to be effectively expanded in order to provide a high-quality image source for training convolutional neural network models for subsequent image classification or object detection.

[0022] Step 120: Perform image processing on the original image to obtain a processed image. Multiple processed images constitute a second image set. The image processing includes one or more of the following: graphic transformation, light effect processing, color and detail processing, and special processing. A third image set is formed based on the first image set and the second image set.

[0023] Step 120 is used to expand the first image set using conventional image processing methods.

[0024] Considering potential influencing factors in actual shooting environments, traditional image processing is applied to the first image set, primarily including four aspects: firstly, graphic transformations, including flipping, rotation, distortion, and scaling; secondly, lighting effects, including brightness, contrast, highlights and shadows, and lighting gradation; thirdly, color and detail, including saturation, hue, color temperature, noise, and sharpness; and fourthly, special processing such as occlusion and distortion. In practice, appropriate traditional image processing algorithms are selected to address potential changes in the shooting environment, either individually or in combination, to augment the first image set, generating a second image set. Finally, a portion or all of the first image set and a portion or all of the second image set are integrated together to form a third image set. Figure 3 As described above, the first image set, after pre-expansion (i.e., step 120), together with the first image set, forms the third image set.

[0025] Shooting conditions suitable for industrial inspection are often imperfect, and various interferences can occur: Interference in the field of view, commonly due to dirt on the lens surface or people or equipment entering the field of view, can be addressed by adding additional target-occluded images or randomly adding masks of random sizes within a certain range to the original image; Unpredictable lighting conditions, commonly due to changes in light source intensity or displacement, can be addressed by adjusting parameters such as brightness and contrast within a certain range, and systematically altering the pixel values ​​of the original image; Blurred field of view, commonly due to changes in camera focus, the presence of smoke or other factors in the field of view, or hardware performance degradation, can be addressed by appropriately enhancing sharpness, blurring, or adding Gaussian noise, and performing pixel filtering and convolution operations on the original image; Displaced products, not fixed within the field of view, can be addressed by horizontally and vertically flipping, rotating, scaling, and randomly cropping the original image. All possible factors should be considered as much as possible to provide better directionality for building the GAN.

[0026] Step 130: Use images from the third image set to train the generative adversarial network, so as to maximize the judgment accuracy of the discriminative model and maximize the ability of the generative model to deceive the discriminative model, thus obtaining the final generative adversarial network.

[0027] Figure 4 This is a schematic diagram of the generative model 240 of the generative adversarial network in one embodiment of the present invention. Figure 5 This is a schematic diagram of the discriminative model 250 of the generative adversarial network in one embodiment of the present invention. Figure 4 and 5As shown, the generation model 240 includes an encoder 241, a noise addition module 242, and a decoder 243. The encoder 241 includes convolutional layers, pooling layers, multiple dense blocks, and multiple transition layers connected to the outputs of the corresponding dense blocks. The decoder 243 includes multiple dense blocks and multiple deconvolutional transition layers (i.e., ...) connected to the outputs of the corresponding dense blocks. Figure 4 (The "deconvolution + upsampling" in the image). The noise addition module 242 is connected between the output of the encoder 241 and the input of the decoder 243. The decoder 243 outputs a generated image of the same size as the input image. The discrimination model 250 is a binary classification model, which includes a pooling layer, a convolutional layer, multiple dense blocks, a transition layer connecting the multiple dense blocks, a global pooling layer, and a fully connected layer.

[0028] When an input image is input into the generative adversarial network (GAN), the encoder of the generative model 240 first extracts features from the input image, and the noise addition module 242 adds random noise. Then, the decoder 243 of the generative model outputs a generated image of the same size as the input image. The generated image is then input into the discriminative model 250, which extracts features from the generated image. Simultaneously or beforehand, the discriminative module 250 also performs feature extraction on the input image (…). Figure 4 The discrimination module 250 performs feature extraction on the input image and outputs the probability value of the true or false category of the generated image based on the features extracted from the generated image and the features extracted from the input image, and finally determines whether the generated image is true or false.

[0029] Specifically, such as Figure 4 and 5 As shown, the encoder 241 includes one convolutional layer, one pooling layer, four dense blocks, and four transition layers. The convolutional layer, pooling layer, and four dense blocks of the encoder 241 are sequentially connected. The four transition layers are respectively connected to the output of the corresponding dense block. Each dense block is connected to each transition layer located after it. The decoder 243 includes four dense blocks and four deconvolutional transition layers respectively connected to the output of the corresponding dense blocks. Each dense block is connected to each deconvolutional transition layer located after it. The discriminant model 250 includes one pooling layer, one convolutional layer, four dense blocks, three transition layers respectively connected between the four dense blocks, one global pooling layer, and one fully connected layer, sequentially connected. Each dense block is connected to each transition layer located after it.

[0030] The generative model 240 and the discriminative model 250 are trained alternately using images from the third image set. The loss function is minimized using gradient descent, and the parameters of the generative model 240 and the discriminative model 250 are adjusted inversely. After iterative training a specified number of times, a smaller loss function is obtained, generating the final generative adversarial network (GAN). It is important to emphasize that the "final GAN" refers to the GAN obtained after training. In reality, during subsequent use, the "final GAN" continues to learn and evolve.

[0031] In step 140, the final generative adversarial network (GAN) generative model 240 generates multiple generated images based on the third image set, and the final discriminative model 250 of the GAN determines the authenticity of the multiple generated images. The multiple generated images judged to be true form a fourth image set, and the generated images judged to be false are discarded. A fifth image set is formed based on the third image set and the fourth image set. In this way, the image set is further expanded using the generative adversarial network (GAN).

[0032] Specifically, the fifth image set includes part or all of the third image set and part or all of the fourth image set. For example... Figure 3 As shown, the third image set is further expanded by a generative adversarial network, and then integrated with the third image set to form the fifth image set.

[0033] In subsequent applications, the images in the fifth image set can be labeled, and the labeled fifth image set can be used to train a convolutional neural network (CNN) model for image classification or object detection, thus obtaining the trained CNN model for image classification or object detection.

[0034] One of the advantages of this invention is that it can fully consider the interference that may occur in the complex image shooting environment. First, the original image set is pre-expanded using traditional image processing methods, and then the pre-expanded image set is further expanded using GAN. Since the pre-expansion provides directional guidance for the further expansion of GAN, GAN can generate higher quality expanded images for the next step of training and detection, thereby enhancing the overall robustness of detection.

[0035] Application Examples

[0036] Image classification CNN is used to detect the opening and closing of baffles in the production area of ​​a company's workshop. The shooting environment in the workshop is poor. There may be stains on the camera lens surface, the lighting conditions around the camera are sometimes bright and sometimes dark, and there may be a high concentration of dust or water vapor in the field of view, which will affect the image quality. The position of the baffle is not fixed, which may result in incomplete images of the baffle in the shooting scene. The detection is required to be compatible with these interference factors.

[0037] Challenges in detection: Poor shooting environment conditions and very few existing images capturing interference factors. Using convolutional neural networks (CNNs) for image classification based on deep learning, the lack of richness in the image training set makes it difficult to achieve the required accuracy in actual detection.

[0038] While keeping the original image classification convolutional neural network unchanged, this invention employs an image data augmentation method that combines traditional image processing and GAN generation. The original image set is first processed using traditional image processing algorithms: to address the possibility of incomplete baffle imaging, random cropping, random small-angle rotation, and random occlusion of small areas are employed; to address unstable image quality, brightness, contrast, and blur variations are applied within a certain range, either individually or in combination, to generate a new image set. The original and new image sets are then used as training sets and fed into the GAN for training. When a smaller loss function is achieved, the final GAN ​​is generated, producing an image set with correct directionality. This image set is then fed into the original image classification convolutional neural network to train and generate the new image classification convolutional neural network.

[0039] Finally, a test set was input into the image classification convolutional neural network. This set contained problem images with known interference factors that were not present in the original image set. The trained image classification convolutional neural network was used for classification, and the location was correctly determined with a high overall accuracy.

[0040] According to another aspect of the invention, the invention can also be implemented as an image augmentation device. Figure 2 This is a schematic diagram of the image augmentation device 200 in one embodiment of the present invention. The image augmentation device 200 includes a pre-augmentation module 210 and a generative adversarial network 230.

[0041] The pre-expansion module 210 processes multiple original images in the first image set to obtain processed images. These processed images constitute a second image set. A third image set is formed based on the first and second image sets. The image processing includes one or more of the following: graphic transformation, lighting effect processing, color and detail processing, and special processing. The generative adversarial network 230 includes a generative model 240 and a discriminative model 250. The generative adversarial network 230 is trained using images from the third image set to maximize the judgment accuracy of the discriminative model 250 and maximize the ability of the generative model 240 to deceive the discriminative model 250, resulting in the final generative adversarial network 230. The generative model of the final generative adversarial network 230 generates multiple generated images based on the third image set. The discriminative model 250 of the final generative adversarial network judges the authenticity of the multiple generated images. The multiple generated images judged as genuine form a fourth image set. A fifth image set is formed based on the third and fourth image sets.

[0042] In one embodiment, images in the fifth image set are labeled, and the labeled fifth image set is used to train a convolutional neural network model for image classification or object detection, resulting in a trained convolutional neural network model. The image transformations include one or more of flipping, rotation, distortion, and scaling; the lighting effects processing includes one or more of brightness processing, contrast processing, highlight and shadow processing, and illumination gradient processing; the color and detail processing includes one or more of saturation processing, hue processing, color temperature processing, noise reduction, and sharpness processing; the special processing includes one or more of occlusion processing and deformation processing; the third image set includes part or all of the first image set and part or all of the second image set; and the fifth image set includes part or all of the third image set and part or all of the fourth image set.

[0043] In one embodiment, the generative model includes an encoder, a noise-adding module, and a decoder. The encoder includes a convolutional layer, a pooling layer, multiple dense blocks, and multiple transition layers connected to the outputs of the corresponding dense blocks. The decoder includes multiple dense blocks and multiple deconvolutional transition layers connected to the outputs of the corresponding dense blocks. The noise-adding module is connected between the output of the encoder and the input of the decoder. The decoder outputs a generated image of the same size as the input image. The discriminative model is a binary classification model, which includes a pooling layer, a convolutional layer, multiple dense blocks, transition layers connected between the multiple dense blocks, a global pooling layer, and a fully connected layer.

[0044] When an input image is input into the generative adversarial network, the encoder of the generative model first extracts features from the input image, and then the noise addition module adds random noise. The decoder of the generative model then outputs a generated image of the same size as the input image. This generated image is then input into the discriminative model, which extracts features from the generated image. The discriminative module also extracts features from the input image. Based on the features extracted from the generated image and the features extracted from the input image, the discriminative module outputs a probability value for the generated image's true / false category, ultimately determining whether the generated image is true or false.

[0045] Since the image augmentation device 200 is technically identical to the image augmentation method 100, the repetitive parts will not be repeated here.

[0046] According to another aspect of the present invention, a storage medium is provided that stores program instructions, which are executed to perform the above-described image augmentation method.

[0047] The image augmentation method includes: providing a first image set and a generative adversarial network (GAN), wherein the first image set includes multiple original images, and the GAN includes a generative model and a discriminative model; performing image processing on the original images to obtain processed images, the multiple processed images constituting a second image set, wherein the image processing includes one or more of the following: graphic transformation, lighting effect processing, color and detail processing, and special processing; forming a third image set based on the first and second image sets; training the GAN using images from the third image set to maximize the judgment accuracy of the discriminative model and maximize the ability of the generative model to deceive the discriminative model, resulting in a final GAN; the generative model of the final GAN ​​generates multiple generated images based on the third image set, and the discriminative model of the final GAN ​​judges the authenticity of the multiple generated images; the multiple generated images judged as genuine form a fourth image set; and a fifth image set is formed based on the third and fourth image sets. The remaining steps of the image augmentation method 100 are not repeated here.

[0048] According to another aspect of the present invention, a computer is provided, comprising a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to perform the image augmentation method described above. The image augmentation method includes: providing a first image set and a generative adversarial network (GAN), wherein the first image set includes multiple original images, and the GAN includes a generative model and a discriminative model; performing image processing on the original images to obtain processed images, wherein the multiple processed images constitute a second image set, wherein the image processing includes one or more of graphic transformation, lighting effect processing, color and detail processing, and special processing; forming a third image set based on the first and second image sets; training the GAN using images from the third image set to maximize the judgment accuracy of the discriminative model and maximize the ability of the generative model to deceive the discriminative model, thereby obtaining a final GAN; the generative model of the final GAN ​​generates multiple generated images based on the third image set, and the discriminative model of the final GAN ​​judges the authenticity of the multiple generated images; the multiple generated images judged as genuine form a fourth image set; and a fifth image set is formed based on the third and fourth image sets. The remaining steps of the image augmentation method 100 will not be repeated here.

[0049] In this invention, for ease of understanding, the relevant solutions are sometimes presented in multiple steps. However, the order of the step numbers does not necessarily imply a sequential relationship. It is also necessary to consider whether there is a sequential dependency between the two steps. If there is no sequential dependency between the two steps, then even if the step numbers are in sequence, it does not mean that there is a sequential relationship between the two steps.

[0050] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0051] In this document, the directional terms such as front, back, top, and bottom are defined based on the location of the components in the accompanying drawings and their relative positions to each other, solely for the purpose of clarity and convenience in expressing the technical solution. It should be understood that the use of these directional terms should not limit the scope of protection claimed in this application.

[0052] Where there is no conflict, the above embodiments and features described herein can be combined with each other.

[0053] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image augmentation method, characterized in that, It includes: A first image set and a generative adversarial network are provided, wherein the first image set includes multiple original images, and the generative adversarial network includes a generative model and a discriminative model; The original image is processed to obtain a processed image. Multiple processed images constitute a second image set. The image processing includes one or more of the following: graphic transformation, light effect processing, color and detail processing, and special processing. A third image set is formed based on the first image set and the second image set. The generative adversarial network is trained using images from the third image set to obtain the final generative adversarial network; The final generative adversarial network (GAN) model generates multiple generated images based on the third image set. The final discriminative model of the GAN then determines the authenticity of these generated images. The generated images judged as authentic form the fourth image set. The fifth image set is formed based on the third and fourth image sets. The generative adversarial network is trained using images from a third image set to maximize the judgment accuracy of the discriminative model and to maximize the ability of the generative model to deceive the discriminative model. The generative model includes an encoder, a noise-adding module, and a decoder. The encoder includes convolutional layers, pooling layers, multiple dense blocks, and multiple transition layers connected to the outputs of the corresponding dense blocks. The decoder includes multiple dense blocks and multiple deconvolutional transition layers connected to the outputs of the corresponding dense blocks. The noise-adding module is connected between the output of the encoder and the input of the decoder. The decoder outputs a generated image of the same size as the input image. The discriminative model is a binary classification model, which includes pooling layers, convolutional layers, multiple dense blocks, transition layers connecting the dense blocks, a global pooling layer, and a fully connected layer. When an input image is fed into the generative adversarial network (GAN), the encoder of the generative model first extracts features from the input image. Random noise is then added by the noise addition module. Next, the decoder of the generative model outputs a generated image of the same size as the input image. This generated image is then input into the discriminative model, which extracts features from both the generated and input images. Based on the features extracted from the generated image and the features extracted from the input image, the discriminative model outputs a probability value for the generated image's true / false category, ultimately determining whether the generated image is true or false. The encoder includes one convolutional layer, one pooling layer, four dense blocks, and four transition layers. The one convolutional layer, one pooling layer, and four dense blocks of the encoder are connected in sequence, and the four transition layers are respectively connected to the output end of the corresponding dense block. Each dense block is connected to each transition layer located behind it. The decoder includes four dense blocks and four deconvolutional transition layers connected to the outputs of the corresponding dense blocks. Each dense block is connected to each deconvolutional transition layer located behind it. The discrimination model includes a pooling layer, a convolutional layer, four dense blocks, three transition layers connected between the four dense blocks, a global pooling layer, and a fully connected layer, with each dense block connected to each transition layer following it.

2. The image augmentation method as described in claim 1, characterized in that, The images in the fifth image set are labeled, and the labeled fifth image set is used to train the convolutional neural network model to obtain the trained convolutional neural network model.

3. The image augmentation method as described in claim 1, characterized in that, The graphic transformations include one or more of the following: flipping, rotation, distortion, and scaling; The light effect processing includes one or more of the following: brightness processing, contrast processing, highlight and shadow processing, and lighting gradient processing. The color and detail processing includes one or more of the following: saturation processing, hue processing, color temperature processing, noise reduction, and sharpness processing. The special processing includes one or more of the following: occlusion processing and deformation processing. The third image set includes part or all of the first image set and part or all of the second image set, and the fifth image set includes part or all of the third image set and part or all of the fourth image set.

4. The image augmentation method as described in claim 1, characterized in that, The generative model and the discriminative model are trained alternately using images from the third image set. The loss function is minimized using the gradient descent algorithm. The parameters of the generative model and the discriminative model are adjusted in reverse. After iterative training a specified number of times, a smaller loss function is obtained, and the final generative adversarial network is generated.

5. An image augmentation device, characterized in that, It includes: The pre-expansion module performs image processing on multiple original images in the first image set to obtain processed images. The multiple processed images constitute the second image set, and a third image set is formed based on the first and second image sets. The image processing includes one or more of the following: graphic transformation, light effect processing, color and detail processing, and special processing. A Generative Adversarial Network (GAN) comprises a generative model and a discriminative model. The GAN is trained using images from a third image set to obtain the final GAN. The generative model of the final GAN ​​generates multiple generated images based on the third image set. The discriminative model of the final GAN ​​determines the authenticity of these multiple generated images. The multiple generated images judged as authentic form a fourth image set. A fifth image set is formed based on the third and fourth image sets. The generative adversarial network is trained using images from a third image set to maximize the judgment accuracy of the discriminative model and to maximize the ability of the generative model to deceive the discriminative model. The generative model includes an encoder, a noise-adding module, and a decoder. The encoder includes convolutional layers, pooling layers, multiple dense blocks, and multiple transition layers connected to the outputs of the corresponding dense blocks. The decoder includes multiple dense blocks and multiple deconvolutional transition layers connected to the outputs of the corresponding dense blocks. The noise-adding module is connected between the output of the encoder and the input of the decoder. The decoder outputs a generated image of the same size as the input image. The discriminative model is a binary classification model, which includes pooling layers, convolutional layers, multiple dense blocks, transition layers connecting the dense blocks, a global pooling layer, and a fully connected layer. When an input image is fed into the generative adversarial network (GAN), the encoder of the generative model first extracts features from the input image. Random noise is then added by the noise addition module. Next, the decoder of the generative model outputs a generated image of the same size as the input image. This generated image is then input into the discriminative model, which extracts features from both the generated and input images. Based on the features extracted from the generated image and the features extracted from the input image, the discriminative model outputs a probability value for the generated image's true / false category, ultimately determining whether the generated image is true or false. The encoder includes one convolutional layer, one pooling layer, four dense blocks, and four transition layers. The one convolutional layer, one pooling layer, and four dense blocks of the encoder are connected in sequence, and the four transition layers are respectively connected to the output end of the corresponding dense block. Each dense block is connected to each transition layer located behind it. The decoder includes four dense blocks and four deconvolutional transition layers connected to the outputs of the corresponding dense blocks. Each dense block is connected to each deconvolutional transition layer located behind it. The discrimination model includes a pooling layer, a convolutional layer, four dense blocks, three transition layers connected between the four dense blocks, a global pooling layer, and a fully connected layer, with each dense block connected to each transition layer following it.

6. The image augmentation apparatus as described in claim 5, characterized in that, The images in the fifth image set are labeled, and the labeled fifth image set is used to train a convolutional neural network model, resulting in the trained convolutional neural network model. The graphic transformations include one or more of the following: flipping, rotation, distortion, and scaling; The light effect processing includes one or more of the following: brightness processing, contrast processing, highlight and shadow processing, and lighting gradient processing. The color and detail processing includes one or more of the following: saturation processing, hue processing, color temperature processing, noise reduction, and sharpness processing. The special processing includes one or more of the following: occlusion processing and deformation processing. The third image set includes part or all of the first image set and part or all of the second image set, and the fifth image set includes part or all of the third image set and part or all of the fourth image set.

7. The image augmentation apparatus as described in claim 5, characterized in that, The generative model and the discriminative model are trained alternately using images from the third image set. The loss function is minimized using the gradient descent algorithm. The parameters of the generative model and the discriminative model are adjusted in reverse. After iterative training a specified number of times, a smaller loss function is obtained, and the final generative adversarial network is generated.

8. A storage medium, characterized in that, It stores program instructions that are executed to perform the image augmentation method as described in any one of claims 1-4.

9. A computer, characterized in that, It includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to perform the image augmentation method as described in any one of claims 1-4.

Images