A small-scale dataset augmentation method based on learnable data augmentation

By performing alternating training of data augmentation and generative adversarial networks on small-scale data sets, the augmentation probability is adjusted using the learnable parameter p, the problem of overfitting the generative adversarial network is solved, and high-quality image generation and data set amplification are achieved.

CN115482433BActive Publication Date: 2025-08-12SHANDONG UNIV +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211121663.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-15
Publication Date
2025-08-12
Estimated Expiration
2042-09-15

AI Technical Summary

Technical Problem

Generative adversarial networks are prone to overfitting when trained on small-scale datasets, resulting in training crashes and failing to generate high-quality images.

Method used

Using a method based on learnable data augmentation, data augmentation is performed on small-scale data sets, and by alternating training of generating adversarial networks, the probability of data augmentation is adjusted using the learnable parameter p, the objective function is optimized, and high-quality images are generated.

Benefits of technology

A large number of high-quality images were successfully generated, amplification of small-scale data sets was achieved, and the situation where the generated images were inconsistent with the original data was avoided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115482433B_ABST
    Figure CN115482433B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for amplifying a small-scale data set based on learnable data enhancement, comprising the following steps: performing data enhancement on the original image in the small-scale data set with a probability p; inputting the data-enhanced image into the discriminator of a generative adversarial network, and at the same time, randomly sampling a normally distributed noise as the input of the generator in the generative adversarial network, the generator generates an image and inputs it into the discriminator, the generator and the discriminator are alternately trained to optimize the objective function; during the training process, the probability p and the parameters of the generative adversarial network are continuously learned and updated; randomly sampling multiple normally distributed noises and inputting them into the generator of the trained generative adversarial network respectively, generating corresponding images respectively, thereby achieving amplification of the small-scale data set. The method disclosed by the present invention can generate a large number of high-quality images, achieve amplification of the small-scale data set, and avoid the situation where the generated images are inconsistent with the original data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a small-scale data set amplification method based on learnable data enhancement. Background Art

[0002] The most typical approach to image augmentation is the Generative Adversarial Network (GAN), which consists of two modules: a generator and a discriminator. The goal of the generator network is to generate realistic images to deceive the discriminator, making it unable to distinguish between real and synthetic images. The discriminator's goal, on the other hand, is to distinguish between the generator's images and real ones, thereby enhancing its ability to distinguish them. By continuously training the generator and discriminator, the model can eventually produce highly realistic images.

[0003] However, generative adversarial networks require a large amount of training data. For small-scale data sets, the network can easily overfit, leading to training collapse.

[0004] In order to solve the problem of overfitting, this paper designs a series of data augmentation methods to increase the diversity of images. At the same time, in order to prevent the generated images from being affected by the augmented data, a learnable parameter p is used as the probability of using various augmentation methods to achieve the purpose of enabling the network to identify the correct source image. Summary of the Invention

[0005] To solve the above technical problems, the present invention provides a small-scale dataset amplification method based on learnable data enhancement, which can generate a large number of high-quality images, realize the amplification of small-scale datasets, and avoid the situation where the generated images are inconsistent with the original data.

[0006] To achieve the above object, the technical solution of the present invention is as follows:

[0007] A small-scale dataset augmentation method based on learnable data augmentation includes the following steps:

[0008] Step 1: Data enhancement: Perform data enhancement on the original image in the small-scale dataset with probability p to obtain the data-enhanced image;

[0009] Step 2: Generative Adversarial Network Training: The data-augmented image is input into the discriminator of the generative adversarial network. At the same time, a normally distributed noise is randomly sampled as the input of the generator in the generative adversarial network. The generator generates an image and inputs it into the discriminator. The discriminator determines whether the input image is a real image or a generated image. The generator and discriminator are trained alternately to optimize the objective function. During the training process, the probability p and the parameters of the generative adversarial network are continuously learned and updated. Repeat steps 1 and 2 to complete the training of the generative adversarial network.

[0010] Step 3: Small-scale dataset expansion: Randomly sample multiple normally distributed noises and input them into the generator of the trained generative adversarial network to generate corresponding images, thereby expanding the small-scale dataset.

[0011] In the above scheme, in step 1, the data enhancement method includes geometric transformation, pixel transformation and image filtering.

[0012] In a further technical solution, the geometric transformation includes displacement transformation, geometric scaling and non-geometric scaling.

[0013] In a further technical solution, the pixel transformation includes brightness change, contrast change, saturation change, noise addition and random erasure.

[0014] Preferably, the specific method of step one is as follows: the original image is subjected to displacement transformation, geometric scaling, non-geometric scaling, brightness change, contrast change, saturation change, noise addition, random erasure and image filtering in sequence with probability p to obtain a data enhanced image.

[0015] In a further technical solution, the displacement transformation is to perform a displacement on the entire image with a probability p, and the transformation process is as follows:

[0016] t x , t y ~U(-0.1, 0.1)

[0017]

[0018] Among them, the probability p is a learnable parameter with an initial value of 0.5; t x , t y Respectively represent the multiples of image width and height scaling; i 11 ~U(0,1) is a random number between 0 and 1, used to determine whether the displacement transformation operation is performed; U() represents uniform distribution, Translate represents displacement transformation operation, w and h represent the width and height of the image; round() represents rounding; X0 represents the original image; X 11 is the image after displacement transformation;

[0019] The geometric scaling is to scale the width and height of the image in the same proportion with probability p. The transformation process is as follows:

[0020] s~U(0.5,2)

[0021]

[0022] Among them, i 12~U(0,1) is a random number between 0 and 1, which is used to determine whether the geometric scaling operation is performed; Scale represents the scaling operation, s is the scaling factor of the image width and height; Crop is the cropping operation, which is used to convert the image width and height back to w and h to keep the image size unchanged, X 12 It is the image after geometric scaling;

[0023] The non-uniform scaling is to scale the width and height of the image in different proportions with probability p. The transformation process is as follows:

[0024] s1~U(0.5, 2), s2~U(0.5, 2)

[0025]

[0026] Among them, i 13 ~U(0,1) is a random number between 0 and 1, used to determine whether the non-proportional scaling operation is performed. s1 and s2 are the scaling multiples of the image width and height respectively. X 13 Represents an image after non-uniform scaling.

[0027] In a further technical solution, the brightness change is to change the brightness of the image with probability p, and the change process is as follows:

[0028] b~U(0.5,1.5)

[0029]

[0030] Among them, i 21 ~U(0,1) is a random number from 0 to 1, which is used to determine whether the brightness change operation is performed. U() represents uniform distribution, Bright represents the brightness change operation, b is the multiple of the brightness change, and X 13 Represents the image after non-uniform scaling, X 21 Represents the image after brightness change;

[0031] The contrast change is to change the contrast of the image with probability p, and the change process is as follows:

[0032] c~U(0.7,1.2)

[0033]

[0034] Among them, i 22 ~U(0,1) is a random number from 0 to 1, which is used to determine whether the contrast change operation is performed. Contrast represents the contrast change operation, c is the multiple of the contrast change, and X 22 Represents the image after contrast change;

[0035] The saturation change is to change the saturation of the image with probability p, and the change process is as follows:

[0036] s~U(0.6,1.2)

[0037]

[0038] Among them, i 23 ~U(0,1) is a random number between 0 and 1, which is used to determine whether the saturation change operation is performed. Saturation represents the saturation change operation, s is the multiple of the saturation change, and X 23 Represents the image after saturation change;

[0039] The noise addition adds a random noise to the image with probability p, and the change process is as follows:

[0040] r, g, b~N(0, 1)

[0041]

[0042] Among them, i 24 ~U(0,1) is a random number between 0 and 1, used to determine whether to add noise, (m,n) represents a pixel coordinate on the image, and 0≤m<w, 0≤n<h, w and h represent the width and height of the image, r, g, b are three normally distributed random numbers, corresponding to the three components R, G, B of the pixel respectively; N() represents normal distribution; X 24 Represents the image after the noise addition operation;

[0043] The random erasure is to randomly select an area in the image with probability p and remove it. The change process is as follows:

[0044] c x , c y ~U(0.3, 0.6)

[0045] left=round((c x -0.25)×w)

[0046] low=round((c y -0.25)×h)

[0047] right=round((c x +0.25)×w)

[0048] high=round((c y +0.25)×h)

[0049]

[0050]

[0051] Among them, i 25 ~U(0,1) is a random number between 0 and 1, used to determine whether to perform random erasure, (left, high) represents the coordinates of the upper left vertex of the erasure area, (right, low) represents the coordinates of the lower right vertex of the erasure area, and round() represents rounding; (c x , c y ) represents the coordinates of the center point of the erased area, mask is the mask of the erased area, ⊙ represents the exclusive OR operation, X 25 Represents the image after random erasure is performed.

[0052] In a further technical solution, the image filtering is to perform a filtering operation on the image with a probability p, using filters of four different sizes, and the process is as follows:

[0053] size∈{(3, 3), (5, 5), (7, 7), (9, 9)}

[0054]

[0055] Among them, i 31 ~U(0,1) is a random number from 0 to 1, which is used to determine whether to perform image filtering operation. Filter represents the image filtering operation, size represents the size of the four filters, and X 25 represents the image after random erasure, X 31 is the image after filtering enhancement, which randomly selects one from four different filters.

[0056] Preferably, in step 2, the generative adversarial network adopts StyleGAN.

[0057] In a further technical solution, in step 2, when the generator is trained, the parameters of the discriminator are fixed, and the objective function is minimized:

[0058]

[0059] When the discriminator is trained, the parameters of the generator are fixed, and the objective function is maximized:

[0060]

[0061] Among them, V(D, G) is a training function containing probability p and generative adversarial network parameters, x is a real image after data augmentation, z is randomly sampled noise, D and G represent the discriminator and generator respectively, D() and G() represent the outputs of the discriminator and generator respectively, and E represents the expected value.

[0062] Through the above technical solution, the small-scale dataset amplification method based on learnable data enhancement provided by the present invention has the following beneficial effects:

[0063] (1) The present invention solves the overfitting problem when training networks using small-scale datasets by performing a series of data enhancements on the images in the small-scale datasets, successfully generates a large number of high-quality images, and achieves the purpose of dataset amplification.

[0064] (2) The present invention uses a learnable parameter p as the probability of data enhancement, so that the network can learn the distribution of the original data (without data enhancement) during the training process, avoiding the situation where the generated image is inconsistent with the original data.

[0065] (3) The present invention realizes the training of the generative adversarial network by maximizing or minimizing the objective function at different stages of training. Then, the trained generative adversarial network is used to generate a large amount of data with high authenticity from small-scale data, thereby realizing the expansion of small-scale data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0067] Figure 1 This is a flow chart of a small-scale dataset amplification method based on learnable data enhancement disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0069] The present invention provides a small-scale dataset augmentation method based on learnable data enhancement, such as Figure 1 As shown, the following steps are included:

[0070] Step 1: Data enhancement: Perform data enhancement on the original image in the small-scale dataset with probability p to obtain the data-enhanced image.

[0071] Data augmentation methods include geometric transformation, pixel transformation, and image filtering. Geometric transformation includes displacement, scaling, and non-scaling. Pixel transformation includes brightness, contrast, saturation, noise addition, and random erasure. The order of these methods is not fixed and can be adjusted.

[0072] In this example, a small-scale flower dataset was used as training images. It contains 17 flower categories, with 80 images per category. The flowers in the dataset exhibit significant variations in posture and lighting. Data augmentation was performed in the following order: the original image was subjected to a displacement transformation with probability p, geometric scaling, non-geometric scaling, brightness change, contrast change, saturation change, noise addition, random erasure, and image filtering to obtain the enhanced image.

[0073] The specific data enhancement process is as follows:

[0074] 1. Displacement transformation

[0075] The displacement transformation is to displace the entire image with probability p. The transformation process is as follows:

[0076] t x , t y ~U(-0.1, 0.1)

[0077]

[0078] Among them, the probability p is a learnable parameter with an initial value of 0.5; t x , t y Respectively represent the multiples of image width and height scaling; i 11 ~U(0,1) is a random number between 0 and 1, used to determine whether the displacement transformation operation is performed; U() represents uniform distribution, Translate represents displacement transformation operation, w and h represent the width and height of the image; round() represents rounding; X0 represents the original image; X 11 is the image after displacement transformation.

[0079] 2. Proportional scaling

[0080] Geometric scaling is to scale the width and height of the image in the same proportion with probability p. The transformation process is as follows:

[0081] s~U(0.5,2)

[0082]

[0083] Among them, i 12 ~U(0,1) is a random number between 0 and 1, which is used to determine whether the geometric scaling operation is performed; Scale represents the scaling operation, s is the scaling factor of the image width and height; Crop is the cropping operation, which is used to convert the image width and height back to w and h to keep the image size unchanged, X 12 It is the image after geometric scaling.

[0084] 3. Non-proportional scaling

[0085] Non-proportional scaling is to scale the width and height of the image in different proportions with probability p. The transformation process is as follows:

[0086] s1~U(0.5, 2), s2~U(0.5, 2)

[0087]

[0088] Among them, i 13 ~U(0,1) is a random number between 0 and 1, used to determine whether the non-proportional scaling operation is performed. s1 and s2 are the scaling multiples of the image width and height respectively. X 13 Represents an image after non-uniform scaling.

[0089] 4. Brightness changes

[0090] The brightness change is to change the brightness of the image with probability p, and the change process is as follows:

[0091] b~U(0.5,1.5)

[0092]

[0093] Among them, i 21 ~U(0,1) is a random number from 0 to 1, which is used to determine whether the brightness change operation is performed. U() represents uniform distribution, Bright represents the brightness change operation, b is the multiple of the brightness change, and X 13 Represents the image after non-uniform scaling, X 21 Represents the image after brightness change.

[0094] 5. Contrast changes

[0095] Contrast change is to change the contrast of the image with probability p, and the change process is as follows:

[0096] c~U(0.7,1.2)

[0097]

[0098] Among them, i 22 ~U(0,1) is a random number from 0 to 1, which is used to determine whether the contrast change operation is performed. Contrast represents the contrast change operation, c is the multiple of the contrast change, and X 22 Represents the image after contrast change.

[0099] 6. Saturation changes

[0100] Saturation change is to change the saturation of the image with probability p, and the change process is as follows:

[0101] s~U(0.6,1.2)

[0102]

[0103] Among them, i 23 ~U(0,1) is a random number between 0 and 1, which is used to determine whether the saturation change operation is performed. Saturation represents the saturation change operation, s is the multiple of the saturation change, and X 23 Represents the image after saturation change.

[0104] 7. Noise Addition

[0105] Noise addition is to add random noise to the image with probability p. The change process is as follows:

[0106] r, g, b~N(0, 1)

[0107]

[0108] Among them, i 24 ~U(0,1) is a random number between 0 and 1, used to determine whether to add noise, (m,n) represents a pixel coordinate on the image, and 0≤m<w, 0≤n<h, w and h represent the width and height of the image, r, g, b are three normally distributed random numbers, corresponding to the three components R, G, B of the pixel respectively; N() represents normal distribution; X 24 Represents the image after the noise addition operation.

[0109] 8. Random Erase

[0110] Random erasing is to randomly select an area in the image with probability p and remove it. The change process is as follows:

[0111] c x , c y ~U(0.3, 0.6)

[0112] left=round((c x -0.25)×w)

[0113] low=round((c y -0.25)×h)

[0114] right=round((c x +0.25)×w)

[0115] high=round((c y +0.25)×h)

[0116]

[0117]

[0118] Among them, i 25 ~U(0,1) is a random number between 0 and 1, used to determine whether to perform random erasure, (left, high) represents the coordinates of the upper left vertex of the erasure area, (right, low) represents the coordinates of the lower right vertex of the erasure area, and round() represents rounding; (c x , c y ) represents the coordinates of the center point of the erased area, mask is the mask of the erased area, ⊙ represents the exclusive OR operation, X 25 Represents the image after random erasure is performed.

[0119] 9. Image filtering

[0120] Image filtering is to filter the image with probability p, using four filters of different sizes. The process is as follows:

[0121] size∈{(3, 3), (5, 5), (7, 7), (9, 9)}

[0122]

[0123] Among them, i 31 ~U(0,1) is a random number from 0 to 1, which is used to determine whether to perform image filtering operation. Filter represents the image filtering operation, size represents the size of the four filters, and X 25 represents the image after random erasure, X 31 is the image after filtering enhancement. This operation randomly selects one of four different filters.

[0124] Step 2: Generative Adversarial Network Training: The data-augmented image is input into the discriminator of the generative adversarial network. At the same time, a normally distributed noise is randomly sampled as the input of the generator in the generative adversarial network. The generator generates an image and inputs it into the discriminator. The discriminator determines whether the input image is a real image or a generated image. The generator and discriminator are trained alternately to optimize the objective function. During the training process, the probability p and the parameters of the generative adversarial network are continuously learned and updated. Repeat steps 1 and 2 to complete the training of the generative adversarial network.

[0125] In this embodiment, the generative adversarial network uses StyleGAN, which uses a progressively growing network as the generator and discriminator. That is, the shallow layers of the network generate images with lower resolution. As the network becomes deeper, the resolution of the generated images gradually increases, and finally a higher-resolution synthetic image is obtained. The specific process is as follows:

[0126] (1) The data-enhanced image X 31Put it into the discriminator D, and use the discriminator to determine whether the class of the image is a real image or a synthetic image, that is, class = D(X 31 );

[0127] (2) Randomly sample a noise z as the input of the generator G, put it into the generator, and generate an image Right now

[0128] (3) Input the generated image into the discriminator to determine whether the image is a real image or a synthetic image, that is,

[0129] Repeat steps (1)-(3) continuously, so that the generator can generate images that are as similar to real images as possible, and the discriminator can distinguish whether the input image is synthetic or real as much as possible, and finally the discriminator cannot distinguish the source of the image.

[0130] The overall training objectives are:

[0131]

[0132] When the generator is trained, the parameters of the discriminator are fixed, and the objective function is minimized:

[0133]

[0134] When the discriminator is trained, the parameters of the generator are fixed, and the objective function is maximized:

[0135]

[0136] Among them, V(D, G) is a training function containing probability p and generative adversarial network parameters, x is a real image after data augmentation, z is randomly sampled noise, D and G represent the discriminator and generator respectively, D() and G() represent the outputs of the discriminator and generator respectively, and E represents the expected value.

[0137] The learnable probability p is updated during the discriminator training phase. During initial training, due to the small amount of training data, the discriminator is severely overfitted. At this stage, the discriminator's discriminative ability is strong, and no matter how the generator is optimized, it can still distinguish real and fake images. This makes it difficult to train the subsequent generator effectively. Therefore, p is continuously increased to enhance the diversity of the training data. In the later stages of model training, p becomes too large, causing the images generated by the generator to resemble the augmented images more closely than the original images. At this point, p begins to slowly decrease. After model training is complete, the value of p is between 0.8 and 0.9.

[0138] Step 3: Small-scale dataset expansion: Randomly sample multiple normally distributed noises and input them into the generator of the trained generative adversarial network to generate corresponding images, thereby expanding the small-scale dataset.

[0139] Specifically:

[0140] (1) Load the generator G trained in the previous step;

[0141] (2) Randomly sample a noise z from a normal distribution and input it into the generator G to generate the corresponding image Image.

[0142] Image=G(z)

[0143] By randomly sampling multiple noises, corresponding multiple images can be generated respectively to achieve image amplification.

[0144] In this embodiment, after the above process, 10,000 images were finally generated using this method, thereby achieving the expansion of the data set.

[0145] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A small-scale dataset augmentation method based on learnable data augmentation, characterized in that: The following steps are involved: Step 1: Data enhancement: Perform data enhancement on the original image in the small-scale dataset with probability p to obtain the data-enhanced image; Step 2: Generative Adversarial Network Training: The data-augmented image is input into the discriminator of the generative adversarial network. At the same time, a normally distributed noise is randomly sampled as the input of the generator in the generative adversarial network. The generator generates an image and inputs it into the discriminator. The discriminator determines whether the input image is a real image or a generated image. The generator and discriminator are trained alternately to optimize the objective function. During the training process, the probability p and the parameters of the generative adversarial network are continuously learned and updated. Repeat steps 1 and 2 to complete the training of the generative adversarial network. Step 3: Small-scale dataset expansion: Randomly sample multiple normally distributed noises and input them into the generator of the trained generative adversarial network to generate corresponding images, thereby expanding the small-scale dataset. In step 2, when the generator is trained, the parameters of the discriminator are fixed, and the objective function is minimized: ; When the discriminator is trained, the parameters of the generator are fixed, and the objective function is maximized: ; in, is a training function containing probability p and the parameters of the generative adversarial network, is a real image after data augmentation, is the randomly sampled noise, and Represent the discriminator and generator respectively, and Represent the output of the discriminator and the generator respectively, Indicates taking the expected value.

2. A small-scale dataset augmentation method based on learnable data enhancement according to claim 1, characterized in that: In step 1, data enhancement methods include geometric transformation, pixel transformation and image filtering.

3. The method for expanding a small-scale dataset based on learnable data enhancement according to claim 2, characterized in that: The geometric transformation includes displacement transformation, proportional scaling and non-proportional scaling.

4. The method for expanding a small-scale dataset based on learnable data enhancement according to claim 2, characterized in that: The pixel transformation includes brightness change, contrast change, saturation change, noise addition and random erasure.

5. The method for expanding a small-scale dataset based on learnable data enhancement according to claim 1, characterized in that: The specific method of step one is as follows: the original image is subjected to displacement transformation, geometric scaling, non-geometric scaling, brightness change, contrast change, saturation change, noise addition, random erasure and image filtering in sequence with probability p to obtain a data enhanced image.

6. The method for expanding a small-scale dataset based on learnable data enhancement according to claim 5, characterized in that: The displacement transformation is to perform an overall displacement on the image with probability p. The transformation process is as follows: ; ; Among them, the probability p is a learnable parameter with an initial value of 0.5; Respectively represent the multiples of image width and height scaling; , is a random number between 0 and 1, used to determine whether the shift transformation operation is performed; Indicates uniform distribution, Translate indicates displacement transformation operation, Indicates the width and height of the image; Indicates rounding; represents the original image; is the image after displacement transformation; The geometric scaling is to scale the width and height of the image in the same proportion with probability p. The transformation process is as follows: ; ; in, , is a random number from 0 to 1, used to determine whether the geometric scaling operation is performed; Scale represents the scaling operation, s is the scaling multiple of the image width and height; Crop is the cropping operation, which is used to convert the image width and height back to w and h to keep the image size unchanged. It is the image after geometric scaling; The non-uniform scaling is to scale the width and height of the image in different proportions with probability p. The transformation process is as follows: ; ; in, , is a random number between 0 and 1, used to determine whether the non-proportional scaling operation is performed. are the scaling factors of image width and height respectively, Represents an image after non-uniform scaling.

7. The method for expanding a small-scale dataset based on learnable data enhancement according to claim 5, characterized in that: The brightness change is to change the brightness of the image with probability p, and the change process is as follows: ; ; in, , is a random number between 0 and 1, used to determine whether the brightness change operation is performed. Indicates uniform distribution, Bright indicates brightness change operation, b is the multiple of brightness change, represents the image after non-uniform scaling, Represents the image after brightness change; The contrast change is to change the contrast of the image with probability p, and the change process is as follows: ; ; in, , is a random number from 0 to 1, used to determine whether the contrast change operation is performed, Contrast represents the contrast change operation, c is the multiple of the contrast change, Represents the image after contrast change; The saturation change is to change the saturation of the image with probability p, and the change process is as follows: ; ; in, , is a random number between 0 and 1, used to determine whether the saturation change operation is performed. Saturation represents the saturation change operation. is the multiple of saturation change, Represents the image after saturation change; The noise addition adds a random noise to the image with probability p, and the change process is as follows: ; ; in, , is a random number between 0 and 1, used to determine whether to add noise, represents a pixel coordinate on the image, and , Indicates the width and height of the image, are three normally distributed random numbers corresponding to the three components of the pixel ; represents a normal distribution; Represents the image after the noise addition operation; The random erasure is to randomly select an area in the image with probability p and remove it. The change process is as follows: ; ; ; ; ; ; ; in, , is a random number between 0 and 1, used to determine whether to perform random erasure, ( Indicates the coordinates of the upper left vertex of the erased area. Indicates the coordinates of the lower right vertex of the erased area. Indicates rounding; Indicates the coordinates of the center point of the erased area, mask is the mask of the erased area, Indicates the exclusive OR operation, Represents the image after random erasure is performed.

8. A small-scale dataset augmentation method based on learnable data enhancement according to claim 1 or 5, characterized in that: The image filtering is to filter the image with probability p, using four filters of different sizes. The process is as follows: ; ; in, , is a random number from 0 to 1, used to determine whether to perform image filtering operations, Filter represents the image filtering operation, size represents the size of the four filters, represents the image after random erasure is performed, is the image after filtering enhancement, which randomly selects one from four different filters.

9. The method for expanding a small-scale dataset based on learnable data enhancement according to claim 1, characterized in that: In step 2, the generative adversarial network adopts StyleGAN.

Citation Information

Patent Citations

  • Image data expansion method for deep learning model training and learning

    CN109767440A

  • Hand depth image data enhancement method based on generative adversarial network

    CN110427799A