A small data set sample-oriented gan discriminator capability suppression algorithm

Abstract

The application belongs to the field of data enhancement application, and particularly relates to a GAN discriminator capability suppression algorithm for small data set samples. The application adopts a new type of discriminator suppression module, which performs scaling processing on the judgment result before the discriminator performs loss calculation. When the discriminator judges the real data, a judgment value and an average value are randomly selected for judgment. If the judgment value is greater than the average value in the same batch, the judgment value is subtracted by the average value multiplied by lambda, so as to achieve the purpose of reducing the judgment value. When the discriminator judges the false data generated by the generator, a judgment value is still randomly selected for judgment. If the judgment value is less than the average value in the same batch, the judgment value is added by the average value multiplied by lambda. In this way, the balance between the generator and the discriminator can be maintained, so as to avoid the discriminator reaching the optimal state, and solve the training termination problem caused by the JS divergence being 0.

Description

Technical Field

[0001] This invention belongs to the field of data augmentation applications, specifically involving a GAN discriminator capability suppression algorithm for small datasets. Technical Background

[0002] The superior performance of GANs often relies on sufficient datasets. However, in scenarios with small datasets, the performance of GANs or other deep learning models suffers from some degree of quality degradation, especially in GANs, where the discriminator exacerbates the problem. Data augmentation can effectively alleviate the difficulties caused by insufficient data. However, data augmentation is also likely to introduce another problem: sample percolation. Therefore, how to mitigate the discriminator overfitting problem without over-reliance on data augmentation remains a challenge.

[0003] The discriminator overfits because when the real dataset is small, it easily memorizes the distribution of all real data, rapidly improving its discrimination ability and widening the gap between the discriminator and the generator—a phenomenon commonly known as discriminator overfitting and generator underfitting. However, GAN training is an adversarial game; ideally, both sides should reach a Nash equilibrium. When one side is too strong, the other is at a severe disadvantage, leading to training failure. When the discriminator overfits, the generator cannot obtain adversarially meaningful loss values ​​from the discriminator, resulting in the vanishing gradient phenomenon and hindering generator convergence. Overly complex discriminator networks exacerbate the overfitting problem.

[0004] Researchers have made significant efforts and proposed numerous effective methods to prevent generator overfitting, stabilize GAN training, and prevent model collapse, such as gradient penalty, noise injection, and one-sided label smoothing. These methods can stabilize GAN training or prevent model collapse from different perspectives. For example, gradient penalty can prevent discriminator overfitting by penalizing excessively large gradient values, thereby improving the robustness of GANs. Noise injection can increase data diversity, preventing the discriminator from prematurely memorizing all real data distributions, thus reducing the risk of discriminator overfitting. Furthermore, one-sided label smoothing can also improve model robustness while reducing model overfitting. When a model collapses, the generated images have high style similarity, making them unsuitable for practical applications, which illustrates the importance of reducing discriminator overfitting for GAN development. Although many solutions exist, effectively reducing discriminator overfitting remains a crucial problem to be solved in the development of GANs, deserving continued research and effort. Summary of the Invention

[0005] To address the problems in the background art, this invention provides a discriminator capability suppression algorithm for small datasets.

[0006] The technical solution of the present invention is as follows:

[0007] An algorithm for suppressing the discriminant power of GANs for small datasets is proposed, with the following steps:

[0008] Step 1: Input a random vector z that follows a Gaussian distribution into the generator G. The generator G generates a fake image G(z) based on the input random vector.

[0009] Step 2: Input the fake image G(z) generated by generator G into discriminator D, and at the same time input the real image data Xreal into discriminator D.

[0010] The discriminator D has the following structure:

[0011] First, features are extracted using a 3×3 convolutional layer. The output is then fed into the first ResNet block of layer 1, followed by the second ResNet block. Next, the result is passed to a dropout layer for random parameter discarding. Then, dimensionality reduction is performed using a 3×3 average pooling layer in layer 2. The result is then fed into the first ResNet block of layer 2, followed by the second ResNet block. This process is repeated four times, but the last two times do not require a dropout layer. Finally, a view layer flattens the multidimensional vector into a one-dimensional vector. A Leaky ReLU activation function is then used to introduce a non-linear transformation. Finally, a fully connected layer integrates the features into a single value and outputs it.

[0012] Step 3: Discriminator D distinguishes between the input fake image G(z) and the real image Xreal. Discriminator D will determine the probability that a certain image is real or fake.

[0013] Step 4: The discriminator D labels the identification result of real images as d_real and the identification result of fake images as d_fake, and inputs d_real and d_fake into the discriminator suppression module.

[0014] The discriminator suppression module has the following structure:

[0015] The reason for GAN overfitting is that the training dataset is too small, allowing the discriminator to quickly grasp all the features of the real dataset. This means the discriminator can always accurately distinguish between images generated by the generator and real images. When this happens, the loss value returned by the discriminator to the generator hardly causes any numerical perturbation, leading to the gradient vanishing problem, which prevents GANs from generating realistic fake images.

[0016] To address the aforementioned problem, a discriminator capability suppression module is proposed. This module devises a suppression algorithm to ensure a capability balance between the generator G and the discriminator D. The derivation process is as follows:

[0017] As can be seen from the objective function of GAN, when the generator is fixed, the objective function for training the discriminator is as follows:

[0018] maxV(D,G)=∫ x p data (x)logD(x)+p g log(1-D(x))dx

[0019] The formula obtains the maximum value by adjusting D(x), so by taking the partial derivative of D(x) and setting the partial derivative function to 0, we can obtain the value of D(x) as shown in the following formula, which is also the optimal solution of the discriminator D(x).

[0020]

[0021] From the above formula, it can be seen that if the discriminator D is trained to its optimal state, then we can obtain:

[0022]

[0023] Rearranging the terms, we obtain the objective function of the discriminator as:

[0024] V(D,G)=2JS(p data ||p g )-log4

[0025] During GAN training, when the discriminator reaches its optimal state, JS(p) is affected. data ||p g When V(D,G) in the discriminator's objective function equals 0, the network cannot perform effective backpropagation, leading to training termination. Since the discriminator's objective function can only be simplified to solving the JS divergence problem when the discriminator is close to its optimal state, this invention aims to constrain the discriminator's discriminative ability to prevent it from reaching its optimal state. Specifically, the discriminator scales the judgment result before calculating the loss, as shown below:

[0026] Df=S(D ini )±λD avg (λ>0)

[0027] Where D ini This represents the initial discrimination result of the discriminator in a batch, while D... avg S(D) represents the average of the discrimination results in this batch. ini The expression ) indicates that random sampling is performed on the discrimination values ​​of this batch, and Df represents the final discrimination result output by the discriminator. When the discriminator judges real data, it randomly selects a discrimination value and compares it with the average value. If the discrimination value is greater than the average value in the same batch, it subtracts λ times the average value to reduce the discrimination value. When the discriminator judges fake data generated by the generator, it still randomly selects a discrimination value. If the discrimination value is less than the average value in the same batch, it adds λ times the average value to it.

[0028] Step 5: The discriminator suppression module performs constraint processing based on the input d_real and d_fake. Specifically, it randomly samples the input data and uses the average value of the input data as a benchmark to expand or shrink the value.

[0029] Step 6: After the discriminator suppression module has finished processing the data, it calculates the cross-entropy loss function on the constrained values, and calculates the loss value g_loss of generator G and the loss value d_loss of discriminator D respectively.

[0030] Step 7: Input g_loss into the generator G and d_loss into the discriminator D, and update the gradients of both based on their respective loss values.

[0031] Step 8: After gradient update, repeat the above steps until the generated fake image G(z) meets expectations.

[0032] The advantages of this invention include the following:

[0033] (1) Design of the discriminator suppression module: Before the discriminator calculates the loss result, it performs scaling processing. When the discriminator judges real data, it randomly selects a discriminant value and the average value for judgment. If the discriminant value is greater than the average value in the same batch, it subtracts λ times the average value, thereby reducing the discriminant value. When the discriminator judges fake data generated by the generator, it still randomly selects a discriminant value for judgment. If the discriminant value is less than the average value in the same batch, it adds λ times the average value. This maintains the balance between the generator and the discriminator, thereby preventing the discriminator from reaching its optimal state and solving the training termination problem caused by JS divergence being 0.

[0034] (2) Lightweight Discriminator: During the training of deep learning networks, model complexity is often directly proportional to training time and inversely proportional to training efficiency. In other words, the deeper the network layers, the longer the training time and the lower the training efficiency. Especially when using small sample datasets, the small dataset size and excessive network complexity can easily lead to overfitting during training. Therefore, this invention lightweights the discriminator's network structure to reduce network complexity, improve training efficiency, and avoid overfitting. Since the discriminator network extracts and distinguishes features from the input images, and the real dataset has a small amount of data, the discriminator can easily distinguish between real images and generated fake images through multiple feature extractions. Therefore, to reduce the reliance on image feature extraction and discrimination, we reduced one average pooling layer and one residual layer in the network. Since the training uses small 64×64 images, these layers increase network complexity, resulting in longer training time. Furthermore, to further address the overfitting problem, we also introduced a dropout layer. The purpose of the dropout layer is to randomly stop updating certain parameters in the network layer, thereby reducing the overfitting problem in the network. We can set the proportion of parameters that the dropout layer stops updating. Choosing an appropriate proportion according to different experimental requirements can make the network training effect better and achieve twice the result with half the effort. Attached Figure Description

[0035] Figure 1 The overall framework diagram of the proposed algorithm model is shown below;

[0036] Figure 2 This is a schematic diagram of the original discriminator module structure;

[0037] Figure 3 This is a schematic diagram of the lightweight discriminator structure;

[0038] Figure 4 This is a visualization of the training results of CSDGAN on the OxFlowers102 dataset;

[0039] Figure 5 This is a visualization of the training results of CSDGAN and AdaFM methods on the Flowers-25 dataset; Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0041] This invention alleviates the overfitting problem of discriminators in small dataset scenarios. The overall framework of the algorithm model is shown in the figure below. Figure 1As shown, the main process is as follows: 1. Random noise is input into the generator G, which generates a fake image G(z). 2. The generated fake image G(z) and the real image data Xreal are simultaneously input into the discriminator D. 3. The discriminator D distinguishes the input images and outputs the discrimination result d_real for the real image and the discrimination result d_fake for the fake image. 4. d_real and d_fake are input into the discriminator's suppression module for constraint processing, and the cross-entropy loss function is calculated on the constrained values ​​to calculate the loss value g_loss of the generator G and the loss value d_loss of the discriminator D. The generator G and the discriminator D are then updated with gradients based on the corresponding loss values.

[0042] The original discriminator module structure is as follows: Figure 2 As shown, the original discriminator consists of multiple convolutional layers, residual blocks, pooling layers, and fully connected layers. However, during the training of deep learning networks, the complexity of the model is often proportional to the training time. Especially when using small sample datasets, the small dataset size and an overly complex network can easily lead to overfitting during training. The algorithm proposed in this invention lightweights the discriminator by reducing one average pooling layer and one residual layer in the network and adding multiple dropout layers to randomly discard parameter values, thus reducing overfitting. The schematic diagram of the lightweight structure is shown below. Figure 3 As shown.

[0043] Therefore, the main contributions of this invention can be summarized as follows:

[0044] (1) Add a capability suppression module between the discriminator and the generator. This module can enhance the discriminator’s recognition of fake images while blurring its judgment of real images, thereby achieving the purpose of suppressing the discriminator’s capability.

[0045] (2) In order to avoid the overfitting phenomenon caused by the discriminator network being too complex, a lightweight processing method is adopted to reduce the complexity of the discriminator, improve the efficiency of GAN training, and reduce the learning rate of the discriminator.

[0046] Example 1:

[0047] A discriminator suppression algorithm for GANs with small datasets is proposed. The discriminator suppression module mainly constrains the discriminator's output to prevent it from learning too quickly on the real dataset, reducing the probability of overfitting and enhancing the stability of model training. This module has two main functions:

[0048] a. Constraining the discriminator output: Constraining the discriminator output can make the discriminator's output smoother and more stable, reducing the discriminator's overtraining on the real dataset.

[0049] b. Calculate the loss value: Calculate the difference between the constrained discriminator output and the real image, and transmit the corresponding loss value to the generator and discriminator. This loss value is used to update the weights of the generator and discriminator, thereby improving the training effect of the model.

[0050] During training, this invention selected three publicly available datasets as the original data for experiments: OxfordFlowers102, CelebA, and Cathedral. The datasets used in Example 1 are shown in Table 1 as the test set.

[0051] Table 1: Test set information used in Example 1:

[0052] Dataset quantity size Whether to make it public OxfordFlowers102 8000 64*64 public CelebA 8000 64*64 public Cathedral 8000 64*64 public

[0053] Tables 2 and 3 summarize the parameter optimization results of this invention in Example 1. The experimental results in Table 2 show that when the batch size is 16 and the random sampling ratio is 1 / 2, the optimal FID value can be obtained, resulting in the highest generated image quality, although the number of iterations and the time taken are slightly longer. Next, experiments were conducted on the value of λ when the random sampling ratio was 1 / 2, taking values ​​from 1 / 2 to 1 / 8. The experimental results in Table 3 show that when λ is 1 / 6, the optimal FID value can be obtained. Based on the above analysis, in the subsequent experiments in this chapter, we set the discriminator suppression level to a random sampling ratio of 1 / 2 and the λ value to 1 / 6 to achieve the best generated image quality.

[0054] Table 2: Optimal results of the present invention under different random sampling ratios on the OxFlowers102 dataset in Example 1.

[0055]

[0056]

[0057] Table 3: Optimal FID values ​​trained under different λ values ​​in Example 1 of this invention

[0058] λ Optimal FID value Number of iterations Time used / h 1 / 2 21.59 69500 19 1 / 3 20.36 65500 18 1 / 4 19.67 72500 19 1 / 5 19.61 79000 21 1 / 6 18.21 77500 21 1 / 7 18.94 79500 21 1 / 8 19.88 82000 22

[0059] Table 4 summarizes the results of training different methods on multiple datasets, and Figure 4 The visualization of CSDGAN's training results on the OxFlowers102 dataset is presented. By observing these results, we can draw the following conclusions:

[0060] (1) Compared with the classic method AdaFM, the CSDGAN method uses a discriminator suppression module and a lightweight discriminator, which avoids the discriminator from quickly reaching the overfit state. This can significantly reduce the capability gap between the generator and the discriminator, and significantly improve the quality of the generated images and the stability of network training.

[0061] (2) Because models such as WGAN-GP use a large network structure, they cannot be successfully trained on a small dataset.

[0062] (3) Experimental results show that CSDGAN improves upon AdaFM in training results on the OxFlowers102, CelebA, and Cathedral datasets. Specifically, on the OxFlowers102 dataset, the optimal FID value trained by CSDGAN is approximately 21% higher than that trained by AdaFM; on the CelebA dataset, the optimal FID value trained by CSDGAN is approximately 18% higher than that trained by AdaFM; and finally, after training on the Cathedral dataset, the optimal FID value trained by CSDGAN is approximately 5% higher than that trained by AdaFM. In addition, CSDGAN performs better than other methods, showing significant improvement in the quality of the generator images.

[0063] Table 4: Optimal FID values ​​trained on different datasets using different methods in Example 1 of the present invention.

[0064] OxfordFlowers102 CelebA Cathedral DCGAN 124.24 56.08 101.90 WGAN-GP Failed 31.25 Failed AdaFM 23.18 28.84 22.16 TransferGAN Failed 25.31 Failed DADA Failed 37.58 33.12 DAGAN 43.12 49.46 32.15 CSDGAN 18.21 23.43 20.94

[0065] Example 2

[0066] To further verify the effectiveness of the method presented in this section on extreme datasets, we conducted experiments on the extremely small Flower's-25 dataset. Table 5 shows a comparison of the training results of the method presented in this section and the AdaFM method, including the FID value at the end of training, the number of iterations to reach the optimal FID value, and the training time. Experimental results show that the optimal FID value of the method presented in this section at the end of training is 80.11, which is approximately 4% higher than the optimal FID value of 85.4 of the AdaFM method. This indicates that the present invention also has good training performance on extremely small sample datasets and can effectively generate high-quality images. Figure 5 The training results of CSDGAN and AdaFM methods on the Flowers-25 dataset are visualized.

[0067] Table 5: Training results of CSDGAN and AdaFM on the Flowers-25 dataset

[0068] FID Number of iterations Time / h AdaFM 85.4 65000 22h CSDGAN 80.11 56500 19h

Claims

1. A small dataset sample oriented GAN discriminator capability suppression algorithm, characterized in that, The steps are as follows: Step 1: input a random vector z conforming to Gaussian distribution into a generator G, and the generator G generates a false image G(z) according to the input random vector; Step 2: input the false image G(z) generated by the generator G into a discriminator D, and input a real image data Xreal into the discriminator D; Step 3: the discriminator D discriminates the input false image G(z) and the real image Xreal, and discriminates the probability that a certain image is real or false; Step 4: the discrimination result of the discriminator D for the real image is marked as d_real, and the discrimination result for the false image is marked as d_fake, and d_real and d_fake are input into a discriminator suppression module; The structure of the discriminator suppression module is as follows: According to the objective function of GAN, when the generator is fixed, the objective function for training the discriminator is as follows: maxV(D, G) = ∫ x p data (x)logD(x) + p g log(1-D(x))dx The formula obtains the maximum value by adjusting D(x), so the partial derivative of D(x) is taken and the partial derivative function is 0, and the value of D(x) is as follows, which is the optimal solution of the discriminator D(x); As can be seen from the above formula, if the discriminator D is trained to be optimal, the following is obtained: The objective function of the discriminator is obtained by moving the term: V(D,G) = 2JS(p data ||p g )-log4 In the GAN training, when the discriminator reaches the optimal state, JS(p data ||p g ) = 0, V(D, G) in the objective function of the discriminator is equal to a constant log4, at this time the network cannot carry out effective back propagation, resulting in the termination of training; since the objective function of the discriminator can be simplified to solve the JS divergence problem only when the discriminator is close to the optimal state; therefore, the present application aims to constrain the discrimination ability of the discriminator, thereby avoiding the discriminator reaching the optimal state; the specific implementation method is to scale the judgment result of the discriminator before loss calculation, as follows: Df = S(D ini ) ± λD avg (λ > 0) where D ini represents the initial discrimination result of the discriminator in a batch, and D avg is the average value of the discrimination results in the batch, S(D ini ) represents random sampling of the discrimination values in the batch, and Df represents the discrimination result output by the discriminator finally; When the discriminator judges the real data, a discrimination value and an average value are randomly selected for judgment, if the discrimination value is greater than the average value in the same batch, the discrimination value is subtracted by λ times of the average value, so as to achieve the purpose of reducing the discrimination value; when the discriminator judges the false data generated by the generator, a discrimination value is still randomly selected for judgment, if the discrimination value is less than the average value in the same batch, the discrimination value is added by λ times of the average value; Step 5: the discriminator suppression module performs constraint processing on d_real and d_fake inputted, which is to randomly sample the input data and expand or reduce the value based on the average value of the input data; Step 6: the discriminator suppression module calculates the cross-entropy loss function of the constrained value after data processing, and calculates the loss value g_loss of the generator G and the loss value d_loss of the discriminator D respectively; Step 7: input g_loss into the generator G, and input d_loss into the discriminator D, and the two perform gradient update according to the corresponding loss value; Step 8: repeat the above steps after gradient update until the generated false image G(z) meets the expectation.

2. The small dataset sample oriented GAN discriminator capability suppression algorithm of claim 1, wherein, In the second step, the structure of the discriminator D is as follows: Firstly, the feature is extracted through a convolution layer with a kernel size of 3*3, and then output to the first resnet block of the first layer and then output to the second resnet block of the first layer, and then the result is output to the dropout layer for random parameter discarding; then, the dimension reduction processing is performed through the average pooling layer with a pooling window of 3*3 in the second layer; then, the result is output to the first resnet block of the second layer and then output to the second resnet block of the second layer, and then the result is continuously output to the dropout layer for random parameter discarding; this process is repeated four times, and the last two times do not need to add the dropout layer; finally, the view layer is responsible for flattening the multi-dimensional vector into a one-dimensional vector; then, the Leaky ReLU activation function is used to introduce a nonlinear transformation; finally, the Full connect full connection layer is used to integrate the features into a value and output.

Images