Training method for guaranteeing stable convergence of maximum and minimum loss function of GAN model

Abstract

The invention discloses a training method for guaranteeing stable convergence of a maximum and minimum loss function of a GAN model. The invention relates to the field of deep learning of a GAN modeltraining method. According to the method, parameter updating conditions and frequencies of a generator and an adversarial device are reasonably set; the problem of game imbalance between a generator and an adversarial device in the training process of the GAN model is solved, and the game imbalance problem refers to the phenomenon that one of the generator and the adversarial device is quickly converged in the training process, so that the optimized curved surface of the other one is nearly underivable, and smooth training cannot be realized.

Description

technical field [0001] The invention relates to the deep learning field of a GAN model training method, in particular to a training method for ensuring the smooth convergence of the maximum and minimum loss functions of the GAN model. Background technique [0002] GAN models are widely used in image generation, speech generation, text generation and other fields. The training of the GAN model generally includes two steps, one is to update the parameters of the generator according to the gradient of the loss function of the generator, and the other is to update the parameters of the adversary according to the gradient of the loss function of the adversarial device. These two steps are generally carried out alternately until the data distribution generated by the generator approaches the real data distribution, and the adversarial device cannot distinguish the output of the generator from the real data. However, in the actual operation process, this method of blindly alternat...

AI Technical Summary

Problems solved by technology

These two steps are generally carried out alternately until the data distribution generated by the generator approaches the real data distribution, and the adversarial device cannot distinguish the output of the generator from the real data.

However, in the actual operation process, this method of blindly alternating training regardless of the convergence of the generator and the adversarial device will cause the game imbalance between the generator and the adversarial device

The loss function of the generator includes the calculation of the output of the generator by the adversarial device. When the adversarial device converges quickly, if the convergence speed of the generator cannot keep up, there is a high probability that the gradient of the loss function of the generator will continue to increase until the end Gradient explosion is not derivable in the field of numerical calculation, so that the optimizer cannot update the generator parameters, and finally the GAN model training fails

Images