A training method to ensure the smooth convergence of the maximum and minimum loss functions of the gan model

Abstract

The invention discloses a training method for ensuring the stable convergence of the maximum and minimum loss functions of a GAN model, and relates to the field of deep learning of the GAN model training method. The method solves the above problem by reasonably setting the parameter update conditions and frequency of the generator and the antagonist. The game imbalance problem between the generator and the antagonist during the training of the GAN model. The so-called game imbalance problem refers to the rapid convergence of one of the generator and the antagonist during the training process, making the optimization surface of the other party almost unsteerable, so it cannot be smoothly training phenomenon.

Description

technical field [0001] The invention relates to the field of deep learning of a GAN model training method, in particular to a training method for ensuring the stable convergence of the maximum and minimum loss functions of the GAN model. Background technique [0002] GAN models are widely used in image generation, speech generation, text generation and other fields. The training of the GAN model generally includes two steps, one is to update the parameters of the generator according to the gradient of the loss function of the generator, and the other is to update the parameters of the adversary according to the gradient of the loss function of the adversary. These two steps are generally performed alternately until the data distribution generated by the generator is close to the real data distribution, and the adversary cannot distinguish the output of the generator from the real data. However, in actual operation, this method of training alternately regardless of the conve...

AI Technical Summary

Problems solved by technology

These two steps are generally carried out alternately until the data distribution generated by the generator approaches the real data distribution, and the adversarial device cannot distinguish the output of the generator from the real data.

However, in the actual operation process, this method of blindly alternating training regardless of the convergence of the generator and the adversarial device will cause the game imbalance between the generator and the adversarial device

The loss function of the generator includes the calculation of the output of the generator by the adversarial device. When the adversarial device converges quickly, if the convergence speed of the generator cannot keep up, there is a high probability that the gradient of the loss function of the generator will continue to increase until the end Gradient explosion is not derivable in the field of numerical calculation, so that the optimizer cannot update the generator parameters, and finally the GAN model training fails

Images