An image conversion method and system based on a generative adversarial network and a ResNts technology

Abstract

The invention belongs to the technical field of image processing, and discloses an image conversion method and system based on a generative adversarial network and a ResNts technology. The image conversion method comprises the following steps of firstly, designing an enhanced high-resolution dual GAN image conversion algorithm by utilizing a dual learning method combining two GAN confrontation mechanisms and a norm loss function, and realizing image conversion based on unsupervised learning by adopting an unlabeled data set through a model; secondly, realizing the conversion from source distribution to target distribution by introducing constraint conditions for reconstructing a consistency loss function, and then reconstructing the source distribution; and finally, adding a stable normalization layer in the discriminator. Compared with the resolution of images generated by Pix2pixGAN, CycleGAN and DualGAN, the average values of the ERGAN algorithmPSNR / SSIM provided by the invention are improved by 16% / 35%, 2% / 9% and 4% / 6% respectively.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to an image conversion method and system based on generative confrontation network and ResNets technology. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: [0003] Image translation has recently gained increasing research attention. Image conversion aims to map an image in the original domain to an image in another domain, that is, to convert a given sample image into a variety of new scenes, such as between different seasons, different climates, and scenes at different times of the day. Many computer vision and image processing tasks, especially image segmentation and image super-resolution, can also be considered as image transformations. Currently, image transformation based on deep learning is mainly applied to data augmentation. Data enhancement technology based on image transformation has...

AI Technical Summary

Problems solved by technology

In order to solve this problem, researchers currently implement image conversion through generative confrontation network technology, but this requires paired image data sets for training

Recent research results CycleGAN and DualGAN use unpaired data for training, but these two algorithms cannot mutually learn the feature correlation between the source domain image and the target image

In addition to these difficulties, in the design of the image conversion framework in the present invention, mutual dual GAN ​​will also be used, but only mutual dual GAN ​​cannot find the feature correlation between the source domain and the target domain

[0013] 2) In some current image conversion methods, such as Pix2pixGAN, CycleGAN, etc., most of the generator network structures use U-Net and encoder-decoder, and the traditional residual blocks form the ResNet network structure, and try to extract input feature information. Then, due to the proposed The image conversion algorithm must have strong versatility, so the input image style and objects are different, and the generator cannot extract the local feature information of the input image using the traditional network structure

This prevents conversion of high-res detailed texture images

[0014] 3) At present, there is a discriminator network training stability problem in the training of GAN

In a high-dimensional space, the density ratio estimation made by the discriminator is often inaccurate, and it is very unstable during training. After training such a discriminator, the training of the generator basically stagnates.

So far in academia, the Kullback-Leibler (KL) divergence is used to control the discriminator loss function, but KL speeds up the convergence of the training process, and the discriminator enters the ideal state very early, which prevents the discriminator from feeding back more information to the generator , causing the generator to fail to train

Images