GAN Convergence via Multi-Scale Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional generative adversarial networks (GANs) suffer from poor convergence properties during training, where the generator network's performance improvement leads to the discriminator network providing random feedback, causing the generator to train on incorrect data and resulting in a collapse in quality.
Innovation Solution
The proposed solution involves training the GAN in a cyclic configuration with multi-scale reconstruction losses and multi-scale generator losses, using down-sampling on the input image to generate multi-scale synthetic images, and employing a feature-to-image extraction method to improve convergence and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the generator network is trained in a traditional GAN setup, then the generator can produce synthetic data, but the discriminator network provides random feedback after a certain point causing the generator to train on incorrect data and collapse in quality
Solution Approach 1:
The patent segments the training process into multiple phases with different loss functions. In the initial phase, only generator loss is used to train the generator network. In the second phase, both generator loss and discriminator loss are used. This segmentation prevents the generator from training on random feedback by controlling when discriminator feedback is introduced.
Solution Approach 2:
The patent applies preliminary action by first training the generator network with only generator loss before introducing discriminator loss. This preliminary training phase ensures the generator learns basic synthesis capabilities before receiving potentially random discriminator feedback, preventing collapse in quality.
2Reliability
If multi-scale reconstruction losses are introduced to improve convergence, then the generator trains more effectively, but the device complexity increases
Solution Approach 1:
The patent introduces multi-scale reconstruction losses that operate at different resolution levels (e.g., original resolution and down-sampled resolution). This adds a dimensional aspect to the training process, allowing the generator to learn features at multiple scales simultaneously, which improves convergence without requiring fundamentally new architectural components.
Data Source
AI summary
A system and method for neural-network based image-to-image translation is provided. The system includes a generator network which receives an image and generates a first synthetic image based on the received image. The first synthetic image is different from the image in appearance, style, or domain. The system includes a feature-to-image generator that extracts a feature map from an intermediate layer of the generator network and generates a second synthetic image. The system further includes a first discriminator network that generates a result indicative of whether the first synthetic image is real or fake in comparison to a first reference image and a second discriminator network that generates a result indicative of whether the second synthetic image is real or fake in comparison to a second reference image. The system further includes a processor which trains the first generator network in adversarial manner based on losses associated with the results.