Super-Resolution GAN Learning With Self-Attention Discriminator
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods using self-attention generative adversarial networks for super-resolution image generation are limited by the need for fixed input sizes and result in exponential calculation increases with varying input sizes, restricting flexibility and efficiency.
Innovation Solution
Implementing a self-attention mechanism only in the discriminator of a generative adversarial network while using convolutional neural networks for both, allowing flexible input sizes and reducing calculation complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the self-attention mechanism is introduced in both the generator and discriminator networks, then the accuracy of generated images is improved, but the calculation amount increases exponentially with input image size
Solution Approach 1:
The patent extracts the self-attention mechanism from the generator network and places it only in the discriminator network. This selective placement maintains the accuracy benefits of self-attention while eliminating the exponential calculation increase in the generator, as the generator uses standard convolutional layers only.
Solution Approach 2:
Instead of applying self-attention to the generator as traditionally done, the patent inverts the approach by applying it to the discriminator. This inversion allows the generator to process inputs of any size efficiently while the discriminator with self-attention evaluates the generated images for accuracy.
2Measurement precision
If the self-attention mechanism is introduced in the generator network, then the accuracy of generated images is improved, but the input data size is restricted to fixed size
Solution Approach 1:
The patent removes the self-attention mechanism from the generator network, allowing the generator to accept input images of any size without restriction. The generator uses standard convolutional layers that are inherently flexible to different input dimensions, while the discriminator with self-attention handles the accuracy evaluation.
3Measurement precision
If the self-attention mechanism is introduced in both networks, then the accuracy of generated images is improved, but the device complexity increases
Solution Approach 1:
The patent extracts the self-attention mechanism from the generator and places it only in the discriminator. This reduces the overall network complexity by eliminating the need for self-attention computations in the generator, while still maintaining high accuracy through the discriminator's self-attention capability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are a learning method and a learning system of a generative model, a program, a learned model, and a super resolution image generating device that can handle input data of any size and can suppress the amount of calculation at the time of image generation. A learning method according to an embodiment of the present disclosure is a learning method for performing machine learning of a generative model that estimates, from a first image, a second image including higher resolution image information than the first image, the method comprising using a generative adversarial network including a generator which is the generative model and a discriminator which is an identification model that identifies whether provided data is data of a correct image for learning or data derived from an output from the generator and implementing a self-attention mechanism only in a network of the discriminator among the generator and the discriminator.