Target Generator Smoothness Metric for Image Diversity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models face a bottleneck in training due to the scarcity of training examples, leading to generators 'memorizing' existing examples rather than generating diverse new images in the target domain, which is not useful for data augmentation.
Innovation Solution
A computer-implemented method that uses a pre-trained source generator to train a target generator, incorporating a smoothness metric to ensure gradual and realistic image generation by mimicking the smoothness of the source generator, even with limited training examples in the target domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a generator is trained in a GAN setting with limited training examples in the target domain, then the generator can produce images in the target domain, but the generator tends to memorize the training examples and map most noise vectors to exactly one training example, reducing diversity
Solution Approach 1:
The patent applies preliminary action by pre-training a source generator in a source domain with abundant training data before using it to train the target generator. This pre-training establishes a solid foundation of generation capabilities that can be transferred to the target domain, allowing the target generator to produce diverse images even when target domain training data is limited. The source generator's pre-training experience prevents memorization by providing a broader prior knowledge base.
Solution Approach 2:
The patent uses a pre-trained source generator as an intermediary between the abundant source domain data and the scarce target domain data. The source generator acts as a mediator that transfers its learned generation capabilities to the target generator through the smoothness metric, enabling the target generator to produce diverse images without directly memorizing limited target domain examples.
2Manufacturing precision
If a generator is trained to produce realistic images matching the target domain distribution, then the generated images are realistic, but the generator gets stuck on particular training examples and does not produce diverse images
Solution Approach 1:
The patent implements feedback through the smoothness metric that continuously monitors whether gradual changes in the latent space produce gradual changes in the generated images. This feedback mechanism penalizes the generator when it gets stuck on particular training examples, guiding it to produce smoother transitions and more diverse outputs while maintaining realism. The smoothness metric provides continuous guidance during training to balance realism and diversity.
Solution Approach 2:
The patent applies parameter changes by modifying the training objective function to include a smoothness term that operates in the latent space. By adjusting the smoothness parameter and its weighting in the loss function, the training process can control the trade-off between image realism and diversity, enabling the generator to produce diverse realistic images by changing how it maps latent vectors to image space.
3Adaptability or versatility
If cross-domain correspondence regularization is applied to prevent memorization, then image diversity improves, but the method requires a pre-trained generator from a different source domain
Solution Approach 1:
The patent applies universality by designing a smoothness metric that can be applied across different domains and generator architectures. The same smoothness regularization technique works for both cross-domain and same-domain scenarios, making the method universally applicable. The source generator can be from any domain, and the smoothness metric adapts to ensure diversity without requiring domain-specific modifications.
Data Source
Figure 1a
Figure 1b
Figure 2~3a
AI summary
A method (100) for training a target generator G to map input vectors z from a latent space Z to generate images G(z) in a target domain (2), comprising the steps of: • providing (110) a source generator Gs that has been pre-trained to generate, from the input vectors z, images Gs(z) in a source domain (1); • providing (120) a set of training images I∗ in the target domain (2); • providing (130) a discriminator D that is configured to classify whether an inputted image is a sample I from the target domain or an image G(z) generated by the target generator G; • providing (140) a generator smoothness metric S that measures to which extent a gradual change in the input vector z causes a gradual change of the image Gs(z) generated by the source generator Gs, respectively a gradual change of the image G(z) generated by the target generator G; • sampling (150) input vectors z from the latent space Z; • mapping (160), by the target generator G, the input vectors z to images G(z); • pooling (170) the images G(z) with training images I∗ into a pool P; • providing (180) samples p from the pool P to the discriminator D; • classifying (190), by the discriminator D, whether each sample p is a generated image G(z) or a sample I from the target domain (2); • optimizing (200) parameters (6) that characterize the behavior of the discriminator D towards the goal of improving the classification accuracy; and • optimizing (210) parameters (7) that characterize the behavior of the target generator G towards the goals of ∘ deteriorating the classification accuracy; and ∘ minimizing a difference between the generator smoothness metrics S(Gs) of the source generator Gs and S(G) of the target generator G.