GAN Likelihood Estimation Using Inference and Variance Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative adversarial networks (GANs) are implicit models that cannot directly compute sample likelihoods, limiting their application in tasks requiring likelihood estimation and anomaly detection.
Innovation Solution
A method is introduced to train a generative adversarial network system comprising a generator, discriminator, inference, and variance networks, using loss functions to learn an inverse function of the generator, enabling likelihood estimation and anomaly detection by mapping data samples to latent variables and calculating log-likelihood values through Riemannian metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GANs are used as implicit models for data generation, then the quality of generated samples is improved, but the ability to compute sample likelihoods directly is lost
Solution Approach 1:
The patent introduces an inference network as an intermediary component that learns to approximate the inverse function of the generator. This inference network serves as a mediator between the input data and the latent space, enabling likelihood estimation without requiring direct computational inversion of the generator. The inference network h(x) maps input data back to latent variables, and combined with the variance network, enables the computation of implicit likelihoods through the learned manifold approximation.
2Adaptability or versatility
If the discriminator is designed to classify samples from true data distribution and generated samples, then the adversarial training objective is achieved, but the ability to estimate likelihoods of samples not belonging to either distribution is lost
Solution Approach 1:
The patent segments the traditional GAN architecture by separating the classification function (discriminator) from the likelihood estimation function (inference network + variance network). The discriminator maintains its specialized role in distinguishing real from generated samples, while the inference network handles the broader task of mapping arbitrary samples to latent variables for likelihood computation. This segmentation allows each component to excel at its specific function without compromising the other.
3Measurement precision
If an inverse function of the generator is learned to project samples into latent space, then likelihood estimation becomes possible, but measurement consistence may be compromised
Solution Approach 1:
The patent transforms the deterministic inverse function problem into a probabilistic parameter estimation problem. Instead of learning a single deterministic inverse mapping, the system learns to estimate parameters (mean and variance) of the latent variable distribution given an input sample. The variance network explicitly models the uncertainty in this inversion process, allowing the system to provide not just a point estimate but a distribution over possible latent variables, thereby maintaining measurement consistence through proper uncertainty quantification.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
The thriving of deep models and generative models provides approaches to model high dimensional distributions. Generative adversarial networks (GANs) can approximate data distributions and generate data samples from the learned data manifolds as well. Presented herein are embodiments to estimate the implicit likelihoods of GAN models. In one or more embodiments, a stable inverse function of the generator is learned with the help of a variance network of the generator. The local variance of the sample distribution may be approximated by the normalized distance in the latent space. Simulation studies and likelihood testing on data sets validate embodiments, which outperformed several baseline methods in these tasks. An embodiment was also applied to anomaly detection. Experiments show that the embodiments herein can achieve state-of-the-art anomaly detection performance.