Variational Method for Latent Variable Models with Missing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic health records (EHRs) often contain substantial missing data, which complicates analysis due to the difficulty in differentiating between missing values and negative data, and training generative models to handle these complexities is intractable for complex models.
Innovation Solution
A computer-implemented method using a stochastically approximated conditional evidence lower bound to learn predictive distributions with incomplete data, involving a discriminative variational autoencoder (DVAE) that maximizes a controlled gradient of the conditional evidence lower bound, employing neural networks to handle missing values and improve model tractability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If generative models are used to handle missing values in EHR data, then the ability to learn predictive distributions and handle missing data is improved, but the training becomes intractable for complex models
Solution Approach 1:
The patent introduces a variational inference framework as an intermediary between the complex generative model and the training process. By using a variational lower bound (ELBO) as a surrogate objective function, the method enables tractable training of complex generative models with missing data without requiring direct optimization of the intractable likelihood
2Productivity
If the gradient of the conditional evidence lower bound is maximized without control, then the learning speed may increase, but the stability of the maximization process deteriorates
Solution Approach 1:
The patent implements gradient clipping as a feedback mechanism during the maximization of the conditional evidence lower bound. By monitoring and constraining the gradient norm to remain below a threshold, the method maintains stable training while preserving sufficient learning progress
Solution Approach 2:
The patent applies gradient clipping before the gradient updates are applied to the parameters. This preemptive constraint prevents gradient explosion and stabilizes the optimization process before instability can occur, ensuring smooth training convergence
Data Source
AI summary
A computer-implemented method is provided for learning with incomplete data in which some of entries are missing. The method includes acquiring an incomplete set of covariates x including incomplete features {tilde over (x)} and an incomplete pattern m indicating missing entries of the incomplete set of covariates {tilde over (x)}. The method further includes obtaining, by a hardware processor, a predictive distribution pθ(y|x) of an outcome y by using the incomplete set of covariates x and a parameter θ, the parameter θ being unknown. A learning of the parameter θ includes performing a maximization by maximizing a stochastically approximated conditional evidence lower bound. The stochastically approximated conditional evidence lower bound includes a density ratio which is controlled by transforming a portion of parameters of the stochastically approximated conditional evidence lower bound to keep a gradient of the stochastically approximated conditional evidence lower bound below a threshold during the maximization.