VAE-TTLP Model for Subset Conditioning with Learnable Tensor Train Prior
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) face challenges in generating objects that satisfy specific conditions, especially when some conditions are unknown during training or generation.
Innovation Solution
The VAE-TTLP model, which combines a variational autoencoder with a tensor train decomposition, allows for the generation of objects with specific properties by processing latent variables through an object decoder and optimizing reconstruction loss and Kullback-Leibler divergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a standard variational autoencoder is used for object generation, then the model can learn latent representations and generate reconstructed objects, but it cannot effectively generate objects with specific unknown conditions or properties
Solution Approach 1:
The patent segments the latent space into multiple independent latent variables, each corresponding to specific object properties. This allows the model to condition on specific properties by manipulating individual latent variables while keeping others free, enabling flexible generation of objects with partial known conditions without requiring complete specification of all properties.
Solution Approach 2:
The patent introduces an intermediary mapping layer between the latent space and the decoder that enables conditional generation. This intermediary structure allows the model to translate specific property conditions into appropriate latent variable configurations, bridging the gap between known properties and object generation without redesigning the entire VAE architecture.
2Manufacturing precision
If the model is trained to generate objects with all properties specified, then the generation accuracy for known properties improves, but the model cannot handle cases where some properties are unknown
Solution Approach 1:
The patent applies partial action by allowing the user to specify only a subset of properties for conditioning during generation, rather than requiring all properties to be defined. The model learns to generate objects with high accuracy for the specified properties while leaving unspecified properties to be determined by the sampling process, thus achieving both precision for known properties and versatility for unknown ones.
Solution Approach 2:
The patent makes the conditioning mechanism dynamic by allowing flexible specification of which latent variables correspond to which properties. This dynamic configuration enables the model to adapt to different generation scenarios where different subsets of properties are known or unknown, rather than being fixed to a predetermined conditioning scheme.
3Reliability
If reinforcement learning is applied to further train the model for specific characteristics, then the defined characteristics are achieved, but the training complexity and computational resources increase
Solution Approach 1:
The patent applies preliminary action by first training the VAE model on the complete dataset to learn the overall data distribution and property correlations before applying reinforcement learning. This preliminary training phase establishes a solid foundation that reduces the complexity of subsequent RL training, as the model only needs to refine specific characteristics rather than learn from scratch.
Solution Approach 2:
The patent maintains continuity of useful action by using the VAE's reconstruction objective and property prediction objectives throughout training, and then seamlessly integrating reinforcement learning as a continuous refinement phase. The useful action of generating and evaluating objects continues throughout both training stages, with RL building upon the VAE foundation rather than replacing it, thus managing complexity through progressive refinement.
Data Source
AI summary
The proposed model is a Variational Autoencoder having a learnable prior that is parametrized with a Tensor Train (VAE-TTLP). The VAE-TTLP can be used to generate new objects, such as molecules, that have specific properties and that can have specific biological activity (when a molecule). The VAE-TTLP can be trained in a way with the Tensor Train so that the provided data may omit one or more properties of the object, and still result in an object with a desired property.