Stochastic Inference Models Using Unpaired Data via Mediator Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning systems face challenges in training stochastic inference models between multiple random variables using unpaired data, as they require paired data for effective conditional and joint generation, which can be costly and not readily available.
Innovation Solution
A method involving a neural network with a variational autoencoder and a discriminative neural network is employed, using unpaired data to train the network with a loss function that includes terms for reconstruction loss, consistency deviations, and f-divergence, enabling conditional and joint generation without relying on paired data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If paired data is used to train stochastic inference models, then generation quality is improved, but data collection cost and complexity increase
Solution Approach 1:
The patent introduces a mediator network that bridges the gap between unpaired data and generation quality. The mediator learns to map from one data domain to another through unpaired data, enabling the system to achieve paired-data-like performance without actual paired data, thus resolving the contradiction between generation quality and data collection complexity
Solution Approach 2:
The patent changes the fundamental parameter of data requirement from paired to unpaired. By reformulating the training objective and using domain adaptation techniques, the system achieves effective stochastic inference model training with unpaired data, eliminating the need for complex paired data collection while maintaining generation quality
2Measurement precision
If paired data is required for training, then model accuracy is improved, but time and resources for data preparation increase
Solution Approach 1:
The patent creates a virtual copy of the paired data structure through the mediator network. The mediator learns to replicate the conditional relationships and dependencies that would exist in paired data, allowing the model to train on unpaired data while achieving accuracy comparable to paired data training, thus saving time on data preparation
Solution Approach 2:
The patent segments the data processing into independent domains that can be trained separately on unpaired data. By dividing the training into domain-specific unpaired data processing and then combining through the mediator, the system achieves accurate model training without requiring time-consuming paired data assembly, resolving the contradiction between model accuracy and data preparation time
Data Source
AI summary
A system and method for operating a neural network. In some embodiments, the neural network includes a variational autoencoder, and the training of the neural network includes training the variational autoencoder with a plurality of samples of a first random variable; and a plurality of samples of a second random variable, the plurality of samples of the first random variable and the plurality of samples of the second random variable being unpaired, the training of the neural network including updating weights in the neural network based on a first loss function, the first loss function being based on a measure of deviation from consistency between: a conditional generation path from the first random variable to the second random variable, and a conditional generation path from the second random variable to the first random variable.