Generative Model Training via Partial Derivative Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training generative adversarial networks (GANs) is challenging due to the difficulty in obtaining realistic training and test data, especially for automotive perception systems, and existing methods struggle to effectively update the generative model during the training process, leading to inefficient and unpredictable results.
Innovation Solution
The system trains a generative model to generate synthetic instances and a discriminative model to distinguish between synthetic and actual instances by using multiple discriminator scores for parts of the input, allowing for more detailed feedback and updating the gradient based on these scores to focus on improving realistic parts of the synthetic instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple discriminator scores for parts of input instances are used, then measurement precision of synthetic instance quality is improved, but device complexity of the training system increases
Solution Approach 1:
The discriminative model is divided into multiple discriminator components, each responsible for evaluating specific parts or features of synthetic instances. This segmentation allows for more precise quality assessment of different aspects (e.g., local structures, global consistency) while managing complexity through modular design, where each discriminator component can be independently trained and optimized.
2Manufacturing precision
If gradient updates are focused on specific parts based on discriminator scores, then manufacturing precision of synthetic instances is improved, but loss of information about overall quality increases
Solution Approach 1:
The training process applies local quality improvement by updating gradients selectively based on discriminator scores for specific parts of synthetic instances. Regions with lower discriminator scores (indicating poorer quality) receive stronger gradient updates, while well-generated regions receive minimal updates. This local quality approach improves manufacturing precision of problematic areas while preserving good regions, and the hierarchical structure ensures overall quality information is maintained through the aggregation of local improvements.
Data Source
AI summary
A system for training a generative model and a discriminative model. The generative model generates synthetic instances from latent feature vectors by generating an intermediate representation from the latent feature vector and generating the synthetic instance from the intermediate representation. The discriminative model determines multiple discriminator scores for multiple parts of an input instance, indicating whether the part is from a synthetic instance or an actual instance. The generative model is trained by backpropagation. During the backpropagation, partial derivatives of the loss with respect to entries of the intermediate representation are updated based on a discriminator score for a part of the synthetic instance, wherein the part of the synthetic instance is generated based at least in part on the entry of the intermediate representation, and wherein the partial derivative is decreased in value if the discriminator score indicates an actual instance.