Variational Autoencoder Training Using Sigma Points for Low-Variance Gradients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training variational autoencoders (VAEs) requires high-variance policy gradients, necessitating numerous samples to accurately estimate mean and covariance, leading to inefficient training times, especially in high-dimensional spaces.
Innovation Solution
Utilize sigma points to determine a plurality of fixed positions relative to the latent distribution, reducing variance in gradient computation by adapting the machine learning system based on a loss function that incorporates these points, allowing for faster training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sigma points are used to estimate mean and covariance in VAE training, then measurement precision is improved, but training time increases due to requiring 2n+1 training samples
Solution Approach 1:
The patent modifies the loss function by adding a regularization term that penalizes deviations of the estimated mean and covariance from their true values. This parameter change in the optimization objective enables accurate estimation using fewer than 2n+1 samples, resolving the contradiction between precision and training time
Solution Approach 2:
The patent introduces feedback through the regularization term in the loss function, which continuously guides the estimation process by comparing predicted mean and covariance against target values. This feedback mechanism improves estimation accuracy efficiently without requiring extensive sampling
2Ease of operation
If random sampling is used in high-dimensional latent space, then simplicity of operation is maintained, but reliability deteriorates as samples can be very far from the mean
Solution Approach 1:
The patent uses sigma points as intermediary elements between random sampling and the latent distribution. These predefined points serve as mediators that systematically explore the latent space while maintaining connection to the distribution's mean and covariance, improving reliability without sacrificing operational simplicity
Solution Approach 2:
The patent performs preliminary action by pre-defining sigma points based on the latent distribution's mean and covariance before actual sampling occurs. This preliminary setup ensures that subsequent sampling operations are reliable and accurate, preventing the problem of samples being far from the mean
Data Source
AI summary
Computer-implemented method for training a machine learning system. The machine learning system is configured to accept a sensor signal as input for anomaly detection and/or for sampling a trajectory of a traffic participant and/or for sampling of sensor signals and/or for determining a value characterizing a likelihood of a sensor signal with respect to a training dataset. The training includes: determining, by an encoder of the machine learning system and based on a training sensor signal, a first intermediate representation characterizing a mean of a latent distribution of a latent space and a second intermediate representation characterizing a variance and/or covariance of the latent distribution; determining, based on the first intermediate representation and the second intermediate representation, a plurality of sigma points with respect to the latent distribution; determining an output signal.