Robust Deep Generative Model Adversarial Defense
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep generative models are susceptible to adversarial attacks, which can cause significant changes in their output due to perturbations in input data, leading to undesired behavior and negative consequences, with limited defensive solutions available.
Innovation Solution
A computer-implemented method and system that uses a provably robust training process for deep generative models, specifically variational autoencoders, by generating robustness certificates based on evidence lower bounds (ELBO) to defend against admissible perturbations, ensuring the models can distinguish between in-distribution and out-of-distribution data even under perturbation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard deep generative models are used, then the model can generate outputs efficiently, but the model is susceptible to adversarial attacks and perturbations causing incorrect outputs
Solution Approach 1:
The patent applies preliminary action by performing robustness certification during the training phase. The system computes bounds on the evidence lower bound (ELBO) for perturbed inputs before deployment, creating a certificate of robustness that guarantees correct behavior under adversarial perturbations. This preliminary computation embeds defensive properties into the model during training rather than adding post-processing complexity.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of bound propagation through the encoder and decoder networks. Instead of directly defending against perturbations, the system propagates bounds through the latent variable space, creating an intermediate representation that certifies robustness. This intermediary bound propagation layer allows the model to maintain its generative capabilities while gaining robustness guarantees.
2Reliability
If the model is trained to be robust to perturbations, then the model can defend against adversarial attacks, but the training process becomes more complex involving bound propagation and ELBO optimization
Solution Approach 1:
The patent applies parameter changes by modifying the training objective to optimize the lower bound of the ELBO rather than the standard ELBO. The system changes the parameters being optimized during training to include bound information, allowing the model to learn robust representations. This parameter transformation enables robustness training while maintaining compatibility with standard variational autoencoder frameworks.
3Measurement precision
If bound propagation is performed through encoder and decoder networks, then robustness certificates can be established, but the computational process requires generating multiple bounding data sets
Solution Approach 1:
The patent applies partial action by computing bounds only where necessary through the network. Instead of exhaustively evaluating all possible perturbations, the system propagates bounds through the latent variable space and computes ELBO bounds at critical points in the encoding and decoding processes. This partial computation approach provides sufficient robustness certification while maintaining training efficiency.
Data Source
AI summary
A machine learning system includes encoder and decoder networks. The machine learning system is configured to obtain input data, which includes sensor data and a radius of an p norm ball of admissible perturbations. Input bounding data is generated based on the input data. First bounding data and second bounding data are generated by respectively propagating the input bounding data on first and second outputs of the encoder network. Third bounding data is generated in association with a latent variable based on the first bounding data and the second bounding data. Fourth bounding data is generated by propagating the third bounding data on an output of the decoder network. A robustness certificate is established with respect to the input data by generating a lower bound of an evidence lower bound based on the first, second, third, and fourth bounding data. The encoder and the decoder networks are updated based on the robustness certificate to provide robustness to the machine learning system with respect to defending against the admissible perturbations.