Text-Guided Adversarial Makeup for Black-Box Face Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing facial privacy protection methods struggle to effectively deceive black-box face recognition systems without compromising user experience, often requiring large datasets, producing artifacts, or relying on reference images, and are susceptible to dataset biases.
Innovation Solution
A text-guided, encoder-decoder-based approach using a randomly initialized neural network optimizes weights at test-time to generate natural-looking adversarial makeup, employing a robust correspondence module and composite loss functions to ensure identity preservation and deceive black-box FR models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional adversarial makeup transfer methods are used, then privacy protection against black-box FR systems is achieved, but artifacts and unnatural appearance are produced
Solution Approach 1:
The patent changes the optimization parameters from image space to latent space of a pre-trained generator model. By optimizing latent codes rather than direct image pixels, the method achieves natural-looking outputs while maintaining privacy protection, eliminating artifacts that plague conventional image-space optimization approaches
Solution Approach 2:
The patent introduces a pre-trained generator model as an intermediary between the input face image and the adversarial perturbation. This mediator (the generator's latent space) enables natural-looking makeup transfer while preserving privacy, acting as a bridge that transforms raw adversarial modifications into natural appearances
2Object-generated harmful factors
If reference images are used to define makeup style, then natural appearance is improved, but user flexibility and adaptability are reduced
Solution Approach 1:
Instead of copying makeup style from a reference image, the patent uses text prompts to guide the generation of adversarial perturbations in latent space. This text-guided approach provides unlimited stylistic flexibility without being constrained by reference image requirements, while still producing natural-looking results through the pre-trained generator
3Object-generated harmful factors
If pre-trained generator models are used, then natural-looking outputs are achieved, but dataset biases are introduced
Solution Approach 1:
The patent performs preliminary training of the generator model on diverse, representative data before deployment. This pre-training establishes a foundation that captures natural variations in facial appearance across different demographics, enabling the model to generate natural-looking outputs while being less susceptible to specific dataset biases during adversarial optimization
4Reliability
If unrestricted adversarial examples are generated, then privacy protection is improved, but perceptual realism for human observers deteriorates
Solution Approach 1:
The patent optimizes adversarial perturbations in the latent space of a pre-trained generator rather than in image space. This parameter transformation enables unrestricted adversarial examples that maintain high perceptual realism, as the latent space optimization naturally produces coherent, natural-looking facial features while achieving strong privacy protection
Data Source
AI summary
Disclosed are a method and system to protect user facial privacy against unknown face recognition levels without compromising on a user's online experience. An input source to input an original face image. A training circuit configured to train a generator model to output an image that resembles the original face image. An optimizer configured to generate a protected face image based on the trained model that fools a black-box face recognition model, while imitating a makeup style. A display device to display the protected face image online.