Face privacy protection method and system based on adversarial disturbance generation, and storage medium

Abstract

The invention discloses a face privacy protection method based on adversarial disturbance generation. The face privacy protection method comprises the following steps: step 1, acquiring a source face image to be protected; step 2, separating attribute features and identity features from the source face image by using feature de-entanglement; 3, inputting the identity features into an identity editing network, and performing adversarial identity editing by using faces with similar appearances but different identities; and step 4, re-fusing the edited identity features and the retained attribute features to generate a protected face image. According to the scheme, identity replacement is carried out in the feature space, and two-stage perceptual constraints are introduced to balance the protection intensity and the visual naturalness, so that high robustness and mobility protection of various unknown recognition models is realized; therefore, the problems of insufficient generalization ability and image distortion of the existing face privacy protection method in a social network scene are solved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method and system for protecting facial privacy based on adversarial perturbation generation. Background Technology

[0002] Facial privacy protection is a technology based on data security and privacy computing, and it is one of the important research directions in the fields of artificial intelligence security and computer vision. In recent years, with the widespread application of facial recognition technology, privacy protection methods have been gradually integrated into tasks such as identity anonymization, sensitive information hiding, and biometric encryption, such as de-identification processing, adversarial example generation, and privacy-aware image synthesis. The key to achieving the above privacy protection functions lies in the effective anonymization of facial features.

[0003] While traditional facial privacy protection technologies can effectively identify individuals, they often lead to significant perceptual distortion. These artifacts not only degrade user experience and impair data utility but may also expose human traces that can be exploited by attackers. For example, the published document CN113822256AA, published on 2021-12-21, entitled "Face Recognition Method, Electronic Device, and Storage Medium," discloses a face recognition method that includes: acquiring a depth map of the face to be recognized and generating a face point cloud map based on the depth map; performing key point detection on the face point cloud map using a key point detection model to obtain each key point on the face point cloud map; generating a face depth difference feature based on the key point correspondence and the depth value of each key point; the key point correspondence includes multiple key point pairs, and each key point pair includes two key points; the face depth difference feature includes the depth difference between each key point pair; matching the face depth difference feature with each preset face depth difference feature in a preset face database, and obtaining the face recognition result based on the matching result.

[0004] The main shortcomings of existing facial privacy protection methods are: the generated images often contain unnatural artifacts or distortions, disrupting facial coherence, reducing users' willingness to share, and failing to meet the need for authentic identity expression in social scenarios; the privacy protection mechanisms heavily rely on known models, and their effectiveness significantly decreases when facing unknown architectures or commercial API recognition systems, lacking cross-model transferability; and they lack stability to real-world environmental disturbances, causing the protection mechanisms to easily fail in practical end-to-end applications. Therefore, a facial privacy protection recognition method that can balance privacy and utility is needed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to realize a method and system for protecting facial privacy based on adversarial perturbation generation.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a face privacy protection method based on adversarial perturbation generation, comprising the following steps:

[0007] Step 1: Obtain the source face image to be protected;

[0008] Step 2: Separate attribute features and identity features from the source face image using feature deentanglement;

[0009] Step 3: Input the identity features into the identity editing network and perform adversarial identity editing using faces that look similar but have different identities;

[0010] Step 4: Re-integrate the edited identity features with the retained attribute features to generate a protected face image.

[0011] In step 1, the source face image to be protected is obtained. ,in ∈ Let x represent the image space, where x is a specific face image sample in the space. The protected face image y and the source face image x belong to different identities.

[0012] The source face images originate from open environments such as social networks, and must meet the requirements of the generation model. When given the input requirements, the model is defined by the parameter θ, which is used to optimize the output of a visually natural and protected image y = G(x) that can effectively resist malicious identification.

[0013] The entire process pipeline is defined as: identity transformation, facial manipulation. ,in It is an identity feature space, and the generative model is... The parameter θ is a parameter of the generator G.

[0014] In step 2, robust separation of identity and attributes is achieved by constructing an orthogonal feature space through a dual-branch encoder, including the following steps:

[0015] Step 2.1: Construct a dual-branch encoder network, where the identity encoder branch extracts identity features z. id The attribute encoder branch uses a multi-scale feature pyramid structure to capture domain-invariant attribute features z. att This achieves the separation of identity-related and identity-agnostic components;

[0016] Step 2.2: By maximizing the orthogonality constraint between identity features and attribute features, ensure the sufficiency of feature deentanglement, where identity features and attribute features satisfy z. id ·z att Orthogonal relation ≈0.

[0017] Step 3 includes the following steps:

[0018] Step 3.1: Extract reference samples from the identity neighborhood of the source image based on appearance similarity. While ensuring identity orthogonality, perform adversarial editing of identity features through an identity editing network.

[0019] Step 3.2: Introduce identity protection loss Λ pri By minimizing cosine similarity, the generated sample identity representation is forced to deviate from the source identity. The calculation formula is as follows:

[0020] ;

[0021] Where cos(·) calculates the cosine similarity between the original identity embedding and the edited identity embedding, E id (·) is a pre-trained ID extractor, and ε is a predefined threshold. Gradient backpropagation is stopped when the similarity is lower than τ to prevent excessive distortion.

[0022] Step 3.3: Introduce two levels of perceptual constraints to maintain visual usability, including pixel-level L1 loss, which preserves low-level visual attributes without loss of detail; and feature-level LPIPS loss, which maintains high-level attributes consistent with human perception by comparing deep features; the combined perceptual constraints are formalized as follows:

[0023] ;

[0024] Where Fp(·) represents the pre-trained deep feature extractor; λp is the hyperparameter of Fp(·);

[0025] Step 3.4: The overall optimization objective is:

[0026] ;

[0027] Here, λ* is a hyperparameter that controls different loss values. The final edited identity feature is generated through the identity editing network. The similarity between this feature and the source identity in the cosine space is lower than the threshold τ, while satisfying the perceptual constraint to maintain visual naturalness.

[0028] Step 4 includes the following steps:

[0029] Step 4.1: Modify the edited identity feature z using Adaptive Instance Normalization (AdaIN). in Injected into latent feature f eat In this representation, the calculation method is as follows:

[0030] ;

[0031] Where µ feat and σ featσ represents the channel mean and standard deviation, respectively. zin and µ zin Export from a pre-trained ID extractor;

[0032] Step 4.2: Introduce attribute preservation loss Λ att The feature map difference between the original image and the generated image in the multi-scale recursive layer is calculated using the following formula:

[0033] ;

[0034] Where z in To inject identity features, D i (·) represents the i-th layer feature extracted by discriminator D, N i H is the number of elements in this layer, H is the total number of layers, and h is the starting layer index.

[0035] Step 4.3: Employ the identity separation loss Λ id The consistency between injected and extracted identity features is measured using cosine similarity.

[0036] ;

[0037] Where z in To inject identity features, z ex Identity features extracted from the generated image;

[0038] Step 4.4: Introduce feature fusion loss. When the injected identity matches the original identity, it encourages the reconstructed image to be similar to the input image. The calculation formula is as follows:

[0039] ;

[0040] This loss ensures that the visual fidelity of the image is maintained during the identity feature replacement process;

[0041] Step 4.5: Perform adversarial visual enhancement using a multi-scale recursive discriminator, with the adversarial loss taking the form of hinged loss.

[0042] ;

[0043] Where ReLU(·) is the activation function, and D(·) represents the probability output by the multi-scale discriminator. This loss improves the realism of the generated image through adversarial training.

[0044] Step 4.6: The overall optimization objectives are as follows:

[0045]

[0046] Where, λ *These are hyperparameters that control different loss magnitudes. Total loss function Λ total2 By linking the above losses together, the parameters of the identity editing network and the generator are simultaneously optimized using the gradient descent algorithm. The final face image is generated by the reconstruction decoder of the generative adversarial network. This image meets the requirements of visual naturalness, and its identity features are sufficiently different from the source identity features to effectively resist the recognition of the face recognition system.

[0047] The data processing device performs a face privacy protection method based on adversarial perturbation generation as described in any one of claims 1 to 6.

[0048] A storage medium, wherein the storage medium is a computer-readable storage medium for storing software program code, the software program code being used to execute the face privacy protection method based on adversarial perturbation generation.

[0049] This invention addresses the shortcomings of existing face privacy protection methods, such as poor visual quality, weak model generalization ability, and insufficient environmental robustness. It proposes a protection method based on adversarial perturbation generation. By separating identity and attribute features through feature deentanglement technology and combining adversarial editing and multi-scale reconstruction mechanisms, identity replacement is achieved in an identity editing network. Two-level perceptual constraints are introduced to balance visual naturalness and privacy protection strength. This method effectively resists attacks from various face recognition models and maintains stable protection performance in real-world scenarios such as image compression and noise interference, significantly improving the practicality, transferability, and system robustness of privacy protection. Attached Figure Description

[0050] The following is a brief explanation of the content represented by each figure in this specification:

[0051] Figure 1 This is a flowchart of a face privacy protection method based on adversarial perturbation generation. Detailed Implementation

[0052] The following description, with reference to the accompanying drawings, details the specific implementation of the present invention, including the shape and structure of each component, the relative positions and connections between the parts, the function and working principle of each part, the manufacturing process, and the operation and use methods, to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0053] Adversarial approaches inject optimized noise to disrupt feature extraction, obfuscating identities while maintaining visual fidelity. However, they still suffer from noticeable artifacts, reliance on white-box assumptions, and limited portability to black-box APIs. In contrast, generative approaches synthesize alternative biometric links for facial identity separation and retain non-sensitive attributes to maintain practicality, but require model retraining for new recognition systems. Furthermore, social media users, in addition to privacy protection, also highly value experience sharing and visual quality assurance.

[0054] like Figure 1 The diagram shows the principle block of a face privacy protection method based on adversarial perturbation generation. This method includes the following steps:

[0055] Step 1: Obtain the source face image to be protected;

[0056] Step 2: Separate attribute features and identity features from the source face image using feature deentanglement;

[0057] Step 3: Input the identity features into the identity editing network and perform adversarial identity editing using faces that look similar but have different identities;

[0058] Step 4: Re-integrate the edited identity features with the retained attribute features to generate a protected face image.

[0059] The generated images often exhibit unnatural artifacts or distortions, disrupting facial coherence, reducing user willingness to share, and failing to meet the need for authentic identity representation in social scenarios. Privacy protection mechanisms heavily rely on known models, and their effectiveness significantly decreases when facing unknown architectures or commercial API recognition systems, lacking cross-model transferability. Furthermore, they lack stability against real-world environmental perturbations, making protection mechanisms prone to failure in practical end-to-end applications. This paper proposes a protection method based on adversarial perturbation generation. This method replaces the source face identity with visually similar but different features, combining feature deentanglement and two-level perceptual constraints. While ensuring visual quality and semantic consistency, it significantly improves robustness and transferability against unknown models and real-world noise perturbations. The following is a detailed description of the face privacy protection method based on adversarial perturbation generation:

[0060] In step 1, the source face image to be protected is obtained. ,in ∈ Let x represent the image space, where x is a specific face image sample within that space. The source face images originate from open environments such as social networks and must satisfy the requirements of the subsequent generation model. The input requirements are as follows: the model is defined by parameters θ, and its optimization objective is to output a visually natural and effectively resistant to malicious identification protected image y = G(x). To protect privacy, we require that the protected face image y belongs to a different identity than the source face image x. While ensuring privacy involves modifying image content, we prefer to minimize this modification to preserve more data utility attributes such as background, pose, and expression. The entire pipeline is defined as: identity transformation, face manipulation. ,in It is an identity feature space, and the generative model is... The parameter θ is a parameter of the generator G.

[0061] In step 2, feature deentanglement is used to separate attribute features and identity features from the source face image. A robust separation of identity and attributes is achieved by constructing an orthogonal feature space through a dual-branch encoder. Specifically, this includes:

[0062] Step 2.1: Construct a dual-branch encoder network, where the identity encoder branch extracts identity features z. id The attribute encoder branch uses a multi-scale feature pyramid structure to capture domain-invariant attribute features z. att This achieves the separation of identity-related and identity-agnostic components;

[0063] Step 2.2: By maximizing the orthogonality constraint between identity features and attribute features, ensure the sufficiency of feature deentanglement, where identity features and attribute features satisfy z. id ·z att Orthogonal relation ≈0;

[0064] In step 3, the identity features are input into the identity editing network, and adversarial identity editing is performed using faces that are similar in appearance but different in identity. Specifically, this includes:

[0065] Step 3.1: Extract reference samples from the identity neighborhood of the source image based on appearance similarity. While ensuring identity orthogonality, perform adversarial editing of identity features through an identity editing network.

[0066] Step 3.2: Introduce identity protection loss Λ pri By minimizing cosine similarity, the generated sample identity representation is forced to deviate from the source identity. The calculation formula is as follows:

[0067]

[0068] Where cos(·) calculates the cosine similarity between the original identity embedding and the edited identity embedding, E id (·) is a pre-trained ID extractor, and ε is a predefined threshold. Gradient backpropagation is stopped when the similarity is lower than τ to prevent excessive distortion.

[0069] Step 3.3: Introduce two levels of perceptual constraints to maintain visual usability, including pixel-level L1 loss, which preserves low-level visual attributes without loss of detail; and feature-level LPIPS loss, which maintains high-level attributes consistent with human perception by comparing deep features; The combined perceptual constraints are formalized as follows:

[0070]

[0071] Where F p (·) represents a pre-trained deep feature extractor, and λp is a hyperparameter of Fp(·);

[0072] Step 3.4: The overall optimization objective is:

[0073] ;

[0074] Here, λ* is a hyperparameter that controls different loss values. The final edited identity feature is generated through the identity editing network. The similarity between this feature and the source identity in the cosine space is lower than the threshold τ, while satisfying the perceptual constraint to maintain visual naturalness.

[0075] The program quantifies and guarantees "visual naturalness" by introducing a combined perceptual loss function. This function acts as a comprehensive judge, working collaboratively from different dimensions. Specifically, pixel-level L1 loss acts as a "detail guardian," minimizing the absolute difference in pixel values ​​between the generated image and the real image to preserve low-level visual attributes such as accurate color and clear texture, effectively preventing image blurring or over-smoothing. Meanwhile, feature-level LPIPS loss plays the role of a "semantic judge." It no longer focuses on individual pixels but uses a pre-trained deep network to extract high-level features of the image and compares the distances of these features in the semantic space to ensure that the generated image maintains consistency with human visual perception in high-level attributes such as overall structure and texture—key to achieving high naturalness. Ultimately, these two levels of constraints are formalized into a weighted loss function, allowing the optimization process to simultaneously consider detail fidelity and semantic naturalness. When the loss value is unsatisfactory, the signal is backpropagated to the model, driving it to adjust parameters, thus continuously generating visually more natural and deceptive images in iterations, forming an efficient closed-loop optimization system.

[0076] In step 4, the edited identity features are re-fused with the retained attribute features to generate a protected face image, specifically including:

[0077] Step 4.1: Modify the edited identity feature z using Adaptive Instance Normalization (AdaIN). in Injected into latent feature f eat In this representation, the calculation method is as follows:

[0078]

[0079] Where µ feat and σ feat σ represents the channel mean and standard deviation, respectively. zin and µ zin Export from a pre-trained ID extractor;

[0080] (AdaIN injection) is the forward propagation process of the model, whose parameters (such as the generator weights) are constrained by the various loss functions defined in steps 4.2 to 4.5;

[0081] Step 4.2: Introduce attribute preservation loss Λ att The feature map difference between the original image and the generated image in the multi-scale recursive layer is calculated using the following formula:

[0082]

[0083] Where z in To inject identity features, D i (·) represents the i-th layer feature extracted by discriminator D, N i H is the number of elements in this layer, H is the total number of layers, and h is the starting layer index.

[0084] Step 4.3: Employ the identity separation loss Λ id The consistency between injected and extracted identity features is measured using cosine similarity.

[0085]

[0086] Where z in To inject identity features, z ex These are the identity features extracted from the generated image.

[0087] Step 4.4: Introduce feature fusion loss. When the injected identity matches the original identity, it encourages the reconstructed image to be similar to the input image. The calculation formula is as follows:

[0088]

[0089] This loss ensures that the visual fidelity of the image is maintained during the identity feature replacement process;

[0090] Step 4.5: Perform adversarial visual enhancement using a multi-scale recursive discriminator, with the adversarial loss taking the form of hinged loss.

[0091]

[0092] Where ReLU(·) is the activation function, and D(·) represents the probability output by the multi-scale discriminator. This loss improves the realism of the generated image through adversarial training.

[0093] In steps 4.2 to 4.5, each loss function (such as Latt and Lid) independently calculates a scalar value to measure the performance gap of the model in a certain aspect. These loss values ​​are then aggregated into the total loss L. total In the middle. During training, L total The gradient is backpropagated to all parts of the model (including the AdaIN module and the generator), thereby simultaneously adjusting the parameters to optimize all objectives.

[0094] Step 4.6: Generate the final face image through the reconstruction decoder of the generative adversarial network. This image meets the requirements of visual naturalness, and its identity features are sufficiently different from the source identity features to effectively resist the recognition of the face recognition system.

[0095] A face privacy protection system based on adversarial perturbation generation includes a data receiving device, a data processing device, and an output device, such as a conventional PC computer. The data processing device executes the face privacy protection method based on adversarial perturbation generation. The data processing device generally includes a storage medium, which is a computer-readable storage medium for storing software program code used to execute the face privacy protection method based on adversarial perturbation generation.

[0096] This invention addresses the problems of poor visual quality, weak model generalization ability, and insufficient environmental robustness in existing face privacy protection methods. It separates identity and attribute features through feature deentanglement technology, combines adversarial editing and multi-scale reconstruction mechanisms to achieve adversarial editing of identity in the identity editing network, and introduces two-level perceptual constraints to balance visual naturalness and privacy protection strength. It maintains stable protection effect in real-world scenarios such as image compression and noise interference, significantly improving the practicality, transferability, and system robustness of privacy protection.

[0097] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A face privacy protection method based on adversarial perturbation generation, characterized in that, Includes the following steps: Step 1: Obtain the source face image to be protected; Step 2: Separate attribute features and identity features from the source face image using feature deentanglement; Step 3: Input the identity features into the identity editing network and perform adversarial identity editing using faces that look similar but have different identities; Step 4: Re-integrate the edited identity features with the retained attribute features to generate a protected face image.

2. The face privacy protection method based on adversarial perturbation generation according to claim 1, characterized in that, In step 1, the source face image to be protected is obtained. ,in ∈ Let x represent the image space, where x is a specific face image sample in the space. The protected face image y and the source face image x belong to different identities.

3. The face privacy protection method based on adversarial perturbation generation according to claim 2, characterized in that, The source face images originate from open environments such as social networks, and must meet the requirements of the generation model. When given the input requirements, the model is defined by the parameter θ, which is used to optimize the output of a visually natural and protected image y = G(x).

4. The face privacy protection method based on adversarial perturbation generation according to claim 3, characterized in that, The entire process pipeline is defined as: identity transformation, facial manipulation. ,in It is an identity feature space, and the generative model is... The parameter θ is a parameter of the generator G.

5. The face privacy protection method based on adversarial perturbation generation according to claim 1, characterized in that, In step 2, robust separation of identity and attributes is achieved by constructing an orthogonal feature space through a dual-branch encoder, including the following steps: Step 2.1: Construct a dual-branch encoder network, where the identity encoder branch extracts identity features z. id The attribute encoder branch uses a multi-scale feature pyramid structure to capture domain-invariant attribute features z. att This achieves the separation of identity-related and identity-agnostic components; Step 2.2: By maximizing the orthogonality constraint between identity features and attribute features, ensure the sufficiency of feature deentanglement, where identity features and attribute features satisfy z. id ·z att Orthogonal relation ≈0.

6. The face privacy protection method based on adversarial perturbation generation according to claim 1, characterized in that, Step 3 includes the following steps: Step 3.1: Extract reference samples from the identity neighborhood of the source image based on appearance similarity. While ensuring identity orthogonality, perform adversarial editing of identity features through an identity editing network. Step 3.2: Introduce identity protection loss Λ pri By minimizing cosine similarity, the generated sample identity representation is forced to deviate from the source identity. The calculation formula is as follows: ； Where cos(·) calculates the cosine similarity between the original identity embedding and the edited identity embedding, E id (·) is a pre-trained ID extractor, and ε is a predefined threshold. Gradient backpropagation is stopped when the similarity is lower than τ to prevent excessive distortion. Step 3.3: Introduce two levels of perceptual constraints to maintain visual usability, including pixel-level L1 loss, which preserves low-level visual attributes without loss of detail; and feature-level LPIPS loss, which maintains high-level attributes consistent with human perception by comparing deep features; the combined perceptual constraints are formalized as follows: ； Where Fp(·) represents the pre-trained deep feature extractor; λp is the hyperparameter of Fp(·); Step 3.4: The overall optimization objective is: ； Here, λ* is a hyperparameter that controls different loss values. The final edited identity feature is generated through the identity editing network. The similarity between this feature and the source identity in the cosine space is lower than the threshold τ, while satisfying the perceptual constraint to maintain visual naturalness.

7. The face privacy protection method based on adversarial perturbation generation according to claim 1, characterized in that, Step 4 includes the following steps: Step 4.1: Modify the edited identity feature z using Adaptive Instance Normalization (AdaIN). in Injected into latent feature f eat In this representation, the calculation method is as follows: ； Where µ feat and σ feat σ represents the channel mean and standard deviation, respectively. zin and µ zin Export from a pre-trained ID extractor; Step 4.2: Introduce attribute preservation loss Λ att The feature map difference between the original image and the generated image in the multi-scale recursive layer is calculated using the following formula: ； Where z in To inject identity features, D i (·) represents the i-th layer feature extracted by discriminator D, N i H is the number of elements in this layer, H is the total number of layers, and h is the starting layer index. Step 4.3: Employ the identity separation loss Λ id The consistency between injected and extracted identity features is measured using cosine similarity. ； Where z in To inject identity features, z ex Identity features extracted from the generated image; Step 4.4: Introduce feature fusion loss. When the injected identity matches the original identity, it encourages the reconstructed image to be similar to the input image. The calculation formula is as follows: ； This loss ensures that the visual fidelity of the image is maintained during the identity feature replacement process; Step 4.5: Perform adversarial visual enhancement using a multi-scale recursive discriminator, with the adversarial loss taking the form of hinged loss. ； Where ReLU(·) is the activation function, and D(·) represents the probability output by the multi-scale discriminator. This loss improves the realism of the generated image through adversarial training. Step 4.6: The overall optimization objectives are as follows: ； Where, λ * These are hyperparameters that control different loss magnitudes. Total loss function Λ total2 By linking the above losses together, the parameters of the identity editing network and the generator are simultaneously optimized using the gradient descent algorithm. The final face image is generated by the reconstruction decoder of the generative adversarial network. This image meets the requirements of visual naturalness, and its identity features are sufficiently different from the source identity features to effectively resist the recognition of the face recognition system.

8. A face privacy protection system based on adversarial perturbation generation, comprising a data receiving device, a data processing device, and an output device, characterized in that, The data processing device performs a face privacy protection method based on adversarial perturbation generation as described in any one of claims 1 to 6.

9. A storage medium, said storage medium being a computer-readable storage medium for storing software program code, characterized in that: The software program code is used to execute the face privacy protection method based on adversarial perturbation generation as described in any one of claims 1-7.

Images