Deep counterfeiting active defense antagonistic attack method

Abstract

The invention discloses a deep-counterfeiting active defense antagonism attack method, which comprises the following steps: S1, acquiring an original face image, generating a target attribute tag, and loading a pre-trained deep-counterfeiting generation network and a face key point extractor; s2, defining disturbance variables of an A channel and a B channel of a Lab color space and an RGB space, and initializing the disturbance variables; s3, constructing a multi-modal loss function including generative network output difference loss, perception consistency loss, visual concealment constraint loss and facial key point damage loss; and S4, based on the constructed multi-modal loss function, performing staged optimization on the initialized disturbance variable to obtain optimized disturbance. According to the method, the Lab and RGB double-space disturbance variables are defined, and the multi-modal loss function is constructed to perform staged optimization, so that high visual concealment of the generative adversarial sample is ensured, and meanwhile, efficient and robust interference on the deep counterfeit model is realized.

Description

Technical Field

[0001] This invention relates to the field of deepfake defense, and more specifically to a method for proactive defense against deepfake attacks. Background Technology

[0002] With the rapid development of deep learning technology, deepfake technology, represented by Generative Adversarial Networks (GANs), has been able to efficiently generate highly realistic facial images or video sequences. This technology has shown positive application potential in fields such as entertainment and film production. However, its capabilities have also been maliciously exploited for creating and disseminating false information, engaging in identity fraud, and other illegal activities, posing a serious threat to public safety, judicial fairness, and individual privacy rights.

[0003] To address the challenges posed by deepfake technology, existing defense technologies have primarily developed along two paths: passive detection and active defense. Passive detection technology typically operates after the forged content has been generated and disseminated. Its core principle is to analyze the generated forged content and identify subtle traces left by model limitations or imperfections in the generation process, such as statistical inconsistencies between pixels, lighting and texture anomalies, and missing biosignals (like heartbeats). However, such methods have inherent limitations: their detection capabilities heavily rely on the inherent defects of specific forgery models or the generation of "fingerprints." When forgery models are upgraded or iterated, or when simple post-processing (such as compression or filtering) is performed on the generated content, these traces are easily eliminated or altered, leading to a sharp decline in the generalization ability of the detection method and making it difficult to cope with constantly evolving forgery techniques. Active defense technology, on the other hand, adopts a more forward-looking strategy, aiming to establish a protective barrier before forgery occurs. Its core idea is to pre-embed specific, imperceptible defense signals before the original genuine content (such as images and videos) is published or uploaded. When attackers attempt to manipulate protected content using deepfake models, the embedded signals interfere with the model's normal generation logic, leading to severely distorted and unusable outputs, thus preventing the generation of high-quality forgeries at the source. Currently, adversarial perturbations are a major research focus in the field of active defense. These methods add subtle perturbations, imperceptible to the human eye, to the original image, causing the perturbed image (adversarial example) to mislead deepfake models. Existing adversarial perturbation methods are mostly based on pixel-level modifications or global feature optimization, such as fine-tuning pixel values ​​in the RGB or LAB color space; or designing perturbations in the model's latent space or perceptual feature space to seek a balance between stealth and interference effect.

[0004] Despite the advanced concept of active defense, existing technologies, especially those based on adversarial perturbations, still have the following significant shortcomings: 1. Single-dimensional defense, easily circumvented: Most methods attack from only a single dimension, such as targeting only the color space (e.g., LAB attack) or only facial geometry (e.g., keypoint attack). Such single-dimensional perturbations are easily identified and countered by targeted defenses (e.g., color correction, histogram equalization) or post-processing techniques (e.g., keypoint detection and repair), leading to defense failure.

[0005] 2. Low attack efficiency and high computational cost: The process of generating effective adversarial perturbations often relies on a large number of iterative optimizations, which are computationally time-consuming and require high computing power. This makes it difficult for existing methods to meet the stringent processing speed requirements of real-time or online application scenarios (such as live streaming and real-time communication).

[0006] 3. Poor perturbation concealment and impaired visual quality: When optimizing perturbations, a lack of strong constraints on the visual quality of the generated image can easily lead to overly obvious perturbations, manifesting as visible noise, artifacts, or color distortion. This not only affects the appearance and usability of the protected image itself but also makes adversarial examples easier to detect with the naked eye or simple filters.

[0007] 4. Insufficient robustness and weak resistance to post-processing: The generated adversarial perturbations are usually very sensitive to common image processing operations. When the protected image undergoes routine post-processing such as network transmission compression (e.g., JPEG), resizing, rotation, or brightness adjustment, the effectiveness of the perturbation will be greatly reduced or even completely eliminated, rendering the defense barrier ineffective.

[0008] Therefore, there is an urgent need in this field for an active defense method against deepfakes that can overcome the above-mentioned defects, and can achieve efficient, robust and multi-dimensional collaborative adversarial attacks while ensuring high visual concealment, so as to effectively destroy the generation capability of deepfake models. Summary of the Invention

[0009] To overcome the shortcomings of existing technologies, the present invention aims to provide a method for proactive defense against deepfake attacks. This method defines Lab and RGB dual-space perturbation variables and constructs a multimodal loss function for phased optimization, so as to achieve efficient and robust interference against deepfake models while ensuring the high visual concealment of generated adversarial examples.

[0010] To achieve the objective of this invention, the following solution is adopted: A method for proactive defense against deepfake attacks includes the following steps: S1. Obtain the original face image, generate target attribute labels, and load the pre-trained deep fake generation network and facial key point extractor. S2. Define the perturbation variables for the A and B channels of the Lab color space and the RGB space, and initialize the perturbation variables; S3. Based on the original face image, the adversarial sample candidates composed of the original face image and the initialized perturbation variables, the target attribute label, the generator network, and the facial key point extractor, construct a multimodal loss function including generator network output difference loss, perceptual consistency loss, visual concealment constraint loss, and facial key point destruction loss. S4. Based on the constructed multimodal loss function, the initialized perturbation variables are optimized in stages to obtain the optimized perturbation; the staged optimization includes: the first stage optimizes the perturbation variables in the Lab color space, and the second stage optimizes the facial key points in the RGB space based on the optimization results of the first stage. S5. Add the optimized perturbation to the original face image to generate adversarial samples for defending against deepfakes.

[0011] Furthermore, in step S3, the multimodal loss function adopts a dynamic weight adjustment strategy during the optimization process, and the dynamic weights are adaptively adjusted according to the ratio of the current loss to the previous round loss.

[0012] Furthermore, in step S4, the first stage Lab color space optimization uses the Adam optimizer to iteratively optimize the perturbations of the A and B channels in the Lab space; the second stage RGB space optimization uses the momentum gradient descent method to iteratively optimize the perturbations in the RGB space and enhance the damage loss of facial key points.

[0013] Furthermore, in step S2, the perturbation budget ε of the perturbation variables of the A and B channels of the Lab color space and the RGB space is set to 0.05, and the perturbation is clipped to the range of [-ε, ε] after each iteration during the optimization process.

[0014] Furthermore, this deepfake active defense adversarial attack method is configured to generate adversarial examples with an L2 error of less than 0.00013, a structural similarity (SSIM) greater than 0.98, and a peak signal-to-noise ratio (PSNR) greater than 45 dB from the original image, in order to meet the visual concealment requirements.

[0015] Further, in step S3, the perceptual consistency loss is obtained by extracting high-level features of the original image and adversarial sample candidates through the VGG network and calculating the mean square error between the two.

[0016] Further, in step S3, the facial key point destruction loss is obtained by extracting the key point heatmap of the adversarial example candidate through the facial key point extractor, and calculating the difference between the heatmap and the zero heatmap.

[0017] Furthermore, in step S2, the Lab color space can be replaced with the YCrCb color space, and perturbation initialization and optimization are performed in the Cb and Cr channels.

[0018] Furthermore, in step S1, the facial key point extractor uses an HRNet model pre-trained on a 300W dataset, or a key point detection model from the face_alignment library.

[0019] Furthermore, the Adam optimizer can be replaced with the RMSprop optimizer, and the momentum gradient descent method can be replaced with the Nesterov accelerated gradient method.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention significantly improves attack efficiency and drastically reduces computational costs. Existing techniques typically require separate attacks on color space and facial features, each requiring numerous iterations (usually exceeding 100), resulting in long computation times and high resource consumption. This invention organically combines LAB color space attacks with facial keypoint attacks, defining and co-optimizing them in stages within a unified framework, achieving a joint and efficient attack on both color and structural dimensions. This method can simultaneously optimize both dimensions in the same or even fewer iterations. With only a small perturbation (L2 error of only 0.00013), it achieves a 100% effective interference rate, far exceeding the 96.7% of a single Lab attack, significantly improving attack generation efficiency and stability while maintaining visual concealment.

[0021] 2. This invention effectively balances the contradiction between perturbation concealment and attack effectiveness. Existing techniques often fall into the dilemma of either significant perturbations but strong attacks, or concealed perturbations but weak attacks, due to the simplicity of their loss function design. This invention achieves precise and coordinated control of attack effectiveness and visual quality by constructing a multimodal loss function that integrates generator network output difference loss, perceptual loss, visual concealment constraint loss (L2 constraint), and facial keypoint destruction loss. This method ensures that the added perturbations have high visual concealment while effectively forcing the forged images output by the deepfake model to significantly decrease in structural similarity (SSIM) and peak signal-to-noise ratio (PSNR), thus achieving a dual optimization of concealment and destructiveness.

[0022] 3. This invention significantly improves resistance to circumvention and enhances defensive robustness. Traditional single-dimensional attacks (such as color-only attacks or keypoint-only attacks) are easily countered by targeted post-processing techniques (such as color correction and keypoint repair). This invention initiates collaborative perturbations at two levels: color channels (Lab space) and facial keypoint structure (RGB space). This causes forgery models to face dual interference from color distortion and facial feature misalignment when processing adversarial examples. Combined with the constraint of multimodal loss, this method achieves a perturbation success rate of up to 100%, significantly enhancing resistance to targeted defenses and improving the reliability and robustness of the defense scheme.

[0023] 4. This invention optimizes overall resource consumption, making it suitable for practical deployment. Through dynamic weight adjustment strategies, momentum acceleration technology, and a compact process design, this invention effectively optimizes memory usage and computational resource consumption while ensuring excellent attack performance. Experiments show that this method can run efficiently on consumer-grade GPUs, providing a solid and feasible technical foundation for building an online deepfake proactive defense system capable of real-time or near-real-time responses. Attached Figure Description

[0024] Figure 1 This is a flowchart of the deepfake proactive defense adversarial attack method in an embodiment of the present invention; Figure 2 This is a summary diagram of the attack effects on face samples in embodiments of the present invention. Detailed Implementation

[0025] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0026] This invention provides a method for proactive defense against deepfake attacks. This method innovatively introduces a facial keypoint destruction mechanism based on the existing adversarial perturbation method using the Lab color space, achieving two-stage joint optimization through a combined attack approach. First, perturbations are added to the A and B channels of the Lab color space (utilizing the difference in human eye sensitivity to color channels to ensure visual concealment). Then, directional perturbations are applied to the RGB color space (destroying the keypoint heatmap to approach a zero heatmap, thereby compromising facial feature detection accuracy). Both steps employ a multi-loss fusion mechanism, combining keypoint destruction loss, perceptual loss, L2 constraint loss, and generator output difference loss, and using dynamic loss weights. This solves the problem of limited effectiveness of traditional methods against DeepFake models that rely on facial structure, achieving more robust attack performance.

[0027] like Figure 1As shown, the deepfake proactive defense adversarial attack method of this invention includes the following steps: S1. Obtain the original face image, generate target attribute labels, and load the pre-trained deep fake generation network and facial key point extractor.

[0028] In this embodiment, step S1 is the data preparation and model initialization step, the details of which are as follows: The inputs for this step include: the original face image (RGB format), the target domain attribute label set (hair color, gender, etc. in CelebA), the pre-trained generative network (StarGAN), and the facial landmark extractor (HRNet or face_alignment library).

[0029] The outputs of this step are: the preprocessed image tensor, the target label list, the initialized generator network, and the facial landmark extractor.

[0030] Furthermore, this step requires image standardization, specifically as follows: The original RGB image is converted into a tensor with values ​​ranging from -1 to 1. The conversion method is to standardize it using a mean of 0.5 and a standard deviation of 0.5, and denoted as the original image tensor X_nat.

[0031] Further, this step requires generating target tags, specifically: The `create_labels` function generates a list of target domain labels, `c_trg_list`. For hair color-related attributes, only one color is set as valid at a time; for other attributes (such as gender and age), the inverse value of the original attribute is taken.

[0032] Further, this step requires model loading, specifically: Generative Network: Load the StarGAN generator G, whose structure includes an initial convolutional layer (64 filters), 2 downsampling layers, 6 residual blocks, 2 upsampling layers, and an output layer (Tanh activation function), used to simulate the image generation process of depth fake.

[0033] Facial landmark extractor: If using HRNet, load the weights pre-trained on the 300W dataset (HR18-300W.pth) using the HRNetLandmarkExtractor class and output a heatmap of facial landmarks; if using the face_alignment library, generate a heatmap of 68 landmarks using the RealLandmarkExtractor class.

[0034] S2. Define the perturbation variables for the A and B channels of the Lab color space and the RGB space, and initialize the perturbation variables.

[0035] In this embodiment, step S2 is the attack parameter configuration and perturbation initialization step, the specific content of which is as follows: The inputs for this step include: the original image X_nat, attack iteration parameters (Lab space iterations lab_iters, facial landmark space iterations landmark_iters), and perturbation budget ε (e.g., 0.05).

[0036] The output of this step is: initial perturbation in Lab space (A / B channels) and initial perturbation in RGB space.

[0037] Furthermore, the two-space perturbation variables in this step are defined as follows: Lab space perturbation: Initialize a learnable variable (shape matches the height and width of the image, containing 2 channels) corresponding to the A and B channels of the Lab color space. Take advantage of the fact that the human eye has low sensitivity to these two color channels to ensure the concealment of the perturbation.

[0038] RGB space perturbation: Initialize a variable X_adv with the same shape as the original image, which will be used to subsequently destroy the facial key point heatmap.

[0039] S3. Based on the original face image, the adversarial sample candidates composed of the original face image and the initialized perturbation variables, the target attribute label, the generator network, and the facial key point extractor, construct a multimodal loss function including generator network output difference loss, perceptual consistency loss, visual concealment constraint loss, and facial key point destruction loss.

[0040] In this embodiment, step S3 is the multimodal loss function construction step, the specific content of which is as follows: The inputs for this step include: the original image X_nat, the adversarial example X_adv, the target label c_trg, the generative network G, the facial landmark extractor, and the VGG feature extractor.

[0041] The output of this step is: the total loss value of fusion generation differences, perceptual consistency, and facial key point destruction.

[0042] Furthermore, the network output difference loss is generated in this step, specifically as follows: Image forgery without attack: The original image and target label are processed by a generative network to obtain the forgery result without attack: gen_noattack=G(X_nat,c_trg).

[0043] Image forgery under attack: The generative network processes the adversarial sample and target label to obtain the forged result gen_adv=G(X_adv,c_trg) under attack.

[0044] Loss calculation: L_gen=F.mse_loss(gen_adv,gen_noattack), which destroys the effect of depth fake by maximizing this loss.

[0045] Furthermore, the perceptual consistency loss in this step specifically involves: extracting high-level features of the original image and adversarial examples using the first 16 layers of the VGG16 network, and calculating the difference between these two features (measured by mean squared error). The aim is to constrain the perturbation from changing the overall visual perception of the image.

[0046] Furthermore, the visual concealment constraint (L2 loss) in this step is specifically as follows: after converting the original image and adversarial sample candidates to a numerical range of 0 to 1, the pixel difference between the two is calculated (measured by mean square error) to limit the intensity of the perturbation and ensure that it is difficult for the human eye to detect.

[0047] Furthermore, the facial keypoint destruction loss in this step specifically involves: extracting facial keypoint heatmaps of adversarial example candidates, and calculating the difference between these heatmaps and the zero heatmap to weaken the accuracy of keypoint detection.

[0048] Furthermore, regarding the fusion of total losses from combined attacks: Phase 1 (Lab Space Optimization): The total loss consists of the generator network output difference loss, perceptual consistency loss, and visual concealment constraint loss; dynamic loss weights were used, with initial base weights of 1, 0.3, and 1.

[0049] The second stage (RGB space optimization): The total loss consists of the generator network output difference loss, facial keypoint destruction loss, 0.3x perceptual consistency loss, and visual concealment constraint loss. Dynamic loss weights were used, with initial base weights of 1, 1, 0.3, and 1.

[0050] Dynamic loss weights: In each iteration, the weights are adjusted iteratively based on the ratio of the current loss to the previous loss. If the loss decreases slowly (ratio > 0.9), the weight is increased by 1.1 times that of the previous round. If the loss decreases too quickly (ratio < 0.5), the weight is decreased by 0.9 times that of the previous round. If the loss decreases normally, the weight remains unchanged.

[0051] S4. Based on the constructed multimodal loss function, the initialized perturbation variables are optimized in stages to obtain the optimized perturbation; the staged optimization includes: the first stage optimizes the perturbation variables in the Lab color space, and the second stage optimizes the facial key points in the RGB space based on the optimization results of the first stage.

[0052] S5. Add the optimized perturbation to the original face image to generate adversarial samples for defending against deepfakes.

[0053] In this embodiment, step S4 is a joint perturbation optimization step, the specific content of which is as follows: The inputs for this step include: initial perturbation variables, total loss function, number of iterations (50 in Lab space and 30 in keypoint space), and perturbation budget ε.

[0054] The output of this step is: the optimized adversarial example.

[0055] Furthermore, the first stage of Lab space perturbation optimization is as follows: Inputs for this stage: original image X_nat, target label list c_trg_list, generator network G, Lab space iteration count lab_iters.

[0056] Optimization process: The perturbation variables in the Lab space are iteratively adjusted using the Adam optimizer (learning rate 1e-4). In each iteration, the perturbation is limited to the range of -ε to ε in order to minimize the total loss in the first stage.

[0057] Output of this stage: Intermediate adversarial sample X_lab_adv optimized by Lab space.

[0058] Furthermore, the second phase of RGB space keypoint orientation optimization specifically includes: The inputs for this stage are: intermediate adversarial sample X_lab_adv, target label list c_trg_list, generator network G, facial landmark extractor, and landmark space iterations landmark_iters.

[0059] Optimization process: Using intermediate adversarial examples as initial values, the momentum gradient descent method (momentum factor 0.5) is used to optimize the perturbation in the RGB space, focusing on enhancing the damage loss of facial key points. After each iteration, the perturbation is clipped to the allowable budget range.

[0060] Output of this stage: The final adversarial sample X_adv obtained after two-stage joint optimization.

[0061] This invention employs an innovative dual-space joint perturbation mechanism, combining Lab color space perturbation with RGB space perturbation. This method leverages the low sensitivity of the human eye to the A / B channels in Lab space to ensure the visual concealment of the perturbation, and achieves effective attack by applying targeted perturbations to the facial feature regions in RGB space. Specifically, the targeted attack strategy based on facial feature heatmaps introduces a facial keypoint extractor to minimize the difference between the key region heatmap and the zero heatmap, thereby specifically disrupting the facial topological features upon which the DeepFake generation model relies, significantly improving the attack's targeting. Furthermore, by designing a multimodal loss function fusion scheme, a staged loss function optimization strategy is adopted: the first stage is dominated by generational difference loss, supplemented by perceptual loss and L2 constraint loss; the second stage strengthens the facial keypoint disruption loss, thus balancing attack effectiveness and visual concealment, effectively overcoming the limitations of traditional single loss functions.

[0062] Experimental example: The deepfake proactive defense adversarial attack method of this invention is applied to the StarGAN model. The implementation process includes experimental environment configuration, data preprocessing, model initialization, attack execution, and effect verification.

[0063] I. Experimental Environment Configuration: The hardware environment uses the following configuration: NVIDIA RTX 3090 graphics card (24GB VRAM), Intel i9-12900K CPU (16 cores and 24 threads), and 64GB of memory.

[0064] The software environment is based on the Python 3.8 programming language, with the deep learning framework being PyTorch 1.12.0 and its companion torchvision 0.13.0. The dependencies used include: OpenCV 4.5.5 for image preprocessing, scikit-image 0.19.3 for calculating SSIM and PSNR metrics, face_alignment 1.3.5 as an alternative facial landmark extractor, numpy 1.21.5 and PIL 9.0.1 for data format conversion, and pywt 1.3.0 for wavelet transform to aid feature analysis.

[0065] II. Dataset Preparation and Preprocessing: The experiment used the CelebA face dataset, which contains 202,599 face images with a resolution of 178×218, each image labeled with 40 attributes. Preprocessing steps included: 1. Cropping: Preserve the 178×178 area in the center of the image to remove edge-irrelevant information.

[0066] 2. Scaling: Adjust the resolution of the cropped images to a uniform 256×256.

[0067] 3. Standardization: Convert pixel values ​​from the range of 0 to 255 to the range of -1 to 1, based on a mean of 0.5 and a standard deviation of 0.5.

[0068] 4. Attribute Filtering: Select the following 5 target attributes for subsequent target tag generation: "Black hair", "Blond hair", "Brown hair", "Male", and "Young".

[0069] III. Model and Parameter Configuration: 1. Generative Networks: The generative network uses a pre-trained StarGAN generator. The model structure parameters are as follows: the initial convolutional layer is a 7×7 convolution with 64 filters and padding of 3; it includes two downsampling layers, each using a 4×4 convolution (stride of 2), with the number of filters doubling sequentially (64→128→256); it includes six residual blocks, each consisting of two 3×3 convolutions, an InstanceNorm layer, and a ReLU activation function; it includes two upsampling layers, each using a 4×4 transposed convolution (stride of 2), with the number of filters halved sequentially (256→128→64); the output layer is a 7×7 convolution with a Tanh activation function, outputting a 3-channel RGB image. Specifically, the initial convolutional layer has 64 filters and 6 residual blocks.

[0070] Weight loading: Use pre-trained model weights, the loaded weights correspond to the results of the model training up to 200,000 iterations.

[0071] 2. Facial Keypoint Extractor: An HRNet model pre-trained on the 300W dataset is used, and the pre-trained weight file "HR18-300W.pth" is loaded. This extractor outputs a heatmap of 68 facial keypoints (corresponding to the 68-point annotation standard of the 300W dataset). The higher the heatmap value, the greater the probability that the corresponding area is a keypoint.

[0072] 3. Attack Parameter Settings: The attack type is a combined attack, namely, combining Lab color space perturbation with image space perturbation based on facial key points. The number of optimization iterations in Lab color space is 50, and the number of optimization iterations in RGB color space is 30. The maximum allowable perturbation amplitude ε is set to 0.05. The batch size (batch_size) is set to 1.

[0073] IV. Attack Execution and Evaluation Steps: 1. Data Loading: The preprocessed CelebA image and attribute labels are read using a data loading tool. Each time, one image is processed, and the loading mode is test mode.

[0074] 2. Target Tag Generation: Generate 5 types of target attribute tags using the tag generation method. The generation rules are as follows: For hair color attributes (black hair, yellow hair, brown hair), only one type of attribute is set to a valid state each time, and the remaining hair color attributes are set to invalid; for non-hair color attributes (male, elderly), the value is taken as the opposite of the original attribute.

[0075] 3. Combination attack execution: The attack consists of two phases, and the specific process is as follows: Phase 1 (Lab Color Space Optimization): Perturbations are added to the a and b channels of the Lab color space. The Adam optimizer is used with a learning rate of 0.0001 and 50 iterations. The loss function consists of generation loss, perceptual loss, and L2 loss, and a dynamic weight adjustment strategy is applied. This phase outputs an intermediate adversarial example, i.e., the image with the initial perturbations.

[0076] The second stage (RGB space optimization): Based on the intermediate adversarial examples mentioned above, further optimization is performed in the RGB color space. The optimization tool used is momentum gradient descent with a momentum factor of 0.5 and 20 iterations. The loss function consists of generation loss, keypoint destruction loss, perceptual loss, and L2 loss, and a dynamic weight adjustment strategy is applied. This stage outputs the final adversarial example, i.e., the image with the final perturbation added.

[0077] 4. Calculation of evaluation indicators: To quantitatively evaluate the performance of this method, the following two types of metrics are calculated: Perturbation stealth metrics: used to evaluate the difference between the original image and the generated final adversarial example. Specifically, they include: L2 error: Calculates the mean square error of corresponding pixel values ​​in two images to measure the overall difference.

[0078] Structural Similarity (SSIM): Evaluates the structural similarity between two images; the closer the value is to 1, the more similar they are.

[0079] Peak Signal-to-Noise Ratio (PSNR): Evaluates the sharpness and quality of an image; a higher value generally indicates better visual quality.

[0080] Attack effectiveness metrics: used to evaluate the method's ability to interfere with deepfake models. Specifically, they include: Obtain "non-attack generated image" (i.e., the fake result obtained by passing the original image through the generation network) and "attack generated image" (i.e. the fake result obtained by passing the final adversarial sample through the same generation network).

[0081] Calculate the L2 error, SSIM, and PSNR between the two sets of forged images.

[0082] The number of samples with an L2 error greater than 0.05 between "images generated with attacks" and "images generated without attacks" is counted, and the proportion of these samples to the total test samples is calculated, which is defined as the "effective interference rate".

[0083] 5. The experimental results are as follows: (1) Disturbance concealment analysis: L2 error: 0.0001287101476918906 (close to 0, indicating that the mean square error of the corresponding pixel values ​​of the original sample and the adversarial sample is small and the perturbation amplitude is extremely small). SSIM: 0.0001287101476918906 (close to 1, indicating that the original sample and the adversarial sample are almost visually identical). PSNR: 45.01058626875839 (far higher than 20, indicating that the disturbance is imperceptible to the human eye).

[0084] Conclusion: The disturbance is almost invisible to the naked eye, thus meeting the concealment requirement.

[0085] (2) Attack effect analysis: L2 error: 0.7069597840309143 (The output of the fake model differs significantly between the original sample and the adversarial sample, indicating that the fake effect is significantly compromised). SSIM: 0.3087604012340307 (indicating a large difference in the structure of the generated images from the original sample and the adversarial sample); PSNR: 8.147743141142678 (far below 20, indicating severe distortion in the generated image); Effective interference rate: 1.0 (meaning 100% of the samples were successfully interfered with).

[0086] (3) Visualization results: like Figure 2 As shown in the figure, this graph compares the attack effects on 5 sets of face samples. Each row in the graph corresponds to one set of samples, and the columns in each row, from left to right, represent: Column 1: Original human face images, serving as the experimental control group.

[0087] Column 2: Adversarial samples generated by this invention.

[0088] Columns 3-4: DeepFake results of the original image with "black hair" attribute edited using the StarGAN model.

[0089] Column 4: DeepFake forgery results of adversarial examples with "black hair" attribute edited using the StarGAN model.

[0090] Column 5: DeepFake forgery results of the original image with "yellow hair" attribute edited using the StarGAN model.

[0091] Column 6: DeepFake forgery results of adversarial examples with "yellow hair" attribute edited using the StarGAN model.

[0092] Column 7: DeepFake forgery result of the original image with "brown hair" attribute edited using the StarGAN model.

[0093] Column 8: DeepFake forgery results of adversarial examples with "brown hair" attribute edited using the StarGAN model.

[0094] Column 9: DeepFake forgery result of the original image with "man" attribute edited using the StarGAN model.

[0095] Column 10: DeepFake forgery results of adversarial examples with "man" attribute edited using the StarGAN model.

[0096] Column 11: DeepFake forgery result of the original image with "old man" attribute edited by the StarGAN model.

[0097] Column 12: DeepFake forgery results of adversarial examples with "old person" attribute editing completed by the StarGAN model.

[0098] The visual features of the second column (adversarial sample) of each group of samples in the figure are highly consistent with those of the first column (original image), verifying the concealment of the perturbation. After DeepFake forgery using the StarGAN model, each group of adversarial samples in the figure exhibits obvious facial distortion and texture disorder, verifying the effectiveness of the attack scheme of this invention.

[0099] V. Comparative Analysis: 1. Comparison of disturbance concealment: The following table shows a comparison of the stealth metrics between the two strategies: Lab attacks and combined attacks. Therefore, the SSIM and PSNR of the Lab attack alone are higher than those of the combined attack. Despite the differences, the combined attack of this invention also meets the core requirement of stealth, that is, the perturbation cannot be directly detected by the human eye.

[0100] 2. Comparison of attack effects: The table below compares the interference effectiveness metrics of the two attack strategies on the forged model: Therefore, the L2 error of the Lab attack alone is higher than that of the combined attack, indicating that the Lab attack alone is more destructive to the output of the fake model; however, the effective interference rate of the combined attack of the present invention is higher than that of the Lab attack alone, indicating that the combined attack of the present invention is more stable and can cover all test samples.

[0101] Based on the above data and images, it can be seen that the combined attack of this invention can effectively disrupt the deepfake effect of the generator network, verifying the effectiveness of the solution. Specifically: Regarding the concealment of perturbations, the extremely low L2 error, the SSIM value close to 1, and the PSNR far exceeding the threshold perceptible to the human eye have quantitatively demonstrated the concealment of the perturbations; from Figure 2 As can be seen intuitively, the adversarial sample in the second column of each group of samples is almost indistinguishable from the original image in the first column in terms of skin color, facial features, and facial expression details. Even when magnified, it is difficult to detect any perturbation traces, which perfectly meets the core requirement of the adversarial attack for concealment and ensures that the attack will not be easily identified due to the perturbation being too obvious.

[0102] In terms of attack effectiveness, the generated image's L2 error is significantly higher than the error of the perturbation itself, while SSIM and PSNR are very low, and the effective interference rate reaches 100%. Figure 2 The attribute editing results also clearly show that after the adversarial sample is DeepFake using the StarGAN model, obvious facial distortion and texture blurring effects are observed. This visual distortion is completely consistent with the quantization results, fully demonstrating that, while maintaining the concealment of the perturbation, the combined attack of this invention successfully destroys the deepfake capability of the generative network, causing the forged image to fail to maintain the original forgery quality and resulting in severe content distortion.

[0103] Regarding alternatives to the facial landmark extractor, in other embodiments, the HRNet model can be replaced by other landmark detection models capable of outputting heatmaps or coordinate information of key facial features (such as eyes, lips, and nose), such as the 2D / 3D landmark detection models provided by the Dlib library or the face_alignment library. Similar attack effects can be achieved as long as sufficient facial structural information can be provided to guide directional perturbations.

[0104] Regarding alternative color spaces, in other embodiments, the perturbation optimization performed in the Lab color space in the first stage can be replaced by applying and optimizing perturbations on the Cb and Cr chromaticity channels of the YCrCb color space. This alternative also relies on the fact that the human eye is relatively insensitive to chromaticity channels to ensure the visual concealment of the perturbation, and its core design logic is completely consistent with the Lab space perturbation.

[0105] Regarding alternative optimizers, in other embodiments, the Adam optimizer used in the first stage can be replaced by other optimization algorithms such as RMSprop; the momentum gradient descent method used in the second stage can be replaced by optimization algorithms with momentum terms such as the Nesterov accelerated gradient method. As long as the perturbation variables can be effectively constrained within the preset budget range during the optimization process, similar optimization results can be achieved.

[0106] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for proactive defense against deepfake attacks, characterized in that, Includes the following steps: S1. Obtain the original face image, generate target attribute labels, and load the pre-trained deep fake generation network and facial key point extractor. S2. Define the perturbation variables for the A and B channels of the Lab color space and the RGB space, and initialize the perturbation variables; S3. Based on the original face image, the adversarial sample candidates composed of the original face image and the initialized perturbation variables, the target attribute label, the generator network, and the facial key point extractor, construct a multimodal loss function including generator network output difference loss, perceptual consistency loss, visual concealment constraint loss, and facial key point destruction loss. S4. Based on the constructed multimodal loss function, the initialized perturbation variables are optimized in stages to obtain the optimized perturbation; the staged optimization includes: the first stage optimizes the perturbation variables in the Lab color space, and the second stage optimizes the facial key points in the RGB space based on the optimization results of the first stage. S5. Add the optimized perturbation to the original face image to generate adversarial samples for defending against deepfakes.

2. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S3, the multimodal loss function adopts a dynamic weight adjustment strategy during the optimization process. The dynamic weights are adaptively adjusted according to the ratio of the current loss to the previous round loss.

3. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S4, the first stage of Lab color space optimization uses the Adam optimizer to iteratively optimize the perturbations of the A and B channels in the Lab space; The second stage of RGB space optimization uses the momentum gradient descent method to iteratively optimize the perturbation of the RGB space and amplify the damage loss of facial key points.

4. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S2, the perturbation budget ε for the perturbation variables of the A and B channels of the Lab color space and the RGB space is set to 0.05, and the perturbation is clipped to the range of [-ε, ε] after each iteration during the optimization process.

5. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, The deepfake active defense adversarial attack method is configured to generate adversarial examples with an L2 error of less than 0.00013, a structural similarity (SSIM) greater than 0.98, and a peak signal-to-noise ratio (PSNR) greater than 45 dB from the original image, in order to meet the visual concealment requirements.

6. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S3, the perceptual consistency loss is obtained by extracting high-level features of the original image and adversarial sample candidates through the VGG network and calculating the mean square error between the two.

7. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S3, the facial key point destruction loss is obtained by extracting the key point heatmap of the adversarial example candidate through the facial key point extractor, and calculating the difference between the heatmap and the zero heatmap.

8. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S2, the Lab color space can be replaced with the YCrCb color space, and perturbation initialization and optimization are performed in the Cb and Cr channels.

9. The deepfake proactive defense adversarial attack method according to claim 1, characterized in that, In step S1, the facial key point extractor uses an HRNet model pre-trained on a 300W dataset, or a key point detection model from the face_alignment library.

10. The deepfake proactive defense adversarial attack method according to claim 3, characterized in that, The Adam optimizer can be replaced by the RMSprop optimizer, and the momentum gradient descent method can be replaced by the Nesterov accelerated gradient method.

Images