Robust adversarial watermark generation method, device and equipment for face deep counterfeiting

Abstract

The invention discloses a face deep counterfeiting-oriented robust countermeasure watermark generation method, device and equipment, and relates to the technical field of deep counterfeiting defense. The method comprises the following steps: inputting an original face image into a forward diffusion process of a denoising diffusion implicit model to obtain a noise potential variable, and executing a reverse denoising process; in each denoising step, a current intermediate image is generated, then the loss of counterfeit output is calculated, gradient information is obtained according to the loss and added to the current intermediate image, and an intermediate noise image is obtained. And a denoising step is executed until a complete denoising process is completed, an intermediate step adversarial image is obtained, a post-processing image is obtained through noise layer processing, then mean square error loss of counterfeit output and structural consistency loss of the intermediate step adversarial image and the original face image are calculated, and a total loss function is obtained through weighted fusion. And based on the total loss function, gradient information is obtained and added to the noise potential variable, and a new round of denoising is carried out. And finally, outputting an intermediate step confrontation image of the last iteration.

Description

Technical Field

[0001] This invention relates to the field of deepfake defense technology, and more specifically, to a robust method, apparatus, and device for generating watermarks against deepfake faces. Background Technology

[0002] With the rapid development of artificial intelligence technology, deepfake technology, especially the editing of facial image attributes and face replacement, has brought serious challenges to information security and privacy protection. To effectively combat unauthorized malicious alteration and prevent the spread of forged content on public platforms such as social networks, the industry urgently needs a defense mechanism that can embed protective mechanisms before image publication, thereby proactively disrupting the output of deepfake models. This proactive defense technology must ensure that the added protective information is invisible to the human eye while interfering with the forgery result, maintaining the normal visual experience of the image.

[0003] Currently, research has explored the application of adversarial attack techniques to proactive defense against deepfake faces. For example, in 2020, Ruiz et al. pioneered the use of adversarial attack algorithms targeting classification models, such as I-FGSM and PGD, to attack deepfake models, causing their output to produce visually noticeable anomalies, thereby achieving the defense objective. Since then, most proactive defense solutions in white-box scenarios have been developed and improved upon based on this.

[0004] However, existing technologies do not adequately consider the robustness of adversarial watermarks. Although some researchers have attempted to improve watermarks' resistance to compression by simulating compressed noise in real social media environments, these methods remain insufficiently robust against image post-processing operations such as Gaussian blur, average blur, and downsampling, which are extremely common in real-world online distribution. This lack of robustness limits the reliability and practicality of existing active defense technologies in complex real-world scenarios. Summary of the Invention

[0005] This invention provides a robust method, apparatus, and device for generating watermarks against deepfake faces, in order to improve at least one of the aforementioned technical problems.

[0006] In a first aspect, the present invention provides a robust anti-watermark generation method for deepfake faces, which includes steps S1 to S5.

[0007] S1. Input the original face image into the forward diffusion process of the denoising diffusion implicit model to obtain the noise latent variables after the preset noise addition step.

[0008] S2. Based on the noise latent variables, perform the inverse denoising process of the denoising diffusion implicit model. In each denoising step: generate the current intermediate image based on the input of the current denoising step. Input the current intermediate image and the original face image into the deepfake model respectively, and then calculate the mean squared error loss between the two outputs of the model. Based on the mean squared error loss, obtain the gradient information of the current intermediate image through the backpropagation mechanism of the deepfake model, and add it to the current intermediate image to obtain the intermediate noisy image.

[0009] S3. Using the intermediate noisy image as input to the previous denoising step, continue denoising until a complete denoising process is completed, obtaining the intermediate step adversarial image. After processing through the noise layer, obtain the post-processed image. Input the post-processed image and the original face image into the deep pseudo-model respectively, and calculate the mean squared error loss between the two outputs of the model, as well as the structural consistency loss between the intermediate step adversarial image and the original face image. Then, perform weighted fusion to obtain the total loss function.

[0010] S4. Based on the total loss function, the gradient information of the noisy latent variables is obtained through the backpropagation mechanism of the deepfake model and added to the noisy latent variables for a new round of denoising.

[0011] S5. When the iteration termination condition is met, output the adversarial image of the intermediate step of the last iteration as a face image with adversarial watermark.

[0012] As a further aspect of the present invention, S1 specifically involves: gradually adding Gaussian noise to the original face image according to a preset noise scheduling strategy, thereby modifying the original face image... Inversion to the first Noise latent variables of the step .in, For preset noise addition steps, For the first The noise latent variables corresponding to each step.

[0013] The forward diffusion process satisfies: In the formula, This represents the number of steps in the positive noise addition process. To add noise step Latent variables at time. This is the cumulative signal retention coefficient. The parameter is Gaussian noise.

[0014] As a further aspect of the present invention, the current intermediate image is a denoising intermediate result containing residual noise during the reverse denoising process.

[0015] The reverse denoising process specifically includes: based on the latent noise variables of the current denoising step and the noise components predicted by the denoising network, first obtaining the estimated clean image corresponding to the current step, and then generating the current intermediate image from the estimated clean image. The image from the previous denoising step has less noise than the image from the current denoising step.

[0016] The deepfake model includes a face attribute editing model and / or a face replacement model.

[0017] As a further aspect of the present invention, when the deepfake model is a face attribute editing model, the forgery output is represented as follows: In the formula, Edit the attribute results. A model for editing facial attributes. Original human face image. The attribute is yet to be edited.

[0018] When the deepfake model is a face replacement model, the forgery output is represented as: In the formula, The result is a face replacement. Face replacement model. Original human face image. The target is a human face image.

[0019] S2 maximizes the forged output of the original face image through a deepfake model. ( The forged output of the current intermediate image and the deepfake model Distance on The goal is to achieve the objective of interfering with the output of the deep forgery model.

[0020] .

[0021] In the formula, This indicates maximization. This indicates an anti-watermarking feature added to the image. Facial attribute editing model and face replacement model A general term.

[0022] S2 mean square error loss Represented as: In the formula This is the current intermediate image. It is an L2 norm. This represents the number of steps in the reverse denoising process.

[0023] As a further aspect of the present invention, based on the mean squared error loss, the gradient information of the current intermediate image is obtained through the backpropagation mechanism of the deep forgery model, and added to the current intermediate image to obtain an intermediate noise image, represented as: In the formula, This is an intermediate noise image. This is the current intermediate image. The preset step size weight. The mean squared error loss is used to calculate the gradient information of the current intermediate image. For gradient. This represents the mean square error loss. This represents the number of steps in the reverse denoising process.

[0024] The preset step size weight is used to control the intensity of the adversarial perturbation injected into the current intermediate image.

[0025] As a further aspect of the present invention, the noise layer processing is used to apply a preset image perturbation to the intermediate step adversarial image after a complete denoising operation is completed, so as to simulate the post-processing operation in the image propagation process. The preset image perturbation includes Gaussian blur.

[0026] The post-processed image after noise layer processing is represented as follows: In the formula, This is a post-processed image. This is the noise layer processing function. For intermediate step adversarial images.

[0027] As a further aspect of the present invention, the mean square error loss of S3 is: In the formula, This represents the mean square error loss. Original human face image. This is a post-processed image. For intermediate step adversarial images. The original face image is forged and output by a deepfake model. To process the forged output of images using a deepfake model. It is an L2 norm.

[0028] The structural consistency loss is: This is the L1 loss function. This is a pixel-by-pixel absolute value operation.

[0029] Total loss function for: In the formula, This is the first weighting coefficient. This is the second weighting coefficient.

[0030] As a further aspect of the present invention, the process of S4 updating the noisy latent variables based on the total loss function is expressed as follows: In the formula, For the updated noise latent variables. Noise latent variables. This provides gradient information for the latent noise variables. For gradient. This is the total loss function.

[0031] The iteration termination condition includes reaching the preset number of iterations.

[0032] If the iteration termination condition is not met, the steps of reverse denoising, intermediate image gradient update, noise layer processing, and noise latent variable update are repeated.

[0033] Secondly, the present invention provides a robust anti-watermark generation device for deepfake faces, which includes an original image acquisition module, an intermediate image module, a total loss module, an iteration module, and an output module.

[0034] The original image acquisition module is used to input the original face image into the forward diffusion process of the denoising diffusion implicit model to obtain the noise latent variables after a preset noise addition step.

[0035] The intermediate image module performs a reverse denoising process using a denoising diffusion implicit model based on noise latent variables. In each denoising step: a current intermediate image is generated based on the input of the current denoising step. The current intermediate image and the original face image are input into the deepfake model, and the mean squared error loss between the model's two outputs is calculated. Based on the mean squared error loss, the gradient information of the current intermediate image is obtained through the backpropagation mechanism of the deepfake model and added to the current intermediate image to obtain the intermediate noisy image.

[0036] The total loss module is used to continue denoising using the intermediate noisy image as input to the previous denoising step until a complete denoising process is completed, resulting in the intermediate step adversarial image. This image is then processed by a noise layer to obtain the post-processed image. The post-processed image and the original face image are input into the deep pseudo-model, and the mean squared error loss between the two outputs of the model, as well as the structural consistency loss between the intermediate step adversarial image and the original face image, are calculated. These are then weighted and fused to obtain the total loss function.

[0037] The iterative module is used to obtain the gradient information of the noisy latent variables based on the total loss function through the backpropagation mechanism of the deepfake model, and add it to the noisy latent variables for a new round of denoising.

[0038] The output module is used to output the adversarial image of the intermediate step of the last iteration as a face image with an adversarial watermark when the iteration termination condition is met.

[0039] Thirdly, the present invention provides a robust anti-watermarking device for deepfake faces, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a robust anti-watermarking method for deepfake faces as described in any paragraph of the first aspect.

[0040] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0041] This invention can combine common white-box defense algorithms to generate adversarial watermark information, thereby achieving effective proactive defense against deepfake faces. By precisely interfering with the forgery process of the trained deepfake model, the forgery output is significantly distorted, which can effectively prevent face images from being maliciously edited, effectively protect the privacy of face images, and greatly improve the security and reliability of face images during the transmission process.

[0042] Meanwhile, this invention fully utilizes the advantages of high fidelity and strong robustness of images generated by the diffusion model. While ensuring the invisibility of the anti-watermark, it significantly improves the invisibility and overall robustness of the anti-watermark. Furthermore, by introducing a noise layer to simulate image processing operations in real-world network scenarios, it specifically and significantly enhances the robustness of the anti-watermark against common image post-processing operations such as JPEG compression, Gaussian blur, average blur, and downsampling. It can still maintain a high defense success rate under the interference of various post-processing operations. Moreover, the generated anti-face image achieves excellent defense effect while also possessing good visual quality, thus balancing defense performance and visual presentation. Attached Figure Description

[0043] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a robust adversarial watermark generation method for deepfake faces. Detailed Implementation

[0045] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention.

[0046] Example 1, please refer to Figure 1The first embodiment of the present invention provides a robust anti-watermarking method for deepfake faces. It can be executed by a robust anti-watermarking device for deepfake faces (hereinafter referred to as: robust anti-watermarking device). Specifically, it is executed by one or more processors in the robust anti-watermarking device to implement steps S1 to S5.

[0047] S1. Input the original face image into the forward diffusion process of the Denoising Diffusion Implicit Models (DDIM) to obtain the noise latent variables after the preset noise addition step, which are used as the initial latent variables for the reverse denoising process.

[0048] In this embodiment, Gaussian noise is gradually added to the original face image according to a preset noise scheduling strategy, thus modifying the original face image. Inversion to the first Noise latent variables of the step And these are used as the initial latent variables in the denoising process. Among them, This is the preset noise addition step (i.e., the final number of steps for positive noise addition). For the first The noise latent variables corresponding to each step.

[0049] Specifically, during the forward pass, the original face image By a noise dispatch A parameterized Markov chain with Gaussian noise added progressively. .

[0050] When the noise step is Latent variables of time Defined as: .

[0051] In the formula, This indicates the number of steps in the forward noise addition process. Indicates from arrive The conditional distribution. For the first Noise latent variables generated step by step. This indicates a normal distribution. This represents the noise scheduling parameters. Represents the identity matrix.

[0052] make , After reparameterization, In the formula, The signal preservation coefficient. This is the cumulative signal retention coefficient. Indicates from step 1 to step 2. The percentage of signals retained cumulatively. For the first The signal preservation coefficient of the step. This is the original human face image. The parameter is Gaussian noise.

[0053] The current intermediate image, intermediate noisy image, and intermediate adversarial image mentioned below are all images containing the face corresponding to the original face image. The inverse denoising process iteratively removes added noise and reconstructs the original face image from the latent noise variables. .

[0054] S2. Using the noise latent variables as initial latent variables, perform the reverse denoising process of the denoising diffusion implicit model. In each step of the denoising process, a current intermediate image is generated based on the input of the current denoising step. Then, the current intermediate image and the original face image are input into the deepfake model respectively to obtain a first fake output and a second fake output, and the mean squared error loss between the two fake outputs is calculated. Based on the mean squared error loss, the gradient information of the current intermediate image is obtained through the backpropagation mechanism of the deepfake model, multiplied by a preset step size weight, and added to the current intermediate image to obtain an intermediate noisy image. The current intermediate image is the intermediate denoising result containing residual noise during the reverse denoising process.

[0055] The reverse denoising process specifically includes: based on the latent noise variables of the current denoising step and the noise components predicted by the denoising network, first obtaining the estimated clean image corresponding to the current step, and then generating the current intermediate image from the estimated clean image. The image from the previous denoising step has less noise than the image from the current denoising step.

[0056] The inverse denoising process removes added noise iteratively from the latent variables of the noise. reconstruction The number of steps in the reverse denoising process is defined as follows: The image generation process of the reverse process is as follows.

[0057] The estimated clean image satisfies: .

[0058] In the formula, This represents the number of steps in the reverse denoising process. For the first The estimated clean image is obtained by reconstructing the noisy latent variables in the step. For the first The noise latent variable of the step. For the first The cumulative signal retention coefficient of the step. For the first The noise component predicted by the time-step denoising network.

[0059] The current intermediate image satisfies: .

[0060] In the formula, For the current intermediate image (i.e., the first one) (Image after denoising step). For the first The cumulative signal retention coefficient of the denoising step. This represents the noise component predicted by the denoising network. The parameter is Gaussian noise. These are the learnable parameters of the denoising network in the denoising diffusion model.

[0061] The deepfake model includes a face attribute editing model and / or a face replacement model. The deepfake model is used to perform attribute editing or face replacement on the face image during the denoising process and the original face image, respectively.

[0062] When the deepfake model is a face attribute editing model, the forged output is represented as: In the formula, Edit the attribute results. A model for editing facial attributes. Original human face image. The attribute is yet to be edited.

[0063] Specifically, given an original human face image A well-trained face attribute editing model and attributes to be edited Then generate the attribute editing results. .

[0064] When the deepfake model is a face replacement model, the forgery output is represented as: In the formula, The result is a face replacement. Face replacement model. Original human face image. The target is a human face image.

[0065] Specifically, a raw human face image The source face is used to replace the target face image, resulting in the face replacement. .

[0066] Face attribute editing model and face replacement model Collectively referred to as deepfake models This embodiment maximizes the forged output of the original face image through the deepfake model. ( The forged output of the current intermediate image and the deepfake model Distance on By maximizing distance The goal is to achieve the objective of interfering with the output of the deep forgery model.

[0067] .

[0068] In the formula, This indicates maximization. This indicates an anti-watermarking feature added to the image.

[0069] Specifically, the mean squared error loss is calculated for each intermediate denoising step after the deep fake model and the original face image after the deep fake model. The formula for calculating the mean squared error loss is as follows: .

[0070] In the formula This is the current intermediate image. It is an L2 norm.

[0071] This embodiment, based on the mean squared error loss, obtains the gradient information of the current intermediate image through the backpropagation mechanism of the deepfake model. This gradient information is multiplied by a preset step size weight and then added to the current intermediate image to obtain an updated intermediate noisy image with gradient information, which is then used to continue the subsequent denoising steps. Specifically, the gradient of the mean squared error loss is calculated using gradient backpropagation technology and superimposed on the intermediate steps of the denoising process.

[0072] .

[0073] In the formula, This is an intermediate noise image. This is the current intermediate image. The preset step size weight (i.e., the guiding parameter for controlling the strength of the anti-watermarking). The mean squared error loss is used to calculate the gradient information of the current intermediate image. For gradient. This represents the mean square error loss.

[0074] The preset step size weight is used to control the strength of the adversarial perturbation injected into the current intermediate image. Preferably, It will be set to 10.

[0075] S3. Using the intermediate noisy image as input to the previous denoising step, continue denoising until a complete denoising process is completed, resulting in the intermediate-step adversarial image. Post-process the intermediate-step adversarial image through a noise layer to obtain the post-processed image. Calculate the mean squared error loss between the forged output of the post-processed image using the deep fake model and the forged output of the original face image using the deep fake model, as well as the structural consistency loss between the intermediate-step adversarial image and the original face image. Assign a first weight coefficient to the mean squared error loss and a second weight coefficient to the structural consistency loss. Then, based on the first and second weight coefficients, perform a weighted fusion of the mean squared error loss and the structural consistency loss to obtain the total loss function.

[0076] It should be noted that the number of steps in the forward diffusion process of the denoising diffusion implicit model... and the number of steps in the reverse denoising process They share the same set of time step indices, differing only in whether they increment or decrement. The quantity mapping relationship from 1 to Quantity mapping relationship from Up to 1. Because The sequence numbers are arranged from largest to smallest, with the smaller sequence number being the previous step of the larger sequence number. Therefore, in this invention, "the previous denoising step" refers to the subsequent further denoising steps.

[0077] Specifically, after completing a full DDIM denoising process, an intermediate-step adversarial image is obtained. Next, this intermediate-step adversarial image is processed through a noise layer to obtain a noise-processed intermediate-step adversarial image (i.e., a post-processed image). Then, the mean squared error loss between the forged output of the post-processed image using the deep fake model and the forged output of the original face image using the deep fake model, as well as the structural consistency loss between the intermediate-step adversarial image and the original face image, are calculated. A first weight coefficient is assigned to the mean squared error loss, and a second weight coefficient is assigned to the structural consistency loss. Based on the first and second weight coefficients, the mean squared error loss and the structural similarity loss are weighted and fused to obtain the total loss function.

[0078] The noise layer processing is used to apply a preset image perturbation to the intermediate step adversarial image after a complete denoising operation, in order to simulate post-processing operations during image propagation. The preset image perturbation includes Gaussian blur.

[0079] The post-processed image after noise layer processing is represented as follows: In the formula, This is a post-processed image. This is the noise layer processing function. For intermediate step adversarial images. Specifically, from Perform DDIM denoising to obtain the intermediate step adversarial image. Then through a Gaussian noise layer The post-processed image after noise layer processing is obtained. .

[0080] Then, the post-processed image after noise layer processing is calculated. The mean squared error distance between the input deep pseudo-model and the original face image after passing through the deep pseudo-model. The mean squared error loss of S3 is: In the formula, This represents the mean square error loss. Original human face image. This is a post-processed image. For intermediate step adversarial images. The original face image is forged and output by a deepfake model. To process the forged output of images using a deepfake model. It is an L2 norm.

[0081] Next, calculate the intermediate step adversarial image. The structural consistency loss between the original face image and the actual face image. Preferably, in this embodiment, the structural consistency loss uses... loss. This is the L1 loss function. This is a pixel-by-pixel absolute value operation.

[0082] Finally, calculate the total loss. In the formula, This is the first weighting coefficient. This is the second weighting coefficient.

[0083] S4. Based on the total loss function, the gradient information of the total loss function with respect to the initial latent variables is obtained through the backpropagation mechanism of the deepfake model, and added to the initial latent variables to carry out a new round of denoising process.

[0084] Specifically, based on the total loss function, the gradient information of the deepfake model is obtained through the backpropagation mechanism of the deepfake model, added to the initial latent variables, and the adversarial latent variables are updated. Then, a new round of denoising process is carried out (i.e., the starting point of the next round of denoising).

[0085] The process of updating noisy latent variables based on the total loss function is represented as follows: In the formula, For the updated noise latent variables. Noise latent variables. This provides gradient information for the latent noise variables. For gradient. This is the total loss function.

[0086] S5. When the iteration termination condition is met, output the intermediate step adversarial image of the last iteration as the final face image with adversarial watermark. The iteration termination condition includes reaching a preset number of iterations. If the iteration termination condition is not reached, repeat the steps of inverse denoising, intermediate image gradient update, noise layer processing, and noise latent variable update.

[0087] Specifically, it checks whether the preset number of iterations has been reached. If not, iterates from S2 to S4. Otherwise, the post-processed image obtained in the last iteration is used as the final face image with anti-watermarking.

[0088] This invention presents an active defense technology solution for deepfake faces in a white-box environment. It interferes with the trained deepfake model, significantly improving its robustness against post-processing techniques such as JPEG compression, Gaussian blur, average blur, and downsampling, while maintaining watermark invisibility and a high defense success rate. This helps protect the privacy of facial images, prevents malicious editing by deepfake models, and enhances the security and reliability of facial images during transmission.

[0089] This invention primarily generates face images with anti-watermarking capabilities. By utilizing a diffusion model to generate high-fidelity and robust images, and combining it with a noise layer, the watermark information is hidden, thereby improving the visual quality and robustness of the generated anti-watermark face images.

[0090] Deepfake proactive defense is a proactive defense technology that protects facial images from being tampered with by deepfake models. Its core idea is to add a certain amount of watermark information to images or videos containing faces before users publish them to public Internet platforms. This disrupts the deepfake results created by malicious users using these facial materials, allowing face observers to easily detect the anomalies of these fake faces and reduce their credibility.

[0091] Robust adversarial watermarking generation method for deepfake faces is an adversarial watermarking technique used to protect face images. This method first converts the original face image into a noisy latent variable representation through a forward denoising process using a diffusion model, and uses this as the starting point for denoising. Subsequently, in the reverse denoising process of the diffusion model, the image is progressively reconstructed. At each step, the mean squared error loss between the current noisy image and the output of the original image after deepfake model processing is calculated. Then, adversarial gradient information is injected into the current generation state through a gradient backpropagation mechanism, guiding the denoising process to generate an adversarial face image with interference capabilities.

[0092] After completing one round of DDIM denoising, an initial adversarial face image is obtained. To further improve the robustness of the adversarial watermark against common post-processing operations, this method introduces a noise layer to simulate blurring operations to perturb the image. It constructs the mean squared error loss between the perturbed image and the original image on the deep forgery model output, as well as the structural consistency loss between the initial adversarial face image and the original face image. These two losses are assigned different weights to obtain the total loss. Based on the total loss, gradient backpropagation is performed again, feeding the updated gradient information back to the initial noise latent variable, and a new round of iterative optimization is conducted. After these multiple rounds of iterative optimization, the final adversarial face image is generated. By utilizing a diffusion model to generate high-fidelity and highly robust adversarial face images, and adding a noise layer to improve the robustness of the generated watermark against JPEG compression, blurring, and downsampling, this method achieves a more robust performance.

[0093] The following is a detailed description of a robust anti-watermark generation method for deepfake faces according to an embodiment of the present invention.

[0094] In this implementation, StarGAN, a deepfake model for attribute editing, and SimSwap, a face-swapping model, were selected for defense. For StarGAN, five attributes—black hair, blonde hair, brown hair, and gender—were targeted. The iteration count was 10. The invention was experimentally evaluated on the CelebA dataset.

[0095] The CelebA dataset is an open-source dataset of face attributes, consisting of 256... The system consists of 256 facial images, containing over 200,000 celebrity images, with each image having 40 attributes. This embodiment selects 50, 100, and 500 facial images for defense, respectively.

[0096] In the evaluation, for the attribute editing model, the Defense Success Rate (DSR) performance measurement method proposed by Ruiz et al. was used. Defense was considered successful when the L2 loss distance between the original face image and the forged output of the deepfake model and the face image with adversarial watermarking and the forged output of the deepfake model was greater than 0.05.

[0097] For the face replacement model, the performance was accurately measured using the Defense Success Rate (DSR) evaluation method proposed by Qu et al. Defense was considered successful when the cosine similarity between the facial feature vector of the forged image and the facial feature vector of the original face image was less than 0.4. Furthermore, Identity Similarity (ID Sim) was also used for evaluation.

[0098] In addition, to ensure that the generated adversarial watermark is invisible, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics are used to measure the quality of the adversarial watermarked face image.

[0099] The success rate and visual indicators of the defense against forgery models of different depths without image processing are shown in Tables 1 and 2.

[0100] Table 1. Defense success rate and visual metrics of the attribute editing model

[0101] Table 2. Defense success rate and visual metrics of face-swapping models

[0102] Experiments show that adversarial face images generated using the diffusion model have good defensive effects, with DSR reaching 100%, visual quality PSNR reaching up to 34dB, and SSIM reaching 0.986.

[0103] Based on this, the robustness of the robust anti-watermark generation method for deepfake faces of the present invention was further tested. The specific experimental setup was as follows: 100 face images were subjected to JPEG compression, Gaussian blur, average blur, and downsampling with different quality factors, and their defense success rate and visual indicators were calculated.

[0104] Table 3 Robustness assessment on the StarGAN model

[0105] In Table 3, QF stands for Quality Factor, KS for Kernel Size, FQ for Factor Quality, and DSR for Defense Success Rate.

[0106] Table 3 shows the defense success rate under attacks with different quality factors, including JPEG compression, Gaussian blur, average blur, and downsampling. Specifically, within the compression range of 90-10 quality factors, a certain defense success rate is still maintained. Within the range of Gaussian kernel size of 1-17, Gaussian blur and average blur still have a defense success rate of over 75%. Within the range of sampling factors of 0.9-0.1, the defense success rate remains above 65%.

[0107] In summary, this invention combines common white-box defense algorithms to generate adversarial watermarking information that protects facial images from deepfakes. This invention fully leverages the high fidelity and robustness of diffusion-based image generation, enhancing the invisibility and robustness of the adversarial watermarking information. While ensuring the invisibility of the adversarial watermark and a high defense success rate, this invention significantly improves the robustness of the adversarial watermark against post-processing techniques such as JPEG compression, Gaussian blur, average blur, and downsampling.

[0108] Example 2: The present invention provides a robust anti-watermark generation device for deepfake faces, which includes an original image acquisition module, an intermediate image module, a total loss module, an iteration module, and an output module.

[0109] The original image acquisition module is used to input the original face image into the forward diffusion process of the denoising diffusion implicit model to obtain the noise latent variables after a preset noise addition step.

[0110] The intermediate image module performs a reverse denoising process using a denoising diffusion implicit model based on noise latent variables. In each denoising step: a current intermediate image is generated based on the input of the current denoising step. The current intermediate image and the original face image are input into the deepfake model, and the mean squared error loss between the model's two outputs is calculated. Based on the mean squared error loss, the gradient information of the current intermediate image is obtained through the backpropagation mechanism of the deepfake model and added to the current intermediate image to obtain the intermediate noisy image.

[0111] The total loss module is used to continue denoising using the intermediate noisy image as input to the previous denoising step until a complete denoising process is completed, resulting in an intermediate step adversarial image. This image is then processed by a noise layer to obtain a post-processed image. The intermediate step adversarial image and the original face image are input into the deep pseudo-model, and the mean squared error loss between the two outputs of the model and the structural consistency loss between the intermediate step adversarial image and the original face image are calculated. These are then weighted and fused to obtain the total loss function.

[0112] The iterative module is used to obtain the gradient information of the noisy latent variables based on the total loss function through the backpropagation mechanism of the deepfake model, and add it to the noisy latent variables for a new round of denoising.

[0113] The output module is used to output the adversarial image of the intermediate step of the last iteration as a face image with an adversarial watermark when the iteration termination condition is met.

[0114] It is understood that the robust anti-watermarking generation device can be an electronic device with computing power, such as a portable laptop computer, desktop computer, server, smartphone, or tablet computer.

[0115] Example 3: This invention provides a robust anti-watermarking device for deepfake faces, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a robust anti-watermarking method for deepfake faces as described in any paragraph of Example 1.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robust anti-watermark generation method for deepfake faces, characterized in that, Include: S1. Input the original face image into the forward diffusion process of the denoising diffusion implicit model to obtain the noise latent variables after the preset noise addition step; S2. Based on the noise latent variables, perform the reverse denoising process of the denoising diffusion implicit model; wherein, in each denoising step: generate the current intermediate image according to the input of the current denoising step; input the current intermediate image and the original face image into the deep fake model respectively, and then calculate the mean square error loss between the two outputs of the model; according to the mean square error loss, obtain the gradient information of the current intermediate image through the backpropagation mechanism of the deep fake model, and add it to the current intermediate image to obtain the intermediate noise image; S3. Continue denoising using the intermediate noisy image as the input of the previous denoising step until a complete denoising process is completed, obtaining the intermediate step adversarial image, which is then processed by the noise layer to obtain the post-processed image. Input the post-processed image and the original face image into the deep pseudo-model respectively, and calculate the mean square error loss between the two outputs of the model, as well as the structural consistency loss between the intermediate step adversarial image and the original face image. Then, perform weighted fusion to obtain the total loss function. S4. Based on the total loss function, the gradient information of the noisy latent variables is obtained through the backpropagation mechanism of the deepfake model and added to the noisy latent variables for a new round of denoising. S5. When the iteration termination condition is met, output the adversarial image of the intermediate step of the last iteration as a face image with adversarial watermark.

2. The robust anti-watermark generation method for deepfake faces according to claim 1, characterized in that, S1 specifically involves: progressively adding Gaussian noise to the original face image according to a preset noise scheduling strategy, thus modifying the original face image... Inversion to the first Noise latent variables of the step ;in, For preset noise addition steps, For the first The noise latent variable corresponding to the step; The forward diffusion process satisfies: In the formula, This represents the number of steps in the positive noise addition process; To add noise step Latent variables at time; The cumulative signal retention coefficient; The parameter is Gaussian noise.

3. The robust anti-watermark generation method for deepfake faces according to claim 1, characterized in that, The current intermediate image is the intermediate result of denoising containing residual noise during the reverse denoising process; The reverse denoising process specifically includes: based on the noise latent variables of the current denoising step and the noise components predicted by the denoising network, firstly, obtaining the estimated clean image corresponding to the current step, and then generating the current intermediate image from the estimated clean image; wherein, the image of the previous denoising step has less noise than the image of the current denoising step. The deepfake model includes a face attribute editing model and / or a face replacement model.

4. A robust anti-watermark generation method for deepfake faces according to claim 3, characterized in that, When the deepfake model is a face attribute editing model, the forged output is represented as: In the formula, Edit the attribute results; A model for editing facial attributes; Original human face image; Attributes to be edited; When the deepfake model is a face replacement model, the forgery output is represented as: In the formula, The result of face replacement; Face replacement model; Original human face image; For the target human face image; S2 maximizes the forged output of the original face image through a deepfake model. ( The forged output of the current intermediate image and the deepfake model Distance on The goal is to interfere with the output of the deepfake model; ； In the formula, Indicates maximization; This indicates an anti-watermarking effect added to the image; Facial attribute editing model and face replacement model A general term; S2 mean square error loss Represented as: In the formula This is the current intermediate image; It is an L2 norm; This represents the number of steps in the reverse denoising process.

5. A robust anti-watermark generation method for deepfake faces according to claim 1, characterized in that, Based on the mean squared error loss, the gradient information of the current intermediate image is obtained through the backpropagation mechanism of the deepfake model and added to the current intermediate image to obtain an intermediate noisy image, represented as: In the formula, This is an intermediate noise image; This is the current intermediate image; Preset step size weights; The mean squared error loss is used to extract gradient information about the current intermediate image. For gradient; This is the mean square error loss; This represents the number of steps in the reverse denoising process; The preset step size weight is used to control the intensity of the adversarial perturbation injected into the current intermediate image.

6. A robust anti-watermark generation method for deepfake faces according to any one of claims 1 to 5, characterized in that, The noise layer processing is used to apply a preset image perturbation to the intermediate step adversarial image after a complete denoising process, in order to simulate the post-processing operation during image propagation; the preset image perturbation includes Gaussian blur. The post-processed image after noise layer processing is represented as follows: In the formula, For post-processing images; This is a noise layer processing function; For intermediate step adversarial images.

7. A robust anti-watermark generation method for deepfake faces according to any one of claims 1 to 5, characterized in that, The mean squared error loss of S3 is: In the formula, This is the mean square error loss; Original human face image; For post-processing images; For intermediate step adversarial images; The original face image is forged using a deepfake model; To process the forged output of images processed by a deepfake model; It is an L2 norm; The structural consistency loss is: ; The loss function is L1. This is a pixel-by-pixel absolute value operation; Total loss function for: In the formula, This is the first weighting coefficient; This is the second weighting coefficient.

8. A robust anti-watermark generation method for deepfake faces according to claim 1, characterized in that, The process of updating the noisy latent variables based on the total loss function in S4 is represented as follows: In the formula, For the updated latent noise variables; Noise latent variables; Gradient information for the latent noise variables; For gradient; This is the total loss function; Iteration termination conditions include reaching a preset number of iterations; If the iteration termination condition is not met, the steps of reverse denoising, intermediate image gradient update, noise layer processing, and noise latent variable update are repeated.

9. A robust anti-watermark generation device for deepfake faces, characterized in that, Used to execute a robust anti-watermark generation method for deepfake faces as described in any one of claims 1 to 8; Robust anti-watermarking device includes: The original image acquisition module is used to input the original face image into the forward diffusion process of the denoising diffusion implicit model to obtain the noise latent variables after a preset noise addition step. The intermediate image module is used to perform the inverse denoising process of the denoising diffusion implicit model based on the noise latent variables. In each denoising step: the current intermediate image is generated according to the input of the current denoising step; the current intermediate image and the original face image are respectively input into the deepfake model, and then the mean squared error loss between the two outputs of the model is calculated; based on the mean squared error loss, the gradient information of the current intermediate image is obtained through the backpropagation mechanism of the deepfake model and added to the current intermediate image to obtain the intermediate noise image. The total loss module is used to continue denoising with the intermediate noisy image as the input of the previous denoising step until a complete denoising process is completed, resulting in the intermediate step adversarial image. After processing by the noise layer, the post-processed image is obtained. The post-processed image and the original face image are respectively input into the deep pseudo-model, and the mean squared error loss between the two outputs of the model and the structural consistency loss between the intermediate step adversarial image and the original face image are calculated. Then, they are weighted and fused to obtain the total loss function. The iterative module is used to obtain the gradient information of the noisy latent variables based on the total loss function through the backpropagation mechanism of the deepfake model, and add it to the noisy latent variables for a new round of denoising. The output module is used to output the adversarial image of the intermediate step of the last iteration as a face image with an adversarial watermark when the iteration termination condition is met.

10. A robust anti-watermark generation device for deepfake faces, characterized in that, It includes a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a robust anti-watermark generation method for deepfake faces as described in any one of claims 1 to 8.

Images