Gradient-based black-box face deepfake adversarial watermarking method, device, medium

Abstract

The application provides a gradient-based black-box face deepfake adversarial watermarking method, device and medium, and relates to the technical field of image processing. The method comprises the following steps: obtaining an original face image, inputting the original face image into a forward noise adding process of a denoising diffusion implicit model, and obtaining a noise latent variable of the first step. The variable is used as an initial denoising starting point for adversarial watermark generation, a reverse denoising process of the denoising diffusion implicit model is performed, a gradient estimation method based on a natural evolution strategy is used in each denoising process, and a loss gradient corresponding to a deepfake model is estimated under a black-box setting. The loss gradient is weighted with a preset guide weight, and then superimposed on an image after denoising of the current denoising step to generate an intermediate adversarial face image of the current denoising step. The intermediate adversarial face images of each denoising step are continuously iterated until all denoising steps are completed, and a final adversarial face image is obtained.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a gradient-based black-box face deepfake anti-watermarking method, device, and medium. Background Technology

[0002] With the rapid development of deepfake technology, the risk of its misuse in creating false information and posing a serious threat to personal privacy and social trust is becoming increasingly prominent. Therefore, developing effective technical means to proactively protect facial images in order to interfere with and disrupt the malicious editing process of deepfake models has become an urgent practical need.

[0003] To address these needs, some proactive defense solutions have been proposed in existing technologies. For example, some researchers have drawn inspiration from adversarial attack strategies in image classification, using algorithms such as I-FGSM and PGD in white-box scenarios to directly attack deepfake models, aiming to produce visually apparent anomalies in their output and thus disrupt the forgery effect. However, such methods typically rely on complete knowledge of the target model's architecture, parameters, and other internal information, a prerequisite that is often difficult to meet in real-world black-box defense scenarios.

[0004] In addition, other studies have attempted to explore defense methods under a black-box setting, such as using deep neural networks as watermark generators to add watermarks to face images. However, these existing black-box methods have shortcomings in defense performance, which may lead to a decrease in the visual quality of the protected image, and training such watermark generators is usually very costly. Summary of the Invention

[0005] This invention provides a gradient-based black-box face deepfake anti-watermarking method, device, and medium to improve at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a gradient-based black-box face deepfake anti-watermarking method, which includes steps S1 to S4.

[0007] S1. Obtain the original face image and input it into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variable of the step. This represents the final number of steps in the positive noise addition process.

[0008] S2, with the first The noise latent variables of each step are used as the initial denoising starting point for adversarial watermark generation. The reverse denoising process of the denoising diffusion implicit model is executed, and a gradient estimation method based on natural evolution strategy is adopted in each denoising process to estimate the loss gradient corresponding to the deep forgery model under black box settings.

[0009] S3. Weight the estimated loss gradient of each step with the preset guiding weights to obtain the adversarial gradient update amount, and add the adversarial gradient update amount to the denoised image of the current denoising step to generate the intermediate adversarial face image of the current denoising step.

[0010] S4. Iterate continuously based on the intermediate adversarial face images from each denoising step until all denoising steps are completed to obtain the final adversarial face image. The final adversarial face image is visually consistent with the original face image and contains an adversarial watermark used to interfere with the output of the black-box deepfake model.

[0011] As a further aspect of the present invention, the original face image is input into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variables of the step specifically include:

[0012] During the forward pass, the original face image Gaussian noise is gradually added through a Markov chain parameterized by noise scheduling.

[0013] Noise addition step Noise latent variables generated at time The distribution is defined as:

[0014] .

[0015] In the formula, This indicates the number of steps in the forward noise addition process. Indicates from arrive The conditional distribution. For the first Noise latent variables generated step by step. This indicates a normal distribution. This represents the noise scheduling parameters. Represents the identity matrix.

[0016] make , After reparameterization, we get: In the formula, The signal preservation coefficient. This is the cumulative signal retention coefficient. Indicates from step 1 to step 2. The percentage of signals retained cumulatively. For the first The signal retention coefficient of the step. This is the original human face image. The parameter is Gaussian noise.

[0017] As a further aspect of the present invention, with the first The noise latent variables of the current denoising step serve as the initial starting point for denoising against watermark generation. A reverse denoising process using the denoising diffusion implicit model is then executed. Specifically, this includes: based on the noise latent variables of the current denoising step and the noise components predicted by the denoising network, first obtaining the estimated clean image corresponding to the current step, and then generating the image of the previous denoising step from the estimated clean image. The image of the previous denoising step has less noise than the image of the current denoising step.

[0018] The estimated clean image satisfies:

[0019] .

[0020] In the formula, This represents the number of steps in the reverse denoising process. For the first The estimated clean image is obtained by reconstructing the noisy latent variables in the step. For the first The noise latent variable of the step. For the first The cumulative signal retention coefficient of the step. For the first The noise component predicted by the time-step denoising network.

[0021] The image from the previous denoising step satisfies:

[0022] .

[0023] In the formula, For the first The image after denoising. For the first The cumulative signal retention coefficient of the denoising step.

[0024] As a further aspect of the present invention, a gradient estimation method based on a natural evolution strategy is employed to estimate the loss gradient corresponding to the deepfake model under a black-box setting, specifically including:

[0025] By sampling several Gaussian random directions and calculating the output of the black-box model on adversarial face images, the mean squared error loss gradient of the fake output and the fake output of the original face image is estimated multiple times.

[0026] The steps for gradient estimation in each iteration are as follows:

[0027] .

[0028] .

[0029] .

[0030] .

[0031] .

[0032] In the formula, This indicates that a random sampling action will be performed. It is a random noise vector sampled from a standard multivariate normal distribution, with the same dimension as the original face image. This indicates a normal distribution. Represents the identity matrix. This indicates that the original face image is along... The result is obtained by adding a positive anti-watermark to the direction and then feeding it into the forgery model. This is a deepfake model. This is the original human face image. This represents the adversarial watermarking scale parameter, used to control the size of the random exploration step. This represents the original face image along... The result is obtained by adding a negative adversarial watermark to the direction and then feeding it into the forgery model. Indicates in The change in adversarial loss caused by positive and negative adversarial watermarking in the direction. This represents the adversarial loss function. This represents the estimated value of the total gradient.

[0033] As a further aspect of the present invention, S3 specifically includes:

[0034] The loss gradient estimated in the current step With guiding weight Multiply them to obtain the adversarial gradient update for the current step.

[0035] The adversarial gradient update is superimposed on the image after the current denoising step to obtain the updated intermediate adversarial face image.

[0036] The updated intermediate adversarial face image satisfies:

[0037] .

[0038] In the formula, To add the adversarial gradient update amount after the th The intermediate adversarial face image of the step. For the first The image after denoising. Indicates the first The loss gradient estimate for each step.

[0039] As a further aspect of the present invention, the deepfake model includes at least one of an attribute editing model, a face replacement model, and an identity-preserving generation model. The adversarial watermarking interferes with the output of the deepfake model by increasing the difference between the outputs of the original face image and the adversarial face image after processing by the deepfake model.

[0040] The target of the interference is:

[0041] .

[0042] In the formula, This indicates maximization. This is a distance metric function. This is a deepfake model. ( ) is the forged output of the original face image after being processed by the deepfake model. To combat the forged output of face images processed by deepfake models. This is the original human face image. To combat watermarks.

[0043] As a further aspect of the present invention, the final adversarial face image is used to interfere with at least one deepfake operation among attribute editing, face replacement, and identity-preserving generation. Specifically, the attribute editing operation generates a forgery result by modifying specified attributes of the original face image. The face replacement operation replaces the original face image as a source face onto a target face to generate a forgery result. The identity-preserving generation operation inputs the original face image into an identity-preserving model to generate a forgery result that preserves identity features.

[0044] Secondly, the present invention provides a gradient-based black-box face deepfake anti-watermarking device, which includes a forward noise addition module, a reverse noise reduction module, an overlay module and an output module.

[0045] The forward noise addition module is used to acquire the original face image and input it into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variable of the step.

[0046] Inverse denoising module, used for the first The noise latent variables of each step are used as the initial denoising starting point for adversarial watermark generation. The reverse denoising process of the denoising diffusion implicit model is executed, and a gradient estimation method based on natural evolution strategy is adopted in each denoising process to estimate the loss gradient corresponding to the deep forgery model under black box settings.

[0047] The overlay module is used to weight the estimated loss gradient of each step with the preset guiding weights to obtain the adversarial gradient update amount, and then overlay the adversarial gradient update amount onto the denoised image of the current denoising step to generate the intermediate adversarial face image of the current denoising step.

[0048] The output module iterates continuously based on the intermediate adversarial face images from each denoising step until all denoising steps are completed, yielding the final adversarial face image. The final adversarial face image is visually identical to the original face image and contains an adversarial watermark designed to interfere with the output of the black-box depth forgery model.

[0049] Thirdly, the present invention provides a gradient-based black-box deepfake face anti-watermarking device, comprising a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a gradient-based black-box deepfake face anti-watermarking method as described in any paragraph of the first aspect.

[0050] Fourthly, the present invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a gradient-based black-box face deepfake anti-watermarking method as described in any paragraph of the first aspect.

[0051] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0052] This invention achieves efficient defense against deepfake models under a black-box setting by introducing a diffusion model and a gradient estimation method based on a natural evolution strategy. This method can effectively generate adversarial face images with high visual fidelity without knowing the model's internal structure. The embedded adversarial watermark remains imperceptible to the human eye while significantly interfering with the output of various types of forgery models, including attribute editing, face replacement, and identity preservation, greatly enhancing resistance to malicious tampering. Furthermore, this invention fully utilizes the generative capabilities of a pre-trained diffusion model, achieving efficient defense without additional training. It possesses good versatility and practicality, providing an effective technical means for the security and privacy protection of face images during transmission. Attached Figure Description

[0053] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating a gradient-based black-box face deepfake anti-watermarking method. Detailed Implementation

[0055] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention.

[0056] Example 1, please refer to Figure 1 The first embodiment of the present invention provides a gradient-based black-box deep face spoofing anti-watermarking method, which can be executed by a gradient-based black-box deep face spoofing anti-watermarking device. Specifically, it is executed by one or more processors in the gradient-based black-box deep face spoofing anti-watermarking device to implement steps S1 to S4.

[0057] S1. Obtain the original face image and input it into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variable of the step. This is the final step in the forward noise addition process. Specifically, the original face image is fed into the forward process of the Denoising Diffusion Implicit Models (DDIM), where noise is added progressively to generate the final noise level. The noise latent variable is used as the initial denoising starting point for subsequent adversarial example generation. Here, the noise latent variable is the noisy image.

[0058] The original face image is input into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variables of the step specifically include:

[0059] During the forward pass, the original face image Gaussian noise is gradually added through a Markov chain parameterized by noise scheduling.

[0060] Noise addition step Noise latent variables generated at time The distribution is defined as:

[0061] .

[0062] In the formula, This indicates the number of steps in the forward noise addition process. Indicates from arrive The conditional distribution. For the first Noise latent variables generated step by step. This represents a normal distribution. This represents the noise scheduling parameters. Represents the identity matrix.

[0063] make , After reparameterization, we get: In the formula, The signal preservation coefficient. This is the cumulative signal retention coefficient. Indicates from step 1 to step 2. The percentage of signals retained cumulatively. For the first The signal retention coefficient of the step. This is the original human face image. The parameter is Gaussian noise.

[0064] Specifically, the original face image is fed into the forward process of DDIM to add noise, thus obtaining the original face image. Inversion to the first The noise latent variables of the step. During the forward pass, the original face image... By a noise dispatch A parameterized Markov chain with Gaussian noise added progressively. .

[0065] Noise addition step Noise latent variables at time Defined as: In the formula, Indicates from arrive The conditional distribution. Represents the identity matrix.

[0066] make , After reparameterization, .

[0067] S2, with the first The noisy latent variable is used as the initial denoising starting point for adversarial watermark generation. A reverse denoising process using a denoising diffusion implicit model is executed. In each denoising step, a gradient estimation method based on a natural evolution strategy is employed to estimate the loss gradient of the deepfake model under a black-box setting. Specifically, starting with this noisy latent variable, a reverse denoising process using a denoising diffusion implicit model is executed. In each denoising step, to effectively estimate the gradient of the deepfake model under a black-box setting, a gradient estimation method based on a natural evolution strategy is used.

[0068] With the first The noise latent variables of the current denoising step serve as the initial starting point for denoising against watermark generation. A reverse denoising process using the denoising diffusion implicit model is then executed. Specifically, this includes: based on the noise latent variables of the current denoising step and the noise components predicted by the denoising network, first obtaining the estimated clean image corresponding to the current step, and then generating the image of the previous denoising step from the estimated clean image. The image of the previous denoising step has less noise than the image of the current denoising step.

[0069] The estimated clean image satisfies:

[0070] .

[0071] In the formula, This represents the number of steps in the reverse denoising process. For the first The estimated clean image is obtained by reconstructing the noisy latent variables in the step. For the first The noise latent variable of the step. For the first The cumulative signal retention coefficient of the step. For the first The noise component predicted by the time-step denoising network.

[0072] The image from the previous denoising step satisfies:

[0073] .

[0074] In the formula, For the first The image after denoising in step (i.e., the first step) -1 step noise latent variables). For the first The cumulative signal retention coefficient of the denoising step.

[0075] Specifically, the inverse denoising process iteratively removes added noise and reconstructs the original face image from the latent variables of the noise. The preferred number of steps in the forward noise addition process. and the number of steps in the reverse denoising process They share the same set of time step indices, differing only in whether they increment or decrement. The quantity mapping relationship from 1 to ; Quantity mapping relationship from Up to 1;

[0076] Deepfake models include at least one of attribute editing models, face replacement models, and identity-preserving generation models. Adversarial watermarking interferes with the output of the deepfake model by increasing the difference between the outputs of the original face image and the adversarial face image after processing by the deepfake model. Specifically, it maximizes the forged output of the original face image after processing by the deepfake model. ( ) and the forged output of adversarial face images processed by deepfake models ( distance By maximizing distance The goal is to achieve the objective of interfering with the output of the deep forgery model.

[0077] The target of the interference is:

[0078] .

[0079] In the formula, This indicates maximization. This is a distance metric function. This is a deepfake model. ( ) is the forged output of the original face image after being processed by the deepfake model. To combat the forged output of face images processed by deepfake models. This is the original human face image. To combat watermarks. This indicates an adversarial face image after adding an adversarial watermark.

[0080] Deepfake models are used to perform attribute editing, face replacement, and identity-preserving regeneration on both adversarial face images and original face images.

[0081] The attribute editing model is used to perform attribute editing operations on faces: given an original facial image. and a trained face attribute editing model By modifying specific attributes Obtain the forged attribute vector generated by the forgery. The entire process of forging attribute editing is represented as follows: .

[0082] Face replacement models are used to perform face replacement operations: given a well-trained face replacement model , take an original facial image As the source face, replace it with the target face. The above yields the forged face vector. The entire process of forging a face replacement operation is represented as follows: .

[0083] Identity-preserving generative models are used to preserve identity forgery operations: given a well-trained identity-preserving model , take an original facial image As the source face, input model The identity preservation vector generated by forgery is obtained. The entire process of identity forgery is represented as follows: .

[0084] A gradient estimation method based on a natural evolution strategy is employed to estimate the loss gradient of a deepfake model under a black-box setting. Specifically, this includes:

[0085] By sampling several Gaussian random directions and calculating the output of the black-box model on adversarial face images, the mean squared error loss gradient of the fake output and the fake output of the original face image is estimated multiple times.

[0086] The steps for gradient estimation in each iteration are as follows:

[0087] .

[0088] .

[0089] .

[0090] .

[0091] .

[0092] In the formula, This indicates that a random sampling action will be performed. It is a random noise vector sampled from a standard multivariate normal distribution, with the same dimension as the original face image. This represents a normal distribution. Represents the identity matrix. This represents the original face image along... The result is obtained by adding a positive anti-watermark to the direction and then feeding it into the forgery model. This is a deepfake model. This is the original human face image. This represents the adversarial watermarking scale parameter, used to control the size of the random exploration step. This represents the original face image along... The result is obtained by adding a negative adversarial watermark to the direction and then feeding it into the forgery model. Indicates in The change in adversarial loss caused by positive and negative adversarial watermarking in the direction. This represents the adversarial loss function. This represents the estimated value of the total gradient.

[0093] In this embodiment, the mean squared error loss gradient of each intermediate step of the denoising process after the deep fake model and the original face image after the deep fake model are estimated. Specifically, combining a gradient estimation method based on a natural evolution strategy, the mean squared error loss gradient of the fake output and the fake output of the original face image is estimated by sampling several Gaussian random directions and calculating the output of the black-box model on the adversarial face image. Gradient estimation is performed through multiple iterations, preferably 8 iterations. Set it to 0.1.

[0094] S3. The estimated loss gradient at each step is weighted with preset guiding weights to obtain the adversarial gradient update amount. This adversarial gradient update amount is then superimposed on the denoised image of the current denoising step to generate the intermediate adversarial face image for the current denoising step. Specifically, based on the estimated mean squared error loss gradient, it is multiplied with preset guiding weights to obtain the weighted adversarial gradient update amount, which is then superimposed on the denoised image of the current step to generate the intermediate face image with adversarial information corresponding to that step.

[0095] In this embodiment, the loss gradient estimated in the current step is used. With guiding weight Multiply the values ​​to obtain the adversarial gradient update for the current step. Then, add the adversarial gradient update to the denoised image of the current denoising step to obtain the updated intermediate adversarial face image.

[0096] The updated intermediate adversarial face image satisfies:

[0097] .

[0098] In the formula, To add the adversarial gradient update amount after the th The intermediate adversarial face image of the step. For the first The image after denoising. Indicates the first The loss gradient estimate for each step. Guiding weights. Used to control the strength of anti-watermarking.

[0099] Specifically, based on the mean squared error loss gradient and the guiding weights... The product of these factors is superimposed on the denoised image at each step to generate the adversarial face image for the current step. In this embodiment, the gradient of the deep pseudo-model is simulated based on a gradient-based natural evolution strategy, and then superimposed on the intermediate steps of denoising. Each intermediate step of denoising is as follows: Preferred, Set it to 0.3.

[0100] S4. Iterate continuously based on the intermediate adversarial face images from each denoising step until all denoising steps are completed to obtain the final adversarial face image. The final adversarial face image is visually consistent with the original face image and embeds an adversarial watermark to interfere with the output of the black-box deepfake model. Specifically, after completing all denoising steps in sequence, the final generated adversarial face image is output. The final generated image is visually highly consistent with the original image, and the embedded adversarial watermark can effectively interfere with the normal output of the black-box deepfake model.

[0101] Specifically, after the entire denoising process is completed, the final generated adversarial face image is output, which can effectively interfere with the normal output of the black-box deepfake model.

[0102] This invention presents an anti-watermarking generation scheme in a black-box environment. It interferes with the forgery process of a trained deepfake model, causing significant distortion in the forged output. While ensuring the invisibility of the anti-watermark, it significantly improves the defense effect against watermarking.

[0103] This invention generates face images containing adversarial watermarking information. By introducing a diffusion model and estimating gradients based on a natural evolution strategy, it ensures effective protection against different types of forgery models when adding adversarial watermarks. This approach not only achieves concealment of the adversarial watermark and enhances the defense capabilities of adversarial face images against models such as attribute editing, face replacement, and identity preservation, but also provides strong support for the security and privacy protection of face images.

[0104] Specifically, the original face image is fed into the forward process of DDIM to obtain the first... The noise latent variable is used as the initial denoising starting point for subsequent adversarial example generation. Starting with this noise latent variable, the reverse denoising process of DDIM is executed. In each denoising step, the mean squared error loss gradient estimated using a gradient estimation method based on a natural evolution strategy is multiplied by a preset guiding weight to obtain a weighted adversarial gradient update. This update is then superimposed on the denoised image at the current time step to generate the intermediate face image corresponding to that step's adversarial information. The product of the mean squared error loss gradient and the guiding weight is superimposed on the denoised image at each step to generate the adversarial face image at the current time step. After completing all denoising steps, the final generated adversarial face image is output.

[0105] The effectiveness of the technical solution of this invention is illustrated below with a set of experimental data.

[0106] This paper selects StarGAN (a deepfake model for attribute editing), SimSwap (a face-swapping model), and Arc2Face (an identity-preserving generation model) for defense. For the attribute editing model StarGAN, this embodiment selects five attributes for defense: black hair, blonde hair, brown hair, and gender. The invention is experimentally evaluated on the CelebA dataset.

[0107] The CelebA dataset is an open-source dataset of face attributes, consisting of 256... The system consists of 256 facial images, encompassing over 200,000 celebrity images, with each image having 40 attributes. This case study selects 100 facial images for defense.

[0108] In this evaluation experiment, a method for accurately measuring the Defense Success Rate (DSR) performance was proposed. For the attribute editing model, defense was considered successful when the L2 loss distance between the forged output of the original face image and the forged output of the face image with adversarial watermarking was greater than 0.05. For the face-swapping model and the identity-preserving generation model, defense was considered successful when the identity similarity between the forged output of the original face image and the forged output of the face image with adversarial watermarking was less than 0.4.

[0109] Meanwhile, to ensure that the generated adversarial watermark is invisible, the adversarial face image quality is measured by Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM).

[0110] The success rate and visual indicators of defense under different models in the black-box scenario are shown in Tables 1 and 2. Adding adversarial watermarks to the original face image can successfully interfere with the forgery output of the deepfake model.

[0111] Table 1. Defense success rate and visual metrics based on the attribute editing model.

[0112]

[0113] Table 2. Defense success rate and visual metrics based on face-swapping and identity preservation models.

[0114]

[0115] Experiments show that the adversarial face images generated by the gradient-based black-box face deepfake adversarial watermarking method in this embodiment have good defense performance. It achieves a defense success rate of 95.5% on StarGAN, and 72% and 100% on SimSwap and Arc2Face models, respectively. The image visual quality remains around 32dB, achieving both the invisibility of the adversarial watermark and the effectiveness of the defense.

[0116] In summary, this invention constructs adversarial watermarking information to protect facial images from deepfake model tampering, achieving proactive defense against facial images and improving their security. This invention fully leverages the high fidelity of images generated by diffusion models, significantly enhancing the anti-watermarking performance. It not only defends against attribute editing models but also against face-swapping models and identity-preserving generation based on diffusion models. Furthermore, this invention utilizes a pre-trained diffusion model combined with a gradient estimation method based on a natural evolution strategy to protect facial images, eliminating the need for additional model training.

[0117] Example 2: The present invention provides a gradient-based black-box face deepfake anti-watermarking device, which includes a forward noise addition module, a reverse noise reduction module, an overlay module and an output module.

[0118] The forward noise addition module is used to acquire the original face image and input it into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variable of the step.

[0119] Inverse denoising module, used for the first The noise latent variables of each step are used as the initial denoising starting point for adversarial watermark generation. The reverse denoising process of the denoising diffusion implicit model is executed, and a gradient estimation method based on natural evolution strategy is adopted in each denoising process to estimate the loss gradient corresponding to the deep forgery model under black box settings.

[0120] The overlay module is used to weight the estimated loss gradient of each step with the preset guiding weights to obtain the adversarial gradient update amount, and then overlay the adversarial gradient update amount onto the denoised image of the current denoising step to generate the intermediate adversarial face image of the current denoising step.

[0121] The output module iterates continuously based on the intermediate adversarial face images from each denoising step until all denoising steps are completed, yielding the final adversarial face image. The final adversarial face image is visually identical to the original face image and contains an adversarial watermark designed to interfere with the output of the black-box depth forgery model.

[0122] Example 3: This invention provides a gradient-based black-box deepfake face anti-watermarking device, which includes a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a gradient-based black-box deepfake face anti-watermarking method as described in any paragraph of Example 1.

[0123] It is understood that the gradient-based black-box face deepfake anti-watermarking device can be an electronic device with computing power, such as a portable laptop, desktop computer, server, smartphone, or tablet computer.

[0124] Example 4: This invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform a gradient-based black-box face deepfake anti-watermarking method as described in any paragraph of Example 1.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A gradient-based black-box face deepfake anti-watermarking method, characterized in that, Include: S1. Obtain the original face image and input it into the forward noise addition process of the denoising diffusion implicit model to obtain the first... The noise latent variable of the step; This represents the final number of steps for positive noise addition. S2, with the first The noise latent variables of each step are used as the initial denoising starting point for adversarial watermark generation. The reverse denoising process of the denoising diffusion implicit model is executed, and a gradient estimation method based on natural evolution strategy is adopted in each denoising process to estimate the loss gradient corresponding to the deep forgery model under black box settings. S3. Weight the estimated loss gradient of each step with the preset guiding weight to obtain the adversarial gradient update amount, and add the adversarial gradient update amount to the denoised image of the current denoising step to generate the intermediate adversarial face image of the current denoising step. S4. Iterate continuously based on the intermediate adversarial face images from each denoising step until all denoising steps are completed to obtain the final adversarial face image; the final adversarial face image is visually consistent with the original face image and is embedded with an adversarial watermark used to interfere with the output of the black-box deepfake model. S1 specifically includes: During the forward pass, the original face image Gaussian noise is added incrementally through a Markov chain parameterized by noise scheduling; Noise addition step Noise latent variables generated at time The distribution is defined as: ； In the formula, Indicates the number of steps in the forward noise addition process; Indicates from arrive Conditional distribution; For the first The noise latent variables generated in the first step; Represents a normal distribution; Indicates noise scheduling parameters; Represents the identity matrix; make , After reparameterization, we get: In the formula, The signal preservation coefficient; The cumulative signal retention coefficient; Indicates from step 1 to step 2. The percentage of signals retained cumulatively per step; For the first The signal preservation coefficient of the step; Original human face image; The parameter is Gaussian noise.

2. The gradient-based black-box face deepfake anti-watermarking method according to claim 1, characterized in that, With the first The noise latent variables of the current denoising step serve as the initial denoising starting point for adversarial watermark generation. The reverse denoising process of the denoising diffusion implicit model is executed, which specifically includes: based on the noise latent variables of the current denoising step and the noise components predicted by the denoising network, the estimated clean image corresponding to the current step is first obtained, and then the image of the previous denoising step is generated from the estimated clean image; wherein, the image of the previous denoising step has less noise than the image of the current denoising step. The estimated clean image satisfies: ； In the formula, This represents the number of steps in the reverse denoising process; For the first The estimated clean image is obtained by reconstructing the noisy latent variables in the first step; For the first The noise latent variable of the step; For the first The cumulative signal retention coefficient of the step; For the first The noise components predicted by the time-to-time denoising network; The image from the previous denoising step satisfies: ； In the formula, For the first The image after denoising steps; For the first The cumulative signal retention coefficient of the denoising step.

3. The gradient-based black-box face deepfake anti-watermarking method according to claim 1, characterized in that, A gradient estimation method based on a natural evolution strategy is employed to estimate the loss gradient of a deepfake model under a black-box setting. Specifically, this includes: By sampling several Gaussian random directions and calculating the output of the black-box model on adversarial face images, the mean squared error loss gradient of the fake output and the fake output of the original face image is estimated multiple times. The steps for gradient estimation in each iteration are as follows: ； ； ； ； ； In the formula, This indicates that a random selection action will be performed; It is a random noise vector sampled from a standard multivariate normal distribution, with the same dimension as the original face image; Represents a normal distribution; Represents the identity matrix; This represents the original face image along... The result obtained by adding a positive adversarial watermark to the direction and then feeding it into the forgery model; For deepfake models; Original human face image; This represents the adversarial watermarking scale parameter, used to control the size of the random exploration step. This represents the original face image along... The result obtained by adding a negative adversarial watermark to the direction and then feeding it into the forgery model; Indicates in The change in adversarial loss caused by positive and negative adversarial watermarking in the direction; Represents the adversarial loss function; This represents the estimated value of the total gradient.

4. The gradient-based black-box face deepfake anti-watermarking method according to claim 1, characterized in that, S3 specifically includes: The loss gradient estimated in the current step With guiding weight Multiply them to obtain the adversarial gradient update amount for the current step; The adversarial gradient update is superimposed on the image after the current denoising step to obtain the updated intermediate adversarial face image; The updated intermediate adversarial face image satisfies: ； In the formula, To add the adversarial gradient update amount after the th The intermediate adversarial face image of the step; For the first The image after denoising steps; Indicates the first The loss gradient estimate for each step.

5. A gradient-based black-box face deepfake anti-watermarking method according to any one of claims 1 to 4, characterized in that, The deepfake model includes at least one of the attribute editing model, face replacement model, and identity-preserving generation model; the adversarial watermarking interferes with the output of the deepfake model by increasing the difference between the output of the original face image and the adversarial face image after processing by the deepfake model. The target of the interference is: ； In the formula, Indicates maximization; It is a distance metric function; For deepfake models; ( The output of the deepfake model is the original face image. To combat the forged output of face images through deepfake models; Original human face image; To combat watermarks.

6. A gradient-based black-box face deepfake anti-watermarking method according to any one of claims 1 to 4, characterized in that, The final adversarial face image is used to interfere with at least one deepfake operation in attribute editing, face replacement, and identity preservation generation; wherein, the attribute editing operation generates a forgery result by modifying specified attributes of the original face image; the face replacement operation replaces the original face image as the source face onto the target face to generate a forgery result; and the identity preservation generation operation inputs the original face image into an identity preservation model to generate a forgery result that preserves identity features.

7. A gradient-based black-box face deepfake anti-watermarking device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a gradient-based black-box face deepfake anti-watermarking method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform a gradient-based black-box face deepfake anti-watermarking method as described in any one of claims 1 to 6.

Images