Active defense method and device for deep face forgery based on discrete wavelet transform

Abstract

The present invention relates to a method and device for active defense against deep fake face based on discrete wavelet transform, and to the field of image processing technology. The method comprises: using a deep fake model to edit the attributes of the original and disturbed images, calculating the mean square error loss and structural similarity loss between the two, and merging them into a total loss function after assigning different weights; obtaining gradient information based on the back propagation of the loss function, determining the optimal direction of the disturbance application, and forming an intermediate disturbance value; embedding low-frequency and high-frequency components into the corresponding components of the disturbed image through discrete wavelet transform, and then reconstructing the image through inverse discrete wavelet transform; checking whether the number of iterations has been reached, if not, introducing a JPEG noise layer to continue iteratively adjusting the disturbance; if so, outputting a face image containing invisible disturbance information. The present invention makes full use of the robustness in the wavelet domain, and improves the resistance of disturbance to JPEG compression while ensuring that the disturbance is invisible.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for active defense against deep forgery of human faces based on discrete wavelet transform. Background Art

[0002] In 2020, Ruiz et al. first proposed using adversarial attacks for active defense against deep fakes of faces. They used adversarial attack algorithms on classification models, such as I-FSGM and PGD algorithms, to destroy the forged results of deep fake models, making the forged results visible to the naked eye.

[0003] Active face deepfake defense technology is a technology that actively embeds protection information into face images. The core idea is to add specific adversarial perturbations to face videos or images before they are published on the Internet. These added perturbations can protect face multimedia content after it is published, preventing it from being harmed by deepfakes.

[0004] Most of the existing active defense solutions in white-box scenarios are improved on the basis of Ruiz et al.’s solution, and most of the perturbations are added in the pixel domain, without fully considering the robustness of the generated perturbations. Some researchers have also proposed modeling the real social media environment to increase the robustness of the generated perturbations against compression, but have not explored the possibility of adding adversarial perturbations to other domains, such as the wavelet domain, to achieve active defense against deep face forgeries. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes an active defense method and device for deep fake of faces based on discrete wavelet transform, which interferes with the forgery process of a trained deep fake model, causing obvious distortion of the forged output, thereby preventing malicious editing of the deep fake model, thereby improving the security and reliability of facial images during the dissemination process.

[0006] The specific plan is as follows:

[0007] On the one hand, the active defense method for deep face forgery based on discrete wavelet transform includes the following steps:

[0008] S101, obtaining an original face image, and initializing a perturbed face image based on the original face image;

[0009] S102, respectively editing the attributes of the perturbed face image and the original face image through the deep fake model to obtain a first forged output and a second forged output; calculating the mean square error loss between the first forged output and the second forged output, and calculating the structural similarity loss between the perturbed face image and the original face image, assigning a first weight coefficient to the mean square error loss, assigning a second weight coefficient to the structural similarity loss, and weightedly fusing the mean square error loss and the structural similarity loss based on the first weight coefficient and the second weight coefficient to obtain a total loss function;

[0010] S103, taking the perturbed face image as input, based on the total loss function, obtaining the gradient information of the deep fake model through the back propagation mechanism of the deep fake model, calculating the sign direction of the gradient based on the gradient information, determining the optimal direction of the perturbation through the sign direction of the gradient, and multiplying the sign direction of the gradient by the set perturbation amplitude to obtain an intermediate perturbation value;

[0011] S104, performing discrete wavelet transform on the intermediate disturbance value to obtain a low-frequency component and a high-frequency component of the intermediate disturbance value; performing discrete wavelet transform on the facial image containing disturbance to obtain a low-frequency component and a high-frequency component of the facial image containing disturbance; embedding the low-frequency component and the high-frequency component of the intermediate disturbance value into the low-frequency component and the high-frequency component of the facial image containing disturbance respectively to embed disturbance information, and then reconstructing the facial image embedded with the disturbance information by inverse discrete wavelet transform to obtain a facial image containing the intermediate disturbance value;

[0012] S105, determining whether the preset number of iterations has been reached; if not, iteratively executing S102-S104, and introducing a JPEG noise layer, gradually adjusting the amplitude and direction of the disturbance during the iteration process; if reached, outputting a face image containing an intermediate disturbance value as the final face image containing invisible disturbance information.

[0013] Furthermore, the deep fake model is used to edit the attributes of the disturbed face image and the original face image, specifically including:

[0014] Given an original face image , a trained deep fake model and can modify specific properties , generate fake images , the formula is as follows:

[0015] ;

[0016] Maximize the fake output of the original face image through the deep fake model And the forged output of the deep forgery model for the perturbed face image The goal of interfering with the output of the deep fake model is achieved by maximizing the distance D. The formula is as follows:

[0017] ;

[0018] in, Represents the original face image An invisible disturbance added to The threshold value representing the size of the constrained perturbation is used to ensure that the added perturbation is imperceptible to the human eye; express The infinite norm of .

[0019] Furthermore, in S102, the mean square error loss The calculation formula is as follows:

[0020] ;

[0021] in, Represents the total number of elements in the tensor, and Represent the original input value and the generated value of the deep fake model respectively; by calculating and The mean square error between the predicted values ​​and the original input values ​​is used to measure the difference between the model predictions and the original input values, guiding the generation and optimization of disturbances.

[0022] Furthermore, in S102, the structural similarity loss The calculation formula is as follows:

[0023] ;

[0024] in, Represents the original face image The mean of Generate fake images The mean of Represents the original face image The variance of Generate fake images The variance of is the covariance between the two face images; and is a constant.

[0025] Furthermore, in S102, the formula of the total loss function is as follows:

[0026] ;

[0027] in, Represents the weight parameter for balancing the mean square error loss; The weight parameter representing the structural similarity loss; represents the mean squared error loss, which is used to measure the pixel-level difference between the deepforged image of the original face and the deepforged image containing the perturbed face; Represents the structural similarity loss, which is used to evaluate the similarity between the perturbed face and the original face image in terms of structure, brightness, and contrast.

[0028] Furthermore, in S103, the calculation formula of the intermediate disturbance value is as follows:

[0029] ;

[0030] in, represents the disturbance amplitude; represents a deep fake model; Represents the output result of the original face image after being forged by the deep fake model; Represents a perturbed face image The output result after being forged by the deep fake model, Represents the loss function About face images The gradient of Represents a symbolic function.

[0031] Furthermore, the S104 specifically includes:

[0032] Perform a one-dimensional discrete wavelet transform on each row of the disturbed face image, and then perform a one-dimensional discrete wavelet transform on each column to obtain the low-frequency component and high-frequency component of the disturbed face image, wherein the low-frequency component and high-frequency component of the face image include a low-frequency-low-frequency component , high-frequency-low-frequency component , low-frequency-high-frequency component and high-frequency-high-frequency components ;

[0033] The low-frequency-low-frequency component It is used to represent the rough outline of the perturbed face image. The calculation formula is as follows:

[0034] ;

[0035] The high-frequency-low-frequency component Used to represent the edge information in the horizontal direction. The calculation formula is as follows:

[0036] ;

[0037] Low-frequency-high-frequency component Used to represent the edge information in the vertical direction. The calculation formula is as follows:

[0038] ;

[0039] The high-frequency-high-frequency component Used to represent the details and noise in the image, the calculation formula is as follows:

[0040] ;

[0041] Wherein, m is the row coordinate of the input image, which is used to indicate the position of the image in the vertical direction; n is the column coordinate of the input image, which is used to indicate the position of the image in the horizontal direction; A value representing the pixel value of the image; Represents low-pass filtering, which is used to extract low-frequency components; Represents high-pass filtering, which is used to extract high-frequency components; and Respectively represent the row and column coordinates of the sub-image generated after wavelet transformation;

[0042] The intermediate disturbance value is also subjected to discrete wavelet transform to obtain the low-frequency component and high-frequency component of the intermediate disturbance value, wherein the low-frequency component and the high-frequency component include , , and Four frequency components; the generated four frequency components are added to the four frequency components of the perturbed face image respectively, and the calculation formula is as follows:

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] Wherein, DWT() represents discrete wavelet transform;

[0050] Perform inverse discrete wavelet transform on the four frequency components of the perturbed face image to obtain the perturbed adversarial face image , the calculation formula is as follows:

[0051] ;

[0052] Among them, IDWT() represents inverse discrete wavelet transform.

[0053] Further, in S105, the JPEG noise layer is introduced to interfere with the deep fake model, and the calculation formula is as follows:

[0054] ;

[0055] in, Indicates compression processing. Represents the original face And the compressed perturbed face image The distance between the fake outputs of the deepfake model; Represents the constraint threshold on the perturbation size.

[0056] On the other hand, the face deep forgery active defense device based on discrete wavelet transform includes:

[0057] A face image initialization module is used to obtain an original face image and initialize a perturbed face image based on the original face image;

[0058] A total loss function acquisition module, which uses a deep fake model to edit the attributes of a perturbed face image and an original face image respectively to obtain a first forged output and a second forged output; calculates the mean square error loss between the first forged output and the second forged output, and calculates the structural similarity loss between the perturbed face image and the original face image, assigns a first weight coefficient to the mean square error loss, assigns a second weight coefficient to the structural similarity loss, and performs weighted fusion of the mean square error loss and the structural similarity loss based on the first weight coefficient and the second weight coefficient to obtain a total loss function;

[0059] An intermediate perturbation value acquisition module is used to take the perturbed face image as input, obtain the gradient information of the deep fake model through the back propagation mechanism of the deep fake model based on the total loss function, calculate the sign direction of the gradient based on the gradient information, determine the optimal direction of the perturbation through the sign direction of the gradient, and multiply the sign direction of the gradient by the set perturbation amplitude to obtain the intermediate perturbation value;

[0060] The face image acquisition module containing the intermediate disturbance value is used to perform discrete wavelet transform on the intermediate disturbance value to obtain the low-frequency component and high-frequency component of the intermediate disturbance value; perform discrete wavelet transform on the face image containing the disturbance to obtain the low-frequency component and high-frequency component of the face image containing the disturbance; embed the low-frequency component and high-frequency component of the intermediate disturbance value into the low-frequency component and high-frequency component of the face image containing the disturbance respectively to realize the embedding of disturbance information, and then reconstruct the face image embedded with the disturbance information by inverse discrete wavelet transform to obtain the face image containing the intermediate disturbance value;

[0061] The iteration module is used to determine whether the preset number of iterations has been reached; if not, the total loss function acquisition module, the intermediate perturbation value acquisition module and the face image acquisition module containing the intermediate perturbation values ​​are iteratively executed, and a JPEG noise layer is introduced to gradually adjust the amplitude and direction of the perturbation during the iteration process; if reached, a face image containing the intermediate perturbation value is output as the final face image containing invisible perturbation information.

[0062] The present invention adopts the above technical solution and has the following beneficial effects:

[0063] (1) The present invention realizes active defense of facial images and improves the security of facial images by constructing disturbance information for protecting facial images from tampering with deep fake models.

[0064] (2) The present invention makes full use of the robustness advantage of the wavelet domain and combines it with the structural similarity loss to further improve the invisibility and robustness of the perturbation, making the generated perturbation visually imperceptible and showing stronger stability under JPEG compression attacks.

[0065] (3) The present invention significantly improves the robustness of disturbances against JPEG compression while ensuring that the disturbances are invisible and have a high defense success rate. In particular, when dealing with JPEG (Joint Photographic Experts Group, a standard for continuous tone static image compression) compression quality factors of 50-90, the defense success rate can still be maintained above 90%. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flow chart of the active defense method for deep face forgery based on discrete wavelet transform according to an example of the present invention;

[0067] Figure 2 This is a diagram of an active facial deep forgery defense device based on discrete wavelet transform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0069] like Figure 1 As shown, the present invention provides a method for actively defending against deep fake face based on discrete wavelet transform, comprising the following steps:

[0070] S101, obtaining an original face image, and initializing a disturbed face image based on the original face image.

[0071] S102, respectively edit the attributes of the disturbed face image and the original face image through the deep fake model to obtain a first fake output and a second fake output respectively; calculate the mean square error loss between the first fake output and the second fake output, calculate the structural similarity loss between the disturbed face image and the original face image, assign a first weight coefficient to the mean square error loss, assign a second weight coefficient to the structural similarity loss, and weightedly fuse the mean square error loss and the structural similarity loss based on the first weight coefficient and the second weight coefficient to obtain a total loss function.

[0072] Specifically, the attributes of the disturbed face image and the original face image are edited by using the deep fake model, specifically including:

[0073] Given an original face image , a trained deep fake model and can modify specific properties , generate fake images , the formula is as follows:

[0074] ;

[0075] Maximize the fake output of the original face image through the deep fake model And the forged output of the deep forgery model for the perturbed face image The goal of interfering with the output of the deep fake model is achieved by maximizing the distance D. The formula is as follows:

[0076] ;

[0077] in, Represents the original face image An invisible perturbation added to the deep fake model, which is used to destroy the fake output of the deep fake model; The threshold value representing the size of the constrained perturbation is used to ensure that the added perturbation is imperceptible to the human eye; express The infinite norm of .

[0078] Specifically, the mean square error loss The calculation formula is as follows:

[0079] ;

[0080] in, Represents the total number of elements in the tensor, and Represent the original input value and the generated value of the deep fake model respectively; by calculating and The mean square error between the predicted values ​​and the original input values ​​is used to measure the difference between the model predictions and the original input values, guiding the generation and optimization of disturbances.

[0081] Specifically, the structural similarity loss The calculation formula is as follows:

[0082] ;

[0083] in, Represents the original face image The mean of Generate fake images The mean of Represents the original face image The variance of Generate fake images The variance of is the covariance between the two face images; and is a constant used to avoid the denominator being zero.

[0084] Specifically, the formula of the total loss function is as follows:

[0085] ;

[0086] in, Represents the weight parameter for balancing the mean square error loss; The weight parameter representing the structural similarity loss; represents the mean squared error loss, which is used to measure the pixel-level difference between the deepforged image of the original face and the deepforged image containing the perturbed face; Represents the structural similarity loss, which is used to evaluate the similarity between the perturbed face and the original face image in terms of structure, brightness, and contrast.

[0087] Specifically, in the embodiment of the present invention, Set to 1, Set to 0.5.

[0088] S103, taking the perturbed face image as input, based on the total loss function, through the back propagation mechanism of the deep fake model, obtaining the gradient information of the deep fake model, calculating the sign direction of the gradient based on the gradient information, determining the optimal direction of the perturbation through the sign direction of the gradient, and multiplying the sign direction of the gradient by the set perturbation amplitude to obtain the intermediate perturbation value.

[0089] Specifically, the calculation formula of the intermediate disturbance value is as follows:

[0090] ;

[0091] in, represents the disturbance amplitude; represents a deep fake model; Represents the output result of the original face image after being forged by the deep fake model; Represents a perturbed face image The output result after being forged by the deep fake model, Represents the loss function About face images The gradient of Represents a symbolic function.

[0092] S104, determine whether the preset number of iterations has been reached; if not, introduce a JPEG noise layer, iteratively execute S103-S105, and gradually adjust the amplitude and direction of the disturbance during the iteration process; otherwise, output a face image containing an intermediate disturbance value as the final face image containing invisible disturbance information.

[0093] Specifically, a one-dimensional discrete wavelet transform is performed on each row of the disturbed face image, and then a one-dimensional discrete wavelet transform is performed on each column to obtain the low-frequency component and the high-frequency component of the disturbed face image, wherein the low-frequency component and the high-frequency component of the face image include a low-frequency-low-frequency component. , high-frequency-low-frequency component , low-frequency-high-frequency component and high-frequency-high-frequency components ;

[0094] The low-frequency-low-frequency component It is used to represent the rough outline of the perturbed face image. The calculation formula is as follows:

[0095] ;

[0096] The high-frequency-low-frequency component Used to represent the edge information in the horizontal direction. The calculation formula is as follows:

[0097] ;

[0098] Low-frequency-high-frequency component Used to represent the edge information in the vertical direction. The calculation formula is as follows:

[0099] ;

[0100] The high-frequency-high-frequency component Used to represent the details and noise in the image, the calculation formula is as follows:

[0101] ;

[0102] Wherein, m is the row coordinate of the input image, which is used to indicate the position of the image in the vertical direction; n is the column coordinate of the input image, which is used to indicate the position of the image in the horizontal direction; A value representing the pixel value of the image; Represents low-pass filtering, which is used to extract low-frequency components; Represents high-pass filtering, which is used to extract high-frequency components; and Respectively represent the row and column coordinates of the sub-image generated after wavelet transformation;

[0103] The intermediate disturbance value is also subjected to discrete wavelet transform to obtain the low-frequency component and high-frequency component of the intermediate disturbance value, wherein the low-frequency component and the high-frequency component include , , and Four frequency components; the generated four frequency components are added to the four frequency components of the perturbed face image respectively, and the calculation formula is as follows:

[0104] ;

[0105] ;

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] Wherein, DWT() represents discrete wavelet transform;

[0111] Perform inverse discrete wavelet transform on the four frequency components of the perturbed face image to obtain the perturbed adversarial face image , the calculation formula is as follows:

[0112] ;

[0113] Among them, IDWT() represents Inverse Discrete Wavelet Transform.

[0114] Specifically, in this embodiment, Haar wavelet is selected as the wavelet basis function in discrete wavelet transform, and in the iterative process, a JPEG compression noise layer is added to increase the robustness against disturbances. The QF factor is set to a random range of 70-90.

[0115] Specifically, in the process of performing discrete wavelet transform on the disturbed face image and the calculated intermediate disturbance value respectively, the four frequency components of the disturbed face image and the intermediate disturbance value are fused in a 1:1 ratio, and then the face image with the disturbance information fused is inversely discrete wavelet transformed to finally obtain a face image with the disturbance value added.

[0116] Specifically, the wavelet functions at different scales can be regarded as bandpass filters, and the scaling functions can be regarded as low-pass filter groups, thereby decomposing the signal into low-frequency and high-frequency parts. These frequency components correspond to different characteristics of the image, which facilitates the effective processing of the signal. The process of wavelet transform can be regarded as decomposing the signal into low-frequency and high-frequency parts. The wavelet functions at different scales can be regarded as bandpass filters, and the scaling functions can be regarded as low-pass filter groups. The coefficients obtained by wavelet decomposition can be reconstructed into the original signal. This process is called inverse discrete wavelet transform. The specific process is to filter the rows first, perform a one-dimensional discrete wavelet transform on each row of the image, and then use high-pass and low-pass filters respectively to perform a one-dimensional discrete wavelet transform on each column, and then apply high-pass and low-pass filters in each direction to generate four frequency components, namely LL, LH, HL and HH; where the filter is a one-dimensional vector h (low-pass filter) and g (high-pass filter).

[0117] S105, determining whether the preset number of iterations has been reached; if not, iteratively executing S102-S104, and introducing a JPEG noise layer, gradually adjusting the amplitude and direction of the disturbance during the iteration process; if reached, outputting a face image containing an intermediate disturbance value as the final face image containing invisible disturbance information.

[0118] Specifically, the introduction of the JPEG noise layer is used to increase the robustness against disturbances, ensuring that the disturbed face image can still effectively interfere with the deep fake model after JPEG compression. The calculation formula is as follows:

[0119] ;

[0120] in, Indicates compression processing. Represents the original face And the compressed perturbed face image The distance between the fake outputs of the deepfake model; Represents the constraint threshold on the perturbation size.

[0121] Specifically, the present invention is an active defense scheme for deep fake faces in a white-box environment, which interferes with the fake process of a trained deep fake model, resulting in obvious distortion of the fake output. On the basis of ensuring the invisibility of the disturbance and a high defense success rate, the robustness of the generated disturbance to resist JPEG compression is significantly improved; the present invention mainly generates face images containing disturbance information. By introducing a new loss function and utilizing the robustness characteristics of the wavelet domain, combined with the application of the JPEG noise layer, it is ensured that the effects of image compression can be effectively resisted when disturbances are added. This method not only achieves the concealment of disturbance information, improves the robustness of the generated adversarial face images in resisting JPEG compression, but also provides strong support for the security and privacy protection of face images.

[0122] Specifically, the active defense against deep fakes of faces based on discrete wavelet transform is a method for constructing adversarial perturbations to protect facial images. It obtains gradient information about the deep fake model containing perturbed faces by calculating the total loss and using the reverse gradient propagation mechanism of the deep neural network. Subsequently, the perturbed face image and gradient information are subjected to discrete wavelet transform respectively, and the perturbation information is added to the different frequency components of the perturbed face image. Finally, the inverse discrete wavelet transform is performed to obtain the perturbed face image with protection information. By embedding the perturbation into the wavelet domain of the image and combining it with the application of the noise layer, the robustness of the generated perturbation in resisting JPEG compression is significantly improved.

[0123] Specifically, in the embodiment of the present invention, the model StarGAN that performs attribute editing in the deep fake model is selected for defense. The five attributes of black hair, blonde hair, brown hair, age and gender are defended respectively. The number of iterations is 10. The threshold range of the perturbation value on each frequency component is controlled to be 0.1. The present invention is experimentally evaluated on the CelebA dataset. The CelebA dataset is an open source face attribute dataset consisting of 256 The 256-face image contains more than 200,000 celebrity images, each with 40 attributes. We select 50, 100, and 500 face images for defense respectively.

[0124] In addition, in the evaluation, the performance of the Defense Success Rate (DSR) was accurately measured using the evaluation method proposed by Ruiz et al. When the L2 loss distance between the forged output of the deep fake model of the original face image and the forged output of the deep fake model of the perturbed face image is greater than 0.05, the defense is considered successful. At the same time, in order to ensure that the generated perturbations are invisible, we use PSNR and SSIM indicators to measure the quality of the perturbed face images.

[0125] The present invention verifies the defense success rate and visual indicators under different numbers of images without compression attacks. The results are shown in Table 1. The perturbations added in the wavelet domain can successfully interfere with the forged output of the deep fake model under different numbers of images. At the same time, on the StarGAN model, five attributes of the face image are modified, including: changing the hair color to black, gold and brown, and adjusting facial attributes such as gender and age. After adding perturbation information to the face image, the image generated by the deep fake model showed obvious distortion. The attribute editing of the deep fake model failed, and many artifacts appeared in the image, causing the image to look extremely unnatural and easy to be identified as a forgery by the human eye. As a result, the attribute editing function of the deep fake model was successfully destroyed.

[0126] Table 1: Defense success rate and visual status under different numbers of images;

[0127]

[0128] As shown in Table 1, the results of the defense success rate and the visual quality of the generated perturbed face images under different magnitudes of face images. The larger the L2 loss, the stronger the destruction effect; DSR represents the defense success rate, and the higher the defense success rate, the better the defense effect; PSNR and SSIM represent the visual quality of the generated face images, and the larger the value, the better the generated image quality. Experiments show that under any magnitude of face images, the defense success rate proposed by this method can reach 100%, and PSNR and SSIM can reach more than 34dB and 0.94 respectively.

[0129] Therefore, adding disturbances in the wavelet domain of the face image can successfully interfere with the forgery generation of the deep fake model. On this basis, the robustness test of the present invention is further carried out. The specific experimental setting is: JPEG compression of 50 face images with different quality factors is performed, and the defense success rate and visual indicators are calculated, as described in Table 2.

[0130] Table 2 Defense success rate and visual status of perturbed face images under JPEG compression with different quality factors;

[0131]

[0132] As shown in Table 2, the results of the defense success rate and visual indicators under the attack of JPEG compression with different quality factors. That is, in the compression range of quality factor 50-90, the defense success rate can still reach more than 90%. At the same time, due to the introduction of structural consistency loss, the visual indicators of the face image after adding perturbations have increased, the PSNR is stable at around 35dB, and the SSIM is stable above 0.96. When the quality factor is 40, the defense success rate is still more than 80%. Although the defense success rate drops significantly in the range of quality factors of 10-30, this situation is relatively rare in daily use, because in daily use, in order to ensure the normal dissemination of face images, such a low compression quality is usually not used. Therefore, in this case, the decline in the defense success rate is still within an acceptable range.

[0133] Specifically, after a compression attack is carried out on a perturbed face image, the degree of compression of the image will continue to increase as the compression factor gradually decreases. This compression process will lead to the loss of image details, while suppressing or blurring the originally added perturbation information. Specifically, when the compression factor is large, the perturbation can still protect the face image. When the compression factor decreases and the image compression becomes more severe, the perturbation information will gradually be partially suppressed by the compression algorithm, resulting in a significant reduction in artifacts after editing by the deep fake model. Therefore, in real scenarios, defense measures against compression attacks can often effectively maintain the perturbation information in the image and ensure the defense effect.

[0134] like Figure 2 As shown, this embodiment also discloses a face deep forgery active defense device based on discrete wavelet transform, including:

[0135] A face image initialization module 21 is used to obtain an original face image and initialize a disturbed face image based on the original face image;

[0136] The total loss function acquisition module 22 edits the attributes of the perturbed face image and the original face image respectively through the deep fake model to obtain a first forged output and a second forged output; calculates the mean square error loss between the first forged output and the second forged output, and calculates the structural similarity loss between the perturbed face image and the original face image, assigns a first weight coefficient to the mean square error loss, assigns a second weight coefficient to the structural similarity loss, and performs weighted fusion of the mean square error loss and the structural similarity loss based on the first weight coefficient and the second weight coefficient to obtain a total loss function;

[0137] The intermediate disturbance value acquisition module 23 is used to take the disturbed face image as input, obtain the gradient information of the deep fake model through the back propagation mechanism of the deep fake model based on the total loss function, calculate the sign direction of the gradient based on the gradient information, determine the optimal direction of the disturbance application through the sign direction of the gradient, and multiply the sign direction of the gradient by the set disturbance amplitude to obtain the intermediate disturbance value;

[0138] The face image acquisition module 24 containing the intermediate disturbance value is used to perform discrete wavelet transform on the intermediate disturbance value to obtain the low-frequency component and high-frequency component of the intermediate disturbance value; perform discrete wavelet transform on the face image containing the disturbance to obtain the low-frequency component and high-frequency component of the face image containing the disturbance; embed the low-frequency component and high-frequency component of the intermediate disturbance value into the low-frequency component and high-frequency component of the face image containing the disturbance respectively to embed the disturbance information, and then reconstruct the face image embedded with the disturbance information by inverse discrete wavelet transform to obtain the face image containing the intermediate disturbance value;

[0139] The iteration module 25 is used to determine whether the preset number of iterations has been reached; if not, the total loss function acquisition module, the intermediate perturbation value acquisition module and the face image acquisition module containing the intermediate perturbation value are iteratively executed, and a JPEG noise layer is introduced to gradually adjust the amplitude and direction of the perturbation during the iteration process; if reached, a face image containing the intermediate perturbation value is output as the final face image containing invisible perturbation information.

[0140] The specific implementation of the active face deep forgery defense device based on discrete wavelet transform is the same as the active face deep forgery defense method based on discrete wavelet transform, and will not be repeated in this embodiment.

[0141] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. An active defense method for deep face forgery based on discrete wavelet transform, characterized in that: The following steps are involved: S101, obtaining an original face image, and initializing a perturbed face image based on the original face image; S102, respectively editing the attributes of the perturbed face image and the original face image through the deep fake model to obtain a first forged output and a second forged output; calculating the mean square error loss between the first forged output and the second forged output, and calculating the structural similarity loss between the perturbed face image and the original face image, assigning a first weight coefficient to the mean square error loss, assigning a second weight coefficient to the structural similarity loss, and weightedly fusing the mean square error loss and the structural similarity loss based on the first weight coefficient and the second weight coefficient to obtain a total loss function; S103, taking the perturbed face image as input, based on the total loss function, obtaining the gradient information of the deep fake model through the back propagation mechanism of the deep fake model, calculating the sign direction of the gradient based on the gradient information, determining the optimal direction of the perturbation through the sign direction of the gradient, and multiplying the sign direction of the gradient by the set perturbation amplitude to obtain an intermediate perturbation value; S104, performing discrete wavelet transform on the intermediate disturbance value to obtain low-frequency components and high-frequency components of the intermediate disturbance value; performing discrete wavelet transform on the disturbed face image to obtain low-frequency components and high-frequency components of the disturbed face image; The low-frequency component and high-frequency component of the intermediate disturbance value are respectively embedded into the low-frequency component and high-frequency component of the perturbed face image to achieve perturbation information embedding, and then the face image embedded with the perturbation information is reconstructed by inverse discrete wavelet transform to obtain the face image containing the intermediate disturbance value; S105, determining whether a preset number of iterations has been reached; If it is not reached, S102-S104 are iteratively executed, and a JPEG noise layer is introduced, and the amplitude and direction of the disturbance are gradually adjusted during the iteration process; if it is reached, a face image containing an intermediate disturbance value is output as the final face image containing invisible disturbance information.

2. The method for actively defending against deep fake face based on discrete wavelet transform according to claim 1, characterized in that: In S102, the deep fake model is used to edit the attributes of the disturbed face image and the original face image, specifically including: Given an original face image x, a trained deep fake model M, and a specific modifiable attribute c, the formula for generating a fake image y is as follows: y=M(x,c); Maximize the distance D between the forged output M(x) of the original face image after the deep forgery model and the forged output M(x+η) of the perturbed face image after the deep forgery model. By maximizing the distance D, the goal of interfering with the output of the deep forgery model is achieved. The formula is as follows: maxD(M(x),M(x+η))s.t.||η|| ∞ ≤∈; Among them, η represents an invisible perturbation added to the original face image x; ∈ represents the threshold of constraining the perturbation size, which is used to ensure that the added perturbation is imperceptible to the human eye; ||η|| ∞ represents the infinity norm of η.

3. The method for actively defending against deep fakes of human faces based on discrete wavelet transform according to claim 2, characterized in that: In S102, the mean square error loss L MSE The calculation formula is as follows: Where n represents the total number of elements in the tensor, y i and Represent the original input value and the generated value of the deep fake model respectively; by calculating y i and The mean square error between the predicted values ​​and the original input values ​​is used to measure the difference between the model predictions and the original input values, and guide the generation and optimization of disturbances.

4. The method for actively defending against deep fakes of human faces based on discrete wavelet transform according to claim 1, characterized in that: In S102, the structural similarity loss L SSIM The calculation formula is as follows: Among them, μ x represents the mean of the original face image x; μ y represents the mean of the generated fake image y; σ x represents the variance of the original face image x; σ y represents the variance of the generated fake image y; σ xy is the covariance between two face images; C1 and C2 are constants.

5. The method for active face deep forgery defense based on discrete wavelet transform according to claim 1, characterized in that: In S102, the formula of the total loss function is as follows: L total =αL MSE +βL SSIM ; Among them, α represents the weight parameter of the balanced mean square error loss; β represents the weight parameter of the structural similarity loss; L MSE represents the mean square error loss, which is used to measure the pixel-level difference between the deep-forged image of the original face and the deep-forged image containing the perturbed face; L SSIM Represents the structural similarity loss, which is used to evaluate the similarity between the perturbed face and the original face image in terms of structure, brightness, and contrast.

6. The method for actively defending against deep fakes of human faces based on discrete wavelet transform according to claim 1, characterized in that: In S103, the calculation formula of the intermediate disturbance value is as follows: Where α represents the perturbation amplitude; G represents the deep fake model; G(x) represents the output result of the original face image after being forged by the deep fake model; Represents a perturbed face image The output result after being forged by the deep fake model, Represents the total loss function L total () is the gradient of the original face image x, and sign() represents the sign function.

7. The method for actively defending against deep fakes of human faces based on discrete wavelet transform according to claim 6, characterized in that: The S104 specifically includes: Performing a one-dimensional discrete wavelet transform on each row of the disturbed face image, and then performing a one-dimensional discrete wavelet transform on each column to obtain a low-frequency component and a high-frequency component of the disturbed face image, wherein the low-frequency component and the high-frequency component of the disturbed face image include a low-frequency-low-frequency component LL, a high-frequency-low-frequency component LH, a low-frequency-high-frequency component HL, and a high-frequency-high-frequency component HH; The low-frequency-low-frequency component LL is used to represent the rough outline of the disturbed face image, and the calculation formula is as follows: LL[k,j]=∑ m ∑ n x[m,n]·h[k-m]·h[j-n]; The high-frequency-low-frequency component LH is used to represent edge information in the horizontal direction, and the calculation formula is as follows: LH[k,j]=∑ m ∑ n x[m,n]·h[km]·g[jn]; The low-frequency-high-frequency component HL is used to represent the edge information in the vertical direction, and the calculation formula is as follows: <h2 style=";text-align:left;direction:ltr">HL[k,j]=∑<h2 style=";text-align:left;direction:ltr"> m <h2 style=";text-align:left;direction:ltr"> ∑<h2 style=";text-align:left;direction:ltr"> n <h2 style=";text-align:left;direction:ltr"> x[m,n]·g[km]·h[jn]; The high-frequency-high-frequency component HH is used to represent the details and noise in the image, and the calculation formula is as follows: HH[k,j]=∑ m ∑ n x[m,n]·g[k-m]·g[j-n]; Where m is the row coordinate of the input image, which is used to indicate the vertical position of the image; n is the column coordinate of the input image, which is used to indicate the horizontal position of the image; x[m,n] indicates the value of the image pixel value; h indicates low-pass filtering, which is used to extract low-frequency components; g indicates high-pass filtering, which is used to extract high-frequency components; k and j respectively indicate the row and column coordinates of the sub-image generated after wavelet transform; The intermediate disturbance value is also subjected to discrete wavelet transform to obtain the low-frequency component and high-frequency component of the intermediate disturbance value, wherein the low-frequency component and high-frequency component of the intermediate disturbance value include ll η , lh η ,hl η and hh η Four frequency components; the generated four frequency components are added to the four frequency components of the perturbed face image respectively, and the calculation formula is as follows: DWT(x)=ll x ,lh x ,hl x ,hh x ; DWT(η)=ll η ,lh η ,hl η ,hh η ; Wherein, DWT() represents discrete wavelet transform; Perform inverse discrete wavelet transform on the four frequency components of the perturbed face image to obtain the perturbed face image. The calculation formula is as follows: Among them, IDWT() represents inverse discrete wavelet transform.

8. The method for actively defending against deep fakes of human faces based on discrete wavelet transform according to claim 7, characterized in that: In S105, the JPEG noise layer is introduced to interfere with the deep fake model, and the calculation formula is as follows: Among them, Com() represents compression processing, D() represents the forged output M(x) of the original face image x after deep forgery model and the compressed perturbed face image The distance between the forged outputs of the deep fake model; ∈ represents the constraint threshold on the perturbation size.

9. The active defense device for deep fake face based on discrete wavelet transform is characterized by: These include: A face image initialization module is used to obtain an original face image and initialize a perturbed face image based on the original face image; A total loss function acquisition module, which uses a deep fake model to edit the attributes of a perturbed face image and an original face image respectively to obtain a first forged output and a second forged output; calculates the mean square error loss between the first forged output and the second forged output, and calculates the structural similarity loss between the perturbed face image and the original face image, assigns a first weight coefficient to the mean square error loss, assigns a second weight coefficient to the structural similarity loss, and performs weighted fusion of the mean square error loss and the structural similarity loss based on the first weight coefficient and the second weight coefficient to obtain a total loss function; An intermediate perturbation value acquisition module is used to take the perturbed face image as input, obtain the gradient information of the deep fake model through the back propagation mechanism of the deep fake model based on the total loss function, calculate the sign direction of the gradient based on the gradient information, determine the optimal direction of the perturbation through the sign direction of the gradient, and multiply the sign direction of the gradient by the set perturbation amplitude to obtain the intermediate perturbation value; The face image acquisition module containing the intermediate disturbance value is used to perform discrete wavelet transform on the intermediate disturbance value to obtain the low-frequency component and high-frequency component of the intermediate disturbance value; perform discrete wavelet transform on the face image containing the disturbance to obtain the low-frequency component and high-frequency component of the face image containing the disturbance; The low-frequency component and high-frequency component of the intermediate disturbance value are respectively embedded into the low-frequency component and high-frequency component of the perturbed face image to achieve perturbation information embedding, and then the face image embedded with the perturbation information is reconstructed by inverse discrete wavelet transform to obtain the face image containing the intermediate disturbance value; Iteration module, used to determine whether the preset number of iterations has been reached; If it is not reached, the total loss function acquisition module, the intermediate perturbation value acquisition module and the face image acquisition module containing the intermediate perturbation value are iteratively executed, and the JPEG noise layer is introduced to gradually adjust the amplitude and direction of the perturbation during the iteration process; if it is reached, the face image containing the intermediate perturbation value is output as the final face image containing the invisible perturbation information.

Images