An Active Deepfake Defense Method and System Based on Image Texture

Abstract

The present invention relates to the technical field of image-level deepfake defense, and specifically relates to an active deepfake defense method and system based on image texture, as follows: Obtain face images to construct a face image dataset and perform preprocessing; construct a texture perturbation generator, input the preprocessed image to obtain a preliminary texture feature map, and then perform texture enhancement to obtain an enhanced texture perturbation map; combine the enhanced feature map with the perturbation generated by the projected gradient descent method PGD to obtain a perturbed image and generate a tampered image; generate a heatmap gradient feature map based on the tampered image, train the texture perturbation generator in combination with a loss function, input a new face image into the optimized texture perturbation generator, and output a distorted image. The present invention is based on a perturbation optimization strategy guided by the attention difference of a dual model, which can effectively balance local details and global semantics and achieve the collaborative optimization of the defense effect and visual quality.

Description

Technical Field

[0001] The present invention relates to the technical field of image-level deepfake defense, and in particular, to an active deepfake defense method and system based on image texture. Background Art

[0002] The deepfake technology in the field of artificial intelligence represents a major breakthrough in the field of image and video processing. Relying on deep neural networks, especially complex model architectures such as generative adversarial networks and autoencoders, this technology can achieve highly realistic synthesis of facial features and limb movements. With the enhancement of computing power and the iterative upgrade of algorithm technology, the fake content generated by deepfakes has achieved a qualitative leap in realism, even reaching a level that is difficult to distinguish by the human eye. However, this technological progress has also brought many social risks.

[0003] In the field of active deepfake defense, the perturbation method pre-adds imperceptible perturbations to clean images, enabling these images to effectively resist deepfake models. Existing perturbation defense methods based on the spatial domain generally adopt a uniform noise injection strategy. Although it can disrupt the generation process of deepfake models by destroying image features, its undifferentiated uniform perturbation pattern leads to two key defects: on the one hand, uniform noise is likely to introduce obvious perturbation noise in smooth regions, significantly reducing visual quality, and compared with edge and texture regions, the human visual system is more sensitive to perturbations in smooth regions. On the other hand, the perturbation intensity in key facial texture regions is insufficient, making it difficult to suppress the generator's generation and imitation of this region.

[0004] Therefore, the present invention proposes an active deepfake defense method and system based on image texture to solve the above problems. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention develops an active deepfake defense method and system based on image texture. Based on the perturbation optimization strategy guided by the attention difference of a dual model, it uses the local feature resolution of ResNet50 to generate initial texture perturbations and perform local enhancement, and combines the global semantic modeling ability of ViT to optimize the perturbation direction, which can effectively balance local details and global semantics and achieve the collaborative optimization of defense effect and visual quality.

[0006] On the one hand, the technical solution for the present invention to solve the technical problem is an active deepfake defense method based on image texture, including the following steps:

[0007] S1. Obtain face images and construct a face image dataset, and then preprocess the face images in the face image dataset;

[0008] S2. Construct a texture perturbation generator, which includes a noise filtering and texture extraction module FLM and a perturbation enhancement module ENM. Input the preprocessed image into the FLM module for bilateral filtering and LBP ripple extraction to obtain a preliminary texture feature map, and then input the preliminary texture feature map and the preprocessed image into the ENM module for texture enhancement to obtain an enhanced texture perturbation map;

[0009] S3. Combine the enhanced feature map with the perturbation generated by the projected gradient descent method PGD, and then embed it into the corresponding preprocessed image to obtain a perturbed image. Then input the perturbed image and the preprocessed image into the deepfake generator to generate a tampered image;

[0010] S4. Normalize the generated tampered image, and then input the processed image into the Vision Transformer (ViT). Use the output of the last attention block of the ViT as the target layer of the Gradient-weighted Class Activation Mapping (Grad-CAM) to calculate the initial heatmap gradient feature map. Then input the output initial heatmap gradient feature map into the Bottleneck block to output the final heatmap gradient feature map;

[0011] S5. Train the texture perturbation generator according to the final heatmap gradient feature map combined with the loss function to obtain an optimized texture perturbation generator. Finally, input the new face image into the optimized texture perturbation generator to output a distorted image.

[0012] S1 is specifically as follows:

[0013] Obtain face images from the CelebA-HQ dataset and construct a face image dataset The constructed face image dataset is denoted as . Preprocess the face images in . After preprocessing, the face image dataset is denoted as , , denotes the -th face image obtained from the CelebA-HQ dataset, denotes the -th image after preprocessing;

[0014] The preprocessing operations include: uniformly adjusting the size of the face images, converting the format type of the images, and performing normalization processing.

[0015] Construct a noise filtering and texture extraction module FLM, and then input the preprocessed image into the FLM module. The specific process is as follows:

[0016] Construct a noise filtering and texture extraction module FLM. The FLM module includes a bilateral filtering module, an LBP texture feature extraction module, and an LBP threshold processing module. LBP represents local binary pattern;

[0017] Input the preprocessed image into the FLM module. First, it passes through the bilateral filtering module. In the bilateral filtering module, first perform grayscale conversion to obtain a feature map , and then perform bilateral filtering on the feature map to obtain a feature map . Then, input into the LBP texture feature extraction module to extract the LBP texture features of the feature map to obtain a feature map . Finally, input the feature map into the LBP threshold processing module to perform threshold screening on the feature map . Set the threshold . If the feature value of a certain pixel point in the feature map is greater than T, then set this pixel point to 1, otherwise set it to 0. After threshold screening, obtain a texture feature image .

[0018] Construct a perturbation enhancement module ENM. Then, input the output preliminary texture feature map of the FLM module and the preprocessed image into the ENM module. The specific process is as follows:

[0019] The ENM module includes a feature extraction layer, an attention layer, a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, a feature connection layer, and a final feature fusion layer. Among them, the first convolutional block includes a two-dimensional convolutional layer. The second convolutional block sequentially includes a convolutional layer, a first batch normalization layer, and a ReLu activation function. The third convolutional block sequentially includes a convolutional layer, a second batch normalization layer, and a ReLu activation function. The fourth convolutional block sequentially includes a convolutional layer, a third batch normalization layer, and a ReLu activation function. The fifth convolutional block is sequentially composed of a fourth batch normalization layer, a ReLu activation function layer, a convolutional layer, a fifth batch normalization layer, and a ReLu activation function;

[0020] Input the texture feature image into the ENM module. The texture feature image first passes through the feature extraction layer to obtain a feature map . Then, input the preprocessed image into the attention layer to obtain the feature map output by the last Bottleneck block of the 4th layer of ResNet50. The feature map serves as the region for texture enhancement;

[0021] Then, input the feature map and the feature map are fused to obtain a feature map integrated with gradient weighting . Then, the feature map is input into the first convolutional block conblock1 to obtain a feature . The feature is input into the second convolutional block conblock2 to obtain a feature . The feature connection layer groups the two input features according to num_attentions, concatenates the two groups of features inside each attention head, and uses the feature connection layer to and perform feature concatenation to obtain a tensor containing all features . Then, the tensor is input into the third convolutional block conblock3 to obtain a feature . Then, the feature connection layer is used again to and perform feature concatenation to obtain a tensor containing all features . Then, the tensor is input into the fourth convolutional block conblock4 to obtain a feature . Then, the feature connection layer is used again to and perform feature concatenation to obtain a tensor containing all features . Finally, the tensor is input into the fifth convolutional block conblock5 to obtain a feature map . is the enhanced texture perturbation map

[0022] S3 is specifically as follows:

[0023] Perturbations generated by the projected gradient descent method PGD are embedded in the preprocessed image, and then combined with the enhanced texture feature map to obtain a texture perturbation image. The enhanced feature map is combined with the perturbations generated by the projected gradient descent method PGD, and then embedded in the corresponding preprocessed image to obtain a perturbed image. Then, the perturbed image and the preprocessed image are input into the deepfake generator together to generate a tampered image;

[0024] Use the PGD algorithm to generate perturbations of the same size as the preprocessed image , and limit the value of each element in the perturbation to be greater than or equal to -0.05 and less than or equal to 0.05. Then, use torch.tensor in python to convert the NumPy-type perturbation to the PyTorch tensor form. Then, the enhanced texture feature map Each pixel value is associated with a perturbation and added together to obtain a texture perturbation map . Finally, the texture perturbation map and the preprocessed image are jointly input into the deepfake generator to generate a tampered image, which includes a tampered output image with texture perturbation and a tampered output image of the original image .

[0025] S4 is as follows:

[0026] Preprocess the generated tampered image and . The preprocessing operation is specifically normalization to obtain the preprocessed tampered image and . Select the feature before the first LayerNorm layer of the last Transformer block of ViT as the target layer for Grad-CAM. Input the preprocessed tampered image and into ViT to obtain the initial heatmap features and , represents the initial heatmap feature containing texture perturbation, represents the initial heatmap feature of the original image. Then input the initial heatmap features and into the Bottleneck block. The Bottleneck block includes a first convolutional layer, a first batch normalization layer, a second convolutional layer, a second batch normalization layer, a third convolutional layer, a third batch normalization layer, a ReLU activation function, and a residual connection;

[0027] The original heatmap feature and first pass through the first convolutional layer and the first batch normalization layer of the Bottleneck block, and finally pass through the ReLU activation function to output the first-layer heatmap gradient feature map and , represents the first-layer heatmap gradient feature map containing perturbed texture, represents the first-layer heatmap gradient feature map of the original image;

[0028] Then input the first-layer heatmap gradient feature map and into the second convolutional layer and the second batch normalization layer, and finally output the second-layer heatmap gradient feature map through the ReLU activation function and , Indicates the heatmap gradient feature map of the second layer containing the perturbed texture, and the heatmap gradient feature map of the original image of the second layer;

[0029] Then, the heatmap gradient feature map of the second layer and are input into the third convolutional layer and the third batch normalization layer, and the heatmap gradient feature map of the third layer is output and , where indicates the heatmap gradient feature map of the third layer containing the perturbed texture, and indicates the heatmap gradient feature map of the original image of the third layer;

[0030] Finally, the heatmap gradient feature map of the third layer and are subjected to residual connection with the original heatmap feature after downsampling and to obtain the heatmap gradient feature map and , where indicates the heatmap gradient feature map containing the perturbed texture, and indicates the heatmap gradient feature map of the original image. Then, and are input into the ReLU activation function to obtain the final heatmap gradient feature map and , where indicates the final heatmap gradient feature map containing the perturbed texture, and indicates the final heatmap gradient feature map of the original image.

[0031] S5 is specifically as follows:

[0032] Construct a loss function, which includes perceptual loss, attention feature loss, and MSE loss. The calculation formula is as follows:

[0033] ,

[0034] where, indicates the total loss, indicates the perceptual loss, indicates the attention feature loss, indicates the MSE loss, indicates the perceptual loss weight parameter, indicates the heatmap feature loss weight parameter, indicates the MSE loss weight parameter;

[0035] The calculation formula of the perceptual loss is as follows:

[0036] ,

[0037] Among them, represents the feature extraction function of the l-th layer in the ViT network. The output of the last attention block of ViT is selected for the l-th layer. represents the number of elements in the feature map of the l-th layer. represents norm;

[0038] The calculation formula of the attention feature loss is as follows:

[0039] - 2 ,

[0040] where H and W respectively represent the height and width of the input image.

[0041] The calculation formula of the MSE loss is as follows:

[0042] .

[0043] On the other hand, the present invention also provides an active deepfake defense system based on image texture, including a module for executing processing instructions for each step in an active deepfake defense method based on image texture.

[0044] The effects provided in the invention content are only the effects of the embodiments, rather than all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0045] The present invention discloses an active deepfake defense method and system based on image texture. Starting from the perspective of facial texture, the present invention provides a framework for pertinently disturbing the texture features of key parts of facial images. Based on the perturbation optimization strategy guided by the attention difference between dual models, the present invention uses the local feature resolution of ResNet50 to generate initial texture perturbations and perform local enhancement, and combines the global semantic modeling ability of ViT to optimize the perturbation direction, which can effectively balance local details and global semantics and achieve the collaborative optimization of defense effect and visual quality.

[0046] Specifically, by embedding perturbation information into the hidden texture part of the image, it can ensure that the generated perturbed image is highly consistent with the original image visually, minimizing the distortion of the original image.

[0047] By enhancing the features of the preliminary texture region through ConvBlock, compressing the spatial information of the feature map using AdaptiveAvgPool2d, and combining the heatmap feature map generated by Grad-CAM to guide the perturbation localization, the concentration of perturbations in the texture region can be significantly improved, the interference in the smooth region can be reduced, and adversarial perturbation images with high visual quality effects can be further generated.

[0048] Through the adversarial loss based on the attention feature difference, focusing on the attention difference in the key region, and calculating the per-pixel mean square error of the ViT attention map of the generated image before and after adding the perturbation, the perturbation can be forced to produce significant differences in the key attention region of the model to maximize the difference between the original tampered output and the perturbed tampered output, thereby improving the defense effect of the perturbation.

[0049] In summary, in order to effectively resist the harm caused by deepfake technology to society, the present invention adopts a perturbation technology based on human face texture features. When a malicious user attempts to tamper with a protected image, significant visual defects will appear in the result, which can not only effectively expose the tampering behavior, but also greatly limit the abuse of deepfake technology, thereby ensuring the security of the image content. Brief Description of the Drawings

[0050] The drawings are used to provide a further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation to the present invention.

[0051] Figure 1 It is a schematic flowchart of the method of the present invention.

[0052] Figure 2 It is a comparison diagram of the defense effects of the present invention and other perturbation methods under different attribute editing models. Detailed Embodiments

[0053] In order to clearly illustrate the technical features of the present solution, the present invention will be described in detail below through specific embodiments and in conjunction with its drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below.

[0054] Embodiment 1

[0055] An active deepfake defense method based on image texture includes the following steps:

[0056] S1. Obtain a face image and construct a face image dataset, and then preprocess the face images in the face image dataset.

[0057] S2. Construct a texture perturbation generator, which includes a noise filtering and texture extraction module FLM and a perturbation enhancement module ENM. Input the preprocessed image into the FLM module for bilateral filtering and LBP ripple extraction to obtain a preliminary texture feature map, and then input the preliminary texture feature map and the preprocessed image into the ENM module for texture enhancement to obtain an enhanced texture perturbation map;

[0058] S3. Combine the enhanced feature map with the perturbation generated by the projected gradient descent method PGD, and then embed it into the corresponding preprocessed image to obtain a perturbed image. Then, input the perturbed image and the preprocessed image into the deepfake generator to generate a tampered image;

[0059] S4. Normalize the generated tampered image, and then input the processed image into the Vision Transformer ViT. Use the output of the last attention block of ViT as the target layer of the Gradient-weighted Class Activation Mapping Grad-CAM to calculate the initial heatmap gradient feature map. Then, input the output initial heatmap gradient feature map into the Bottleneck bottleneck block to output the final heatmap gradient feature map;

[0060] S5. Train the texture perturbation generator according to the final heatmap gradient feature map combined with the loss function to obtain an optimized texture perturbation generator. Finally, input the new face image into the optimized texture perturbation generator to output a distorted image.

[0061] In the specific implementation manner, S1 is as follows:

[0062] Obtain face images from the CelebA-HQ dataset and construct a face image dataset The constructed face image dataset is denoted as . For the face images in the dataset, perform preprocessing. After preprocessing, the face image dataset is denoted as . . denotes the -th face image obtained from the CelebA-HQ dataset, denotes the -th image after preprocessing;

[0063] The preprocessing operations include: uniformly adjusting the size of the face image, converting the format type of the image, and performing normalization processing;

[0064] Among them, the transforms.Resize() function in Python is specifically used to resize the image to 256×256; the transforms.ToTensor() function in Python is used to convert the format of the image to the tensor type; the transforms.Normalize() function in Python is used to normalize the image, converting the pixel values of the image from the range [0.0, 1.0] to the range [-1.0, 1.0].

[0065] In the specific implementation manner, a noise filtering and texture extraction module FLM is constructed, and then the preprocessed image is input into the FLM module. The specific process is as follows:

[0066] Construct a noise filtering and texture extraction module FLM. The FLM module includes a bilateral filtering module, an LBP texture feature extraction module, and an LBP threshold processing module. LBP represents local binary pattern;

[0067] The preprocessed image is input into the FLM module. First, it passes through the bilateral filtering module. In the bilateral filtering module, gray conversion is first performed to obtain a feature map , and then bilateral filtering is performed on the feature map to obtain a feature map . Then, is input into the LBP texture feature extraction module to extract the LBP texture features of the feature map to obtain a feature map . Finally, the feature map is input into the LBP threshold processing module to perform threshold screening on the feature map . Set the threshold . If the feature value of a certain pixel point in the feature map is greater than T, then set this pixel point to 1, otherwise set it to 0. After threshold screening, a texture feature image is obtained;

[0068] Among them, the cv2.bilateralFilter() function in Python is used to perform bilateral filtering on the image. During the bilateral filtering process, the neighborhood size is set to 31, the variance of the color space is set to 15, and the standard deviation of the spatial coordinates is set to 75; the skimage.feature.local_binary_pattern in Python is used to extract LBP texture features. During the process of extracting LBP texture features, the radius of the local binary pattern is set to 1, and the number of sampling points is set to 8; during the threshold screening process, the threshold is set to 1.

[0069] In the specific implementation manner, a perturbation enhancement module ENM is constructed, and then the output preliminary texture feature map of the FLM module and the preprocessed image are input into the ENM module. The specific process is as follows:

[0070] The ENM module includes a feature extraction layer, an attention layer, a first convolution block, a second convolution block, a third convolution block, a fourth convolution block, a fifth convolution block, a feature connection layer, and a final feature fusion layer. Among them, the first convolution block includes a two-dimensional convolution layer, the second convolution block sequentially includes a convolution layer, a first batch normalization layer, and a ReLu activation function, the third convolution block sequentially includes a convolution layer, a second batch normalization layer, and a ReLu activation function, the fourth convolution block sequentially includes a convolution layer, a third batch normalization layer, and a ReLu activation function, and the fifth convolution block is composed of a fourth batch normalization layer, a ReLu activation function layer, a convolution layer, a fifth batch normalization layer, and a ReLu activation function;

[0071] Input the texture feature image into the ENM module. The texture feature image first passes through the feature extraction layer to obtain a feature map , and then the preprocessed image is input into the attention layer to obtain the feature map output by the last Bottleneck block of the 4th layer of ResNet50 . The feature map serves as the region for texture enhancement;

[0072] Then, the feature map and the feature map are fused to obtain a feature map with gradient weighting fused . Then, the feature map is input into the first convolution block conblock1 to finally obtain a feature . The feature is input into the second convolution block conblock2 to obtain a feature . The feature connection layer groups the two input features according to num_attentions, concatenates the two groups of features inside each attention head, and uses the feature connection layer to and perform feature concatenation to obtain a tensor containing all features . Then, the tensor is input into the third convolution block conblock3 to obtain a feature . Then, the feature connection layer is used again to and perform feature concatenation to obtain a tensor containing all features . Then, the tensor is input into the fourth convolution block conblock4 to obtain a feature , and then use the feature connection layer again to and perform feature splicing to obtain a tensor containing all features , and finally input the tensor into the fifth convolutional block conblock5 to obtain a feature map , is the enhanced texture perturbation map;

[0073] Among them, in the feature extraction layer, Grad-CAM gradient-weighted class activation mapping is used to output important regions that help the network make decisions from the ResNet50 deep convolutional neural network; in the feature connection layer, the input features are grouped according to the number of attention heads num_attentions, and the input features are spliced inside each attention head.

[0074] In the specific implementation manner, S3 is specifically as follows:

[0075] Embed the perturbation generated by the projected gradient descent method PGD into the preprocessed image, and then combine it with the enhanced texture feature map to obtain a texture perturbation image. Combine the enhanced feature map with the perturbation generated by the projected gradient descent method PGD, and then embed it into the corresponding preprocessed image to obtain a perturbed image. Then, input the perturbed image and the preprocessed image into the deepfake generator together to generate a tampered image;

[0076] Use the PGD algorithm to generate a perturbation of the same size as the preprocessed image , and limit the value of each element in the perturbation to be greater than or equal to -0.05 and less than or equal to 0.05. Then, use torch.tensor in python to convert the NumPy type perturbation into the PyTorch tensor form, and then add the pixel value of each bit of the enhanced texture feature map to the perturbation to obtain a texture perturbation map . Finally, input the texture perturbation map and the preprocessed image into the deepfake generator together to generate a tampered image. The tampered image includes a tampered output image containing texture perturbation and a tampered output image of the original image .

[0077] In the specific implementation manner, S4 is specifically as follows:

[0078] For the generated tampered image and Perform preprocessing, and the specific preprocessing operation is normalization to obtain the tampered image after preprocessing and , select the feature before the first LayerNorm layer of the last Transformer block in ViT as the target layer for Grad-CAM, and input the tampered image after preprocessing and into ViT to obtain the initial heatmap feature and , denote the initial heatmap features containing texture perturbations, denote the initial heatmap features of the original image, and then input the initial heatmap features and into the Bottleneck block. The Bottleneck block includes a first convolutional layer, a first batch normalization layer, a second convolutional layer, a second batch normalization layer, a third convolutional layer, a third batch normalization layer, a ReLU activation function, and a residual connection;

[0079] The original heatmap feature and first pass through the first convolutional layer and the first batch normalization layer of the Bottleneck block, and finally pass through the ReLU activation function to output the first-layer heatmap gradient feature map and , denote the first-layer heatmap gradient feature map containing perturbed texture, denote the first-layer heatmap gradient feature map of the original image;

[0080] Then input the first-layer heatmap gradient feature map and into the second convolutional layer and the second batch normalization layer, and finally output the second-layer heatmap gradient feature map through the ReLU activation function and , denote the second-layer heatmap gradient feature map containing perturbed texture, denote the second-layer heatmap gradient feature map of the original image;

[0081] Then input the second-layer heatmap gradient feature map and into the third convolutional layer and the third batch normalization layer, and output the third-layer heatmap gradient feature map and , denote the third-layer heatmap gradient feature map containing perturbed texture, denote the third-layer heatmap gradient feature map of the original image;

[0082] Finally, input the third-layer heatmap gradient feature map and and the original heatmap features after downsampling and perform residual connection to obtain the heatmap gradient feature map and , represents the heatmap gradient feature map containing perturbed textures, represents the heatmap gradient feature map of the original image, and then and are input into the ReLU activation function to obtain the final heatmap gradient feature map and , represents the final heatmap gradient feature map containing perturbed textures, represents the final heatmap gradient feature map of the original image.

[0083] In the specific implementation manner, S5 is as follows:

[0084] Construct a loss function, which includes perceptual loss, attention feature loss, and MSE loss. The calculation formula is as follows:

[0085] ,

[0086] where, represents the total loss, represents the perceptual loss, represents the attention feature loss, represents the MSE loss, represents the perceptual loss weight parameter, represents the heatmap feature loss weight parameter, represents the MSE loss weight parameter;

[0087] The calculation formula of the perceptual loss is as follows:

[0088] ,

[0089] where, represents the feature extraction function of the l-th layer in the ViT network. The l-th layer selects the output of the last attention block of the ViT, represents the number of elements in the feature map of the l-th layer, represents norm;

[0090] The calculation formula of the attention feature loss is as follows:

[0091] - 2 ,

[0092] Among them, H and W respectively represent the height and width of the input image;

[0093] The calculation formula of the MSE loss is as follows:

[0094] 。

[0095] Example 2

[0096] An active deepfake defense system based on image texture, including modules for executing the processing instructions of each step in an active deepfake defense method based on image texture.

[0097] Example 3

[0098] In order to better prove the technical effects of the present invention, the method in the present invention is compared with the existing methods in different scenarios.

[0099] The experiment uses the CelebA-HQ face image as the training data set. The CelebA-HQ data set is a data set released by the Chinese University of Hong Kong, which contains human face images and identities. The images are high-resolution versions. In order to verify the reliability of the method in the present invention, a comparative experiment of the method in the present invention and the existing methods is carried out based on the CelebA-HQ data set;

[0100] The existing methods include four existing perturbation models, namely the CMUA perturbation model, the Anti-Forgery perturbation model, the Initiative perturbation model, and the DF_RAP perturbation model. As shown in Table 1, the four perturbation models and the method in the present invention are respectively compared in terms of the average peak signal-to-noise ratio, the structural similarity index, and the perceptual image block similarity. As shown in Table 2, the four perturbation models and the method in the present invention are respectively subjected to adversarial tests under the StarGAN attribute editing model, the AttGAN attribute editing model, the AGGAN attribute editing model, the HiSD attribute editing model, and the StarGAN_v2 expression reproduction model.

[0101] It can be seen from the experimental results in Table 1 and Table 2 that the method of the present invention has better performance than the existing four perturbation methods. It can prove that the quantitative visual quality evaluation effect of the method of the present invention is better. In the defense effect evaluation results of Table 2, use Measure the Euclidean distance between the original tampered image and the tampered image after adding perturbations, and use DSR to represent the defense success rate ( >0.05 is regarded as a successful defense). The bold data in Table 2 are all the best effects. According to the evaluation results in Table 2, the defense evaluation effect of the method of the present invention is the best.

[0102] Table 1 Quantitative Visual Quality Evaluation of Perturbed Images on the CelebA-HQ Data Set

[0103]

[0104] Table 2 Evaluation of the Defense Effect of Perturbed Images on the CelebA-HQ Dataset

[0105]

[0106] As Figure 2 shown, in order to prove that the present invention has better effects compared with other existing perturbation models, the image effects of different methods under different attribute editing models are given, and the differences between the method of the present invention and the existing perturbation models can be intuitively analyzed. Figure 2 Among them, there are two columns of images under five different attribute editing models. Among them, the first column is the original image, and the second column is the original image protected by individual perturbations (except that the first row of all the second columns is the normal tampering result of the original image). It should be noted that since the attribute editing model directly tampers with the target image, and the expression reproduction requires the source image to provide an expression reference, the last column shows the source image when confronting StarGan-V2; from Figure 2 it can be seen that the perturbed images of the present invention are almost visually indistinguishable from the original images, indicating that they have good visual quality. At the same time, the tampering effect of the last row has obvious visual defects compared with other perturbation methods, proving that the perturbation method of the present invention has successfully resisted the tampering of five deepfake generation models.

[0107] Although the specific implementation manners of the invention are described above in conjunction with the accompanying drawings, they are not limitations on the protection scope of the present invention. Based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. An active deepfake defense method based on image texture, characterized in that, Including the following steps: S1. Obtain face images and construct a face image dataset, and then preprocess the face images in the face image dataset; S2. Construct a texture perturbation generator. The texture perturbation generator includes a noise filtering and texture extraction module FLM and a perturbation enhancement module ENM. Input the preprocessed image into the FLM module to perform bilateral filtering and LBP ripple extraction to obtain a preliminary texture feature map, and then input the preliminary texture feature map and the preprocessed image into the ENM module for texture enhancement to obtain an enhanced texture perturbation map; S3. Combine the enhanced feature map with the perturbation generated by the projected gradient descent method PGD, and then embed it into the corresponding preprocessed image to obtain a perturbed image. Then input the perturbed image and the preprocessed image into a deepfake generator to generate a tampered image; S4. Perform normalization processing on the generated tampered image, and then input the processed image into a vision transformer ViT. Use the output of the last attention block of the ViT as the target layer of the gradient-weighted class activation mapping Grad-CAM to calculate the initial heatmap gradient feature map, and then input the output initial heatmap gradient feature map into a Bottleneck bottleneck block to output the final heatmap gradient feature map; S5. Train the texture perturbation generator according to the final heatmap gradient feature map combined with a loss function to obtain an optimized texture perturbation generator. Finally, input a new face image into the optimized texture perturbation generator to output a distorted image.

2. The active deepfake defense method based on image texture according to claim 1, characterized in that, S1 is specifically as follows: Obtained from the CelebA-HQ dataset face images and construct a face image dataset . The constructed face image dataset is denoted as . For the face images in, after preprocessing, the face image dataset is denoted as . . denotes the -th face image obtained from the CelebA-HQ dataset, denotes the -th image after preprocessing; The preprocessing operations include: uniformly adjusting the size of the face image, converting the format type of the image, and performing normalization processing.

3. The active deepfake defense method based on image texture according to claim 2, characterized in that, Construct a noise filtering and texture extraction module FLM, and then input the preprocessed image into the FLM module. The specific process is as follows: Construct a noise filtering and texture extraction module FLM. The FLM module includes a bilateral filtering module, an LBP texture feature extraction module, and an LBP threshold processing module. LBP represents local binary pattern; Input the preprocessed image into the FLM module. First, it passes through the bilateral filtering module. In the bilateral filtering module, grayscale conversion is first performed to obtain a feature map . Then, bilateral filtering is performed on the feature map to obtain a feature map . Then, is input into the LBP texture feature extraction module to extract the LBP texture features of the feature map to obtain a feature map . Finally, the feature map is input into the LBP threshold processing module to perform threshold screening on the feature map . Set the threshold . If the feature value of a certain pixel point in the feature map is greater than T, then set the pixel point to 1; otherwise, set it to 0. After threshold screening, a texture feature image is obtained.

4. An active deepfake defense method based on image texture according to claim 3, characterized in that, Construct a perturbation enhancement module ENM, and then input the output preliminary texture feature map of the FLM module and the preprocessed image into the ENM module. The specific process is as follows: The ENM module includes a feature extraction layer, an attention layer, a first convolutional block, a second convolutional block, a third convolutional block, a fourth convolutional block, a fifth convolutional block, a feature connection layer, and a final feature fusion layer. Among them, the first convolutional block includes a two-dimensional convolutional layer, the second convolutional block sequentially includes a convolutional layer, a first batch normalization layer, and a ReLu activation function, the third convolutional block sequentially includes a convolutional layer, a second batch normalization layer, and a ReLu activation function, the fourth convolutional block sequentially includes a convolutional layer, a third batch normalization layer, and a ReLu activation function, and the fifth convolutional block sequentially includes a fourth batch normalization layer, a ReLu activation function layer, a convolutional layer, a fifth batch normalization layer, and a ReLu activation function; Input the texture feature image into the ENM module. The texture feature image first passes through the feature extraction layer to obtain a feature map . Then, the preprocessed image is input into the attention layer to obtain the feature map output by the last Bottleneck block of the 4th layer of ResNet50 . The feature map serves as the region for texture enhancement; Then, the feature map and the feature map are fused to obtain a feature map incorporating gradient weighting. Then, the feature map is input into the first convolutional block conblock1 to obtain a feature . The feature is input into the second convolutional block conblock2 to obtain a feature . The feature connection layer groups the two input features according to num_attentions, concatenates the two groups of features within each attention head, and uses the feature connection layer to and for feature concatenation to obtain a tensor containing all features. Then, the tensor is input into the third convolutional block conblock3 to obtain a feature . Then, the feature connection layer is used again to and for feature concatenation to obtain a tensor containing all features. Then, the tensor is input into the fourth convolutional block conblock4 to obtain a feature . Then, the feature connection layer is used again to and for feature concatenation to obtain a tensor containing all features. Finally, the tensor is input into the fifth convolutional block conblock5 to obtain a feature map . is the enhanced texture perturbation map.

5. A method for actively defending against deepfakes based on image texture according to claim 4, characterized in that S3 is specifically as follows: Embed the perturbation generated by the projection gradient descent method PGD into the preprocessed image, and then combine it with the enhanced texture feature map to obtain a texture perturbation image. Combine the enhanced feature map with the perturbation generated by the projection gradient descent method PGD, and then embed it into the corresponding preprocessed image to obtain a perturbed image. Then, input the perturbed image and the preprocessed image into the deepfake generator to generate a tampered image; Generate perturbations of the same size as the pre - processed image using the PGD algorithm and limit the value of each element in the perturbation to be greater than or equal to - 0.05 and less than or equal to 0.

05. Then use torch.tensor in python to convert the NumPy - type perturbation to the PyTorch tensor form. Next, add each pixel value of the enhanced texture feature map to the perturbation to obtain the texture perturbation map . Finally, input the texture perturbation map and the pre - processed image into the deepfake generator together to generate the tampered image, where the tampered image includes the tampered output image with texture perturbation and the tampered output image of the original image . .

6. The active deepfake defense method based on image texture according to claim 5, characterized in that, S4 is specifically as follows: The generated tampered image and are preprocessed. The preprocessing operation is specifically normalization to obtain the preprocessed tampered image and . Select the feature before the first LayerNorm layer of the last Transformer block of ViT as the target layer of Grad-CAM. Input the preprocessed tampered image and into ViT to obtain the initial heatmap feature and . represents the initial heatmap feature containing texture perturbations, represents the initial heatmap feature of the original image. Then, input the initial heatmap features and into the Bottleneck block. The Bottleneck block includes a first convolutional layer, a first batch normalization layer, a second convolutional layer, a second batch normalization layer, a third convolutional layer, a third batch normalization layer, a ReLU activation function, and a residual connection; Original heatmap features And First, it passes through the first convolutional layer and the first batch normalization layer of the Bottleneck block, and finally passes through the ReLU activation function to output the first-layer heatmap gradient feature map And , Indicates the heatmap gradient feature map of the first layer containing perturbed textures, Indicates the heatmap gradient feature map of the original image of the first layer; Then, the first-layer heatmap gradient feature map and are input into the second convolutional layer and the second batch normalization layer, and finally, the second-layer heatmap gradient feature map and are output through the ReLU activation function, where represents the heatmap gradient feature map of the second layer containing the perturbed texture, and represents the heatmap gradient feature map of the original image of the second layer; Then, the second-layer heatmap gradient feature map and are input into the third convolutional layer and the third batch normalization layer, and the third-layer heatmap gradient feature map and are output. represents the heatmap gradient feature map of the third layer containing perturbed textures, and represents the heatmap gradient feature map of the original image of the third layer; Finally, the third-layer heatmap gradient feature map and are subjected to residual connection with the original heatmap features after downsampling and to obtain the heatmap gradient feature map and , where represents the heatmap gradient feature map containing perturbed textures, represents the heatmap gradient feature map of the original image. Then, and are input into the ReLU activation function to obtain the final heatmap gradient feature map and , where represents the final heatmap gradient feature map containing perturbed textures, represents the final heatmap gradient feature map of the original image.

7. The method for actively defending against deepfakes based on image texture according to claim 1, wherein S5 is specifically as follows: Construct a loss function, which includes perceptual loss, attention feature loss, and MSE loss. The calculation formula is as follows: ， Among them, Represents the total loss, Represents the perceptual loss, Represents the attention feature loss, Represents the MSE loss, Represents the perceptual loss weight parameter, Represents the heatmap feature loss weight parameter, Represents the MSE loss weight parameter; The calculation formula of the perceptual loss is as follows: ， Among them, represents the feature extraction function of the l-th layer in the ViT network. The output of the last attention block of ViT is selected for the l-th layer, represents the number of elements in the feature map of the l-th layer, represents norm; The calculation formula of the attention feature loss is as follows: - 2 ， where H and W respectively represent the height and width of the input image; The calculation formula of the MSE loss is as follows: 。 8. An active deepfake defense system based on image texture, characterized in that: It includes modules for executing the processing instructions of each step in the method for actively defending against deepfakes based on image texture described in any one of claims 1-7.

Images