An Active Deepfake Detection Method Based on Watermark Image Difference

Through the active Deepfake detection method based on the watermark image difference value, the watermark is removed using the quaternion extreme Fourier moment and the Unet network, and combined with multiple loss function optimization, the lack of generalization of cross-domain detection in the existing technology is solved. The problem of this is to achieve high precision and robust Deepfake detection.

CN118799163BActive Publication Date: 2024-11-15QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411273778.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-11-15
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

The existing Deepfake detection technology is insufficient generalization and poor robustness when facing cross-domain detection, and cannot effectively capture forgery traces, resulting in a high misjudgment rate.

Method used

Using the active Deepfake detection method based on the difference value of the watermark image, using the quaternion extreme harmony Fourier moment (QPFHM) as the watermark embedding tool, the watermark was removed through the two-stage detection framework and the Unet network, and combined with MSE, local and perceived losses. Optimize, use the Xception network to classify the authenticity of the difference information.

Benefits of technology

High-precision detection of Deepfake is realized, especially when facing unknown forgery methods and cross-domain detection, the detection robustness is effectively distinguished by watermark image difference information, and the detection robustness is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118799163B_ABST
    Figure CN118799163B_ABST
Patent Text Reader

Abstract

The present invention discloses an active Deepfake detection method based on watermark image difference, which relates to the field of image processing technology and includes the following steps: S1: constructing a two-stage active Deepfake detection framework, including a watermark attack network training stage and Deepfake detection; S2: training a watermark attack network stage; S21: using quaternion polar harmonic Fourier moment as a watermark embedding tool to embed watermark information and using a watermark attack network to remove the watermark; S22: to ensure that the watermark attack network can accurately remove the quaternion polar harmonic Fourier moment watermark. The present invention uses quaternion polar harmonic Fourier moment as a watermark embedding tool, and uses the difference between the embedded watermark image and the original image as the main basis for Deepfake detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an active Deepfake detection method based on watermark image difference. Background Art

[0002] As the spread of false information in the media becomes more and more rampant, the credibility of digital content may be undermined by the ease of creating false facts. Information may be modified many times before it is conveyed to potential users. Recent advances in artificial intelligence, especially in the field of image and video processing, have made it increasingly possible to change the content of information, as media content may be subject to various types of manipulation. With the rapid development of image synthesis technology, such as GAN, Diffusion, AI-generated images have reached a high level, making them almost indistinguishable from real images to human observers. Among the various possible media manipulation methods, Deepfake is a recently emerged technology that is able to generate synthetic human images.

[0003] Current Deepfake detection technology mainly relies on passively capturing traces left during the forgery process. These traces are located differently for different forgery methods, so the effect is unsatisfactory when facing cross-domain detection. In order to solve this problem, these forgery traces are fully explored from the perspective of active detection, and Deepfake detection is performed through changes in the watermark area. The existing Deepfake detection methods mainly face problems of insufficient generalization and robustness. When facing unknown forgery methods or cross-domain detection, the performance of these methods will be significantly reduced. The specific reason is that the model does not capture the forgery traces accurately enough and thus makes misjudgments. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an active Deepfake detection method based on watermark image difference, using quaternion polar harmonic Fourier moment (QPFHM) as a watermark embedding tool, and using the difference between the embedded watermark image and the original image as the main basis for Deepfake detection.

[0005] The present invention adopts the following technical solutions to achieve the invention objectives:

[0006] An active Deepfake detection method based on watermark image difference, characterized by comprising the following steps:

[0007] S1: Construct a two-stage active Deepfake detection framework, including training watermark attack network stage and Deepfake detection;

[0008] S2: training watermark attack network stage;

[0009] S21: Use quaternion polar harmonic Fourier moments as a watermark embedding tool to embed watermark information and use watermark attack network to remove watermark;

[0010] S22: In order to ensure that the watermark attack network can accurately remove the quaternion polar harmonic Fourier moment watermark, three losses are used for optimization: MSE loss, local loss and perceptual loss;

[0011] S3: Deepfake detection stage; use the watermark attack network trained in the watermark attack network training stage to perform watermark attack, so that the input watermarked image can remove the watermark information, thereby obtaining the difference between the watermarked image and the watermark-removed image, and then use an Xception classifier network to classify the authenticity of the difference forgery traces.

[0012] As a further limitation of the present technical solution, the specific process of S21 is as follows:

[0013] S211: For polar coordinate system For the color image below, the quaternion polar harmonic Fourier moment is expressed as:

[0014] (1)

[0015] in: is the radial distance of polar coordinates;

[0016] is the polar coordinate angle;

[0017] is a unit pure quaternion;

[0018] is the order;

[0019] is the repeatability;

[0020] Represents radial basis functions:

[0021] (2)

[0022] S212: After calculating the quaternion polar harmonic Fourier moment of the color image by equation (2), a finite number of quaternion polar harmonic Fourier moments are used to calculate the quaternion polar harmonic Fourier moment of the color image. To approximate reconstruction , in polar coordinates, the color image function is reconstructed using a finite number of quaternion polar harmonic Fourier moments The expression is written as:

[0023] (3)

[0024] S213: Embedding process;

[0025] First, the original color face image Convert to polar coordinates and express it as ;

[0026] Then, according to the definition of orthogonal moments and quaternion theory, we can calculate High-order quaternion orthogonal moments ,in and denote the order and repetition respectively;

[0027] from Filter out the robust quaternion orthogonal moment set ;

[0028] Selected watermark For length A binary random sequence, based on the key and watermark sequence Adjust the moment amplitude to obtain the watermarked moment set ;

[0029] Respectively and The image is reconstructed by formula (3) and , and finally get the image with watermark ;

[0030] S214: The watermarked image With the original image Input into the watermark attack network, the watermark attack network uses the Unet network, removes the watermark, and outputs the image .

[0031] As a further limitation of the present technical solution, the Unet network is a symmetrical U-shaped structure, consisting of an encoder and a decoder. The encoder gradually extracts image features and reduces the spatial dimension by downsampling; the decoder gradually restores the spatial dimension and reconstructs the image by upsampling, while combining the feature information of the encoder through a residual connection.

[0032] As a further limitation of the present technical solution, the specific process of S22 is as follows:

[0033] S221: MSE loss; MSE loss function is one of the most commonly used pixel-by-pixel loss functions in image restoration tasks. The output obtained from the watermark attack network is the image , the MSE loss is expressed as:

[0034] (4)

[0035] S222: Local loss; According to the principle of quaternion polar harmonic Fourier moment, the final location of the embedded watermark is always within the maximum inscribed circle of the image, and a novel local loss function is designed:

[0036] (5)

[0037] in: Represents the input tensor in the image coordinate The index value at

[0038] Represents the input tensor in the image coordinate The index value at

[0039] A collection of indices representing the maximum inline circular region defined in the input tensor;

[0040] Specifically, for each image in a batch, Include all indexes that satisfy the following conditions :

[0041] , are coordinates with the center of the image as the origin, H and W are the height and width of the image respectively;

[0042] N m is the total number of pixels in the mask area, i.e. The number of elements in ;

[0043] S223: Perceptual Loss: Therefore, a pre-trained VGG16 model is used to extract key feature maps. The perceptual loss network is calculated on the selected network layer, the original image Image output from the watermark attack network The Euclidean distance between them, the specific loss function is defined as follows:

[0044] (6)

[0045] in: It is the first Layer-specific graph output;

[0046] , and Respectively The channels, height and width of the layer feature map;

[0047] S224: The total loss of the watermark attack network is expressed as:

[0048] (7)

[0049] in: and is a hyperparameter that controls the weight of the loss function.

[0050] As a further limitation of the present technical solution, the specific process of S3 is as follows:

[0051] Fix the watermark attack network trained in the watermark attack network training phase and pass the watermarked image As input to the second stage;

[0052] S31: First, the watermarked image Perform Deepfake to get the detection image ;

[0053] S32: The watermarked image With detection image Input into the watermark attack network to obtain the unforged post-attack images And the forged attack image ;

[0054] S33: Since the difference information cannot be easily detected, the amplified difference information is used for forgery detection, which is expressed as:

[0055] (8)

[0056] in: It means using Xception network to classify the difference image into true and false;

[0057] Indicates the magnification factor of the difference image;

[0058] The prediction result of the Xception network for the input;

[0059] Using a simple BCE loss, the Xception network can be trained:

[0060] (9)

[0061] in: Indicates the label of the face, 0 represents true and 1 represents false;

[0062] Indicates the result predicted by Xception.

[0063] Compared with the prior art, the advantages and positive effects of the present invention are: the present invention proposes a novel active Deepfake detection method. By using the difference information after the watermark image is forged, high-precision Deepfake detection can be achieved. The present invention designs a two-stage active Deepfake detection framework. And a new local loss function is designed to assist the watermark attack network in removing the watermark. The existing active Deepfake detection methods mainly focus on comparing the watermark information extracted by the decoder with the embedded information to determine whether the image is forged. From another perspective, this method designs a two-stage active Deepfake detection framework to perform forgery detection through the difference information after the watermark is embedded. At the same time, for the watermark attack network, the present invention proposes a new local loss function to help the network better attack the area where the watermark is embedded. Given the robustness of QPHFM against conventional attacks, the present invention can also maintain a high detection rate when facing compression; no matter what kind of Deepfake the watermark image passes through, the basic principle is to generate a highly realistic image that cannot be distinguished by the naked eye whether it is a forgery, but this will always change the characteristics of the watermark area. By utilizing this feature, the present invention can also maintain an excellent detection rate when facing different Deepfake forgery methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is the training watermark attack network and Deepfake detection stage of the present invention.

[0065] Figure 2 It is the Unet network adopted by the watermark attack network of the present invention. DETAILED DESCRIPTION

[0066] A specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation.

[0067] The present invention comprises the following steps:

[0068] S1: Construct a two-stage active Deepfake detection framework, including training watermark attack network stage and Deepfake detection.

[0069] S2: Training watermark attack network stage.

[0070] S21: Use quaternion polar harmonic Fourier moment (QPFHM) as a watermark embedding tool to embed watermark information and use the watermark attack network to remove the watermark.

[0071] The specific process of S21 is as follows:

[0072] Since QPHFMs have excellent description ability for color images, QPHFMs are used for watermark embedding;

[0073] S211: For polar coordinate system For the color image below, the quaternion polar harmonic Fourier moment can be expressed as:

[0074] (1)

[0075] in: is the radial distance of polar coordinates;

[0076] is the polar coordinate angle;

[0077] is a unit pure quaternion;

[0078] is the order;

[0079] is the repeatability;

[0080] Represents radial basis functions:

[0081] (2)

[0082] S212: After the quaternion harmonic Fourier moment of the color image is calculated by equation (2), a finite number of (assuming the highest order is The maximum repeatability is ) Quaternion polar harmonic Fourier moment To approximate reconstruction , in polar coordinates, the color image function is reconstructed using a finite number of quaternion polar harmonic Fourier moments The expression can be written as:

[0083] (3)

[0084] S213: Embedding process;

[0085] First, the original color face image Convert to polar coordinates and express it as ;

[0086] Then, according to the definition of orthogonal moments and quaternion theory, we can calculate High-order quaternion orthogonal moments ,in and denote the order and repetition respectively;

[0087] from Filter out the robust quaternion orthogonal moment set ;

[0088] Selected watermark For length The binary random sequence is used to adjust the matrix amplitude according to the key and watermark sequence to obtain the matrix set containing the watermark. ;

[0089] Respectively and The image is reconstructed by formula (3) and , and finally we can get the image with watermark ;

[0090] S214: The watermarked image With the original image Input into the watermark attack network, the watermark attack network uses the Unet network, removes the watermark, and outputs the image .

[0091] The Unet network is a symmetrical U-shaped structure consisting of an encoder and a decoder. The encoder gradually extracts image features and reduces the spatial dimension by downsampling; the decoder gradually restores the spatial dimension and reconstructs the image by upsampling, while combining the feature information of the encoder through a residual connection.

[0092] S22: In order to ensure that the watermark attack network can accurately remove the quaternion polar harmonic Fourier moment watermark, three losses are used for optimization: MSE loss, local loss and perceptual loss.

[0093] The specific process of S22 is as follows:

[0094] S221: MSE loss; MSE loss function is one of the most commonly used pixel-by-pixel loss functions in image restoration tasks. The output obtained from the watermark attack network is the image , the MSE loss can be expressed as:

[0095] (4)

[0096] S222: Local loss; According to the principle of quaternion polar harmonic Fourier moment, the final location of the embedded watermark is always within the maximum inscribed circle of the image, and a novel local loss function is designed:

[0097] (5)

[0098] in: Represents the input tensor in the image coordinate The index value at

[0099] Represents the input tensor in the image coordinate The index value at

[0100] A collection of indices representing the maximum inline circular region defined in the input tensor;

[0101] Specifically, for each image in a batch, Include all indexes that satisfy the following conditions :

[0102] , are coordinates with the center of the image as the origin, H and W are the height and width of the image respectively;

[0103] N m is the total number of pixels in the mask area, i.e. The number of elements in ;

[0104] S223: Perceptual loss: In order to improve the perception of the watermark attack network to the image feature information, we use the perceptual loss function. Compared with the simple mean square error loss function, the perceptual loss function can better reconstruct the details and edges of the image. This is because the human visual system does not perceive the image by comparing pixel by pixel, but by identifying the features of the image. Therefore, a pre-trained VGG16 model (Visual Geometry Group 16) is used to extract the key feature maps. The perceptual loss network calculates the original image at the selected network layer. Image output from the watermark attack network The Euclidean distance between them, the specific loss function is defined as follows:

[0105] (6)

[0106] in: It is the first Layer-specific graph output;

[0107] , and Respectively The channels, height and width of the layer feature map;

[0108] S224: The total loss of the watermark attack network can be expressed as:

[0109] (7)

[0110] in: and To control the hyperparameters of the loss function weight, in this method Set to 0.1, Set to 0.5.

[0111] S3: Deepfake detection stage; use the watermark attack network trained in the watermark attack network training stage to perform watermark attack, so that the input watermarked image can remove the watermark information, so that the difference between the watermarked image and the watermark-removed image can be obtained, and then an Xception classifier network is used to classify the authenticity of the forged traces of the difference.

[0112] The specific process of S3 is as follows:

[0113] Fix the watermark attack network trained in the watermark attack network training phase and pass the watermarked image As input to the second stage;

[0114] S31: First, the watermarked image Perform Deepfake to get the detection image ;

[0115] S32: The watermarked image With detection image Input into the watermark attack network to obtain the unforged post-attack images And the forged attack image ;

[0116] S33: Since the difference information cannot be easily detected, the amplified difference information is used for forgery detection, which can be expressed as:

[0117] (8)

[0118] in: It means using Xception network to classify the difference image into true and false;

[0119] Indicates the magnification factor of the difference image;

[0120] The prediction result of the Xception network for the input;

[0121] The Xception network can be trained using a simple BCE loss:

[0122] (9)

[0123] in: Indicates the label of the face, 0 represents true and 1 represents false;

[0124] Indicates the result predicted by Xception.

[0125] The above disclosure is only a specific embodiment of the present invention, but the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. An active Deepfake detection method based on watermark image difference, It is characterized in that The following steps are involved: S1: Construct a two-stage active Deepfake detection framework, including training watermark attack network stage and Deepfake detection; S2: training watermark attack network stage; S21: Use quaternion polar harmonic Fourier moments as a watermark embedding tool to embed watermark information and use watermark attack network to remove watermark; S22: In order to ensure that the watermark attack network can accurately remove the quaternion polar harmonic Fourier moment watermark, three losses are used for optimization: MSE loss, local loss and perceptual loss; S3: Deepfake detection stage: Use the watermark attack network trained in the watermark attack network training stage to perform watermark attack, so that the input watermarked image can remove the watermark information, thus obtaining the difference between the watermarked image and the watermark-removed image, and then use an Xception classifier network to classify the authenticity of the forged traces of the difference; The specific process of S21 is as follows: S211: For polar coordinate system For the color image below, the quaternion polar harmonic Fourier moment is expressed as: (1) in: is the radial distance of polar coordinates; is the polar coordinate angle; is a unit pure quaternion; is the order; is the repeatability; Represents radial basis functions: (2) S212: After calculating the quaternion polar harmonic Fourier moment of the color image by equation (2), a finite number of quaternion polar harmonic Fourier moments are used to calculate the quaternion polar harmonic Fourier moment of the color image. To approximate reconstruction , in polar coordinates, the color image function is reconstructed using a finite number of quaternion polar harmonic Fourier moments The expression is written as: (3) S213: Embedding process; First, the original color face image Convert to polar coordinates and express it as ; Then, according to the definition of orthogonal moments and quaternion theory, we can calculate High-order quaternion orthogonal moments ,in and denote the order and repetition respectively; from Filter out the robust quaternion orthogonal moment set ; Selected watermark For length A binary random sequence, based on the key and watermark sequence Adjust the moment amplitude to obtain the watermarked moment set ; Respectively and The image is reconstructed by formula (3) and , and finally get the image with watermark ; S214: The watermarked image With the original image Input into the watermark attack network, the watermark attack network uses the Unet network, removes the watermark, and outputs the image ; The specific process of S22 is as follows: S221: MSE loss; MSE loss function is one of the most commonly used pixel-by-pixel loss functions in image restoration tasks. The output obtained from the watermark attack network is the image , the MSE loss is expressed as: (4) S222: Local loss; According to the principle of quaternion polar harmonic Fourier moment, the final location of the embedded watermark is always within the maximum inscribed circle of the image, and a novel local loss function is designed: (5) in: Represents the input tensor in the image coordinate The index value at Represents the input tensor in the image coordinate The index value at A collection of indices representing the maximum inline circular region defined in the input tensor; Specifically, for each image in a batch, Include all indexes that satisfy the following conditions : , are coordinates with the center of the image as the origin, H and W are the height and width of the image respectively; N m is the total number of pixels in the mask area, i.e. The number of elements in ; S223: Perceptual Loss: Therefore, a pre-trained VGG16 model is used to extract key feature maps. The perceptual loss network is calculated on the selected network layer, the original image Image output from the watermark attack network The Euclidean distance between them, the specific loss function is defined as follows: (6) in: It is the first Layer-specific graph output; , and Respectively The channels, height and width of the layer feature map; S224: The total loss of the watermark attack network is expressed as: (7) in: and is a hyperparameter that controls the weight of the loss function; The specific process of S3 is as follows: Fix the watermark attack network trained in the watermark attack network training phase and pass the watermarked image As input to the second stage; S31: First, the watermarked image Perform Deepfake to get the detection image ; S32: The watermarked image With detection image Input into the watermark attack network to obtain the unforged post-attack images And the forged attack image ; S33: Since the difference information cannot be easily detected, the amplified difference information is used for forgery detection, which is expressed as: (8) in: It means using Xception network to classify the difference image into true and false; Indicates the magnification factor of the difference image; The prediction result of the Xception network for the input; Using a simple BCE loss, the Xception network can be trained: (9) in: Indicates the label of the face, 0 represents true and 1 represents false; Indicates the result predicted by Xception.

2. The active Deepfake detection method based on watermark image difference according to claim 1, Features: The Unet network is a symmetrical U-shaped structure consisting of an encoder and a decoder. The encoder gradually extracts image features and reduces the spatial dimension by downsampling. The decoder gradually recovers the spatial dimensions and reconstructs the image by upsampling, while combining the feature information of the encoder through a residual connection.