A face forgery detection method and device based on frequency domain feature guidance

Abstract

The application discloses a face forgery detection method and device based on frequency domain feature guidance, and the method comprises the following steps: acquiring a face image to be detected; inputting the face image to be detected into a trained detection model to obtain a face forgery detection result; the detection model processing process comprises the following steps: in each space-frequency fusion unit, a spatial domain feature extraction module extracts spatial domain features from the face image or fusion features output by a previous unit; a frequency domain feature extraction module obtains frequency domain guidance features and corrected low-frequency-low-frequency subbands according to the face image or the fusion features output by the previous unit; a frequency domain perception module generates frequency domain attention features according to the frequency domain guidance features, and fuses the frequency domain attention features with the spatial domain features to obtain fusion features; and the fusion features output by the last space-frequency fusion unit are input into a classifier as classification features to obtain the face forgery detection result.

Description

Technical Field

[0001] This application relates to a face forgery detection method and apparatus based on frequency domain features, belonging to the field of image processing technology. Background Technology

[0002] Image-level face forgery detection techniques can be categorized into two types based on the feature domain of interest: spatial and frequency domains. However, feature analysis in either the spatial or frequency domain alone is insufficient for complex face forgery detection tasks. Although some existing methods attempt to fuse features from the spatial and frequency domains, they still suffer from problems such as simple model structure and weak feature extraction capabilities, significantly limiting the detection model's ability to discriminate complex and multi-faceted forged faces.

[0003] Therefore, it is crucial to invent a robust and efficient method that combines spatial and frequency domain techniques to improve the performance of face forgery detection. Summary of the Invention

[0004] Objective: In view of at least one of the above technical problems, this application provides a face forgery detection method and apparatus based on frequency domain features. The method starts from the spatial domain and frequency domain information of face images, and guides the model to more accurately discover potential forgery traces by continuously extracting implicit frequency domain features, so as to achieve robust and efficient face forgery detection.

[0005] The technical solution adopted in this application is as follows:

[0006] Firstly, this application provides a face forgery detection method based on frequency domain features, including:

[0007] Acquire the face image to be detected;

[0008] Input the face image to be detected into the trained detection model to obtain the face forgery detection result;

[0009] The detection model includes multiple spatial-frequency fusion units and a classifier connected in sequence. Each spatial-frequency fusion unit includes a spatial feature extraction module, a frequency domain feature extraction module, and a frequency domain sensing module. The processing includes:

[0010] In the first spatial-frequency fusion unit, the spatial domain feature extraction module extracts features from the face image to be detected in the spatial domain to obtain the first spatial domain feature; the frequency domain feature extraction module obtains the first frequency domain guiding feature and the first corrected low-frequency-low-frequency sub-band based on the face image to be detected; and the frequency domain perception module generates the first frequency domain attention feature based on the first frequency domain guiding feature, and fuses the first frequency domain attention feature with the first spatial domain feature to obtain the first fused feature.

[0011] In the t-th space-frequency fusion unit, the spatial domain feature extraction module extracts features from the (t-1)-th fused feature image in the spatial domain to obtain the t-th spatial domain feature; the frequency domain feature extraction module obtains the t-th frequency domain guiding feature and the t-th corrected low-frequency-low-frequency subband based on the (t-1)-th fused feature; the frequency domain perception module generates the t-th frequency domain attention feature based on the t-th frequency domain guiding feature, and fuses the t-th frequency domain attention feature with the t-th spatial domain feature to obtain the t-th fused feature; where... , This represents the total number of space-frequency fusion units;

[0012] The output of the last space-frequency fusion unit is the first The fused features are input into the classifier as classification features to obtain the face forgery detection results.

[0013] Secondly, this application provides a face forgery detection device based on frequency domain features, including a processor and a storage medium;

[0014] The storage medium is used to store instructions;

[0015] The processor is configured to operate according to the instructions to execute the method described in accordance with the first aspect.

[0016] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0017] Fourthly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0018] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0019] Beneficial Effects: The face forgery detection method and apparatus based on frequency domain feature guidance provided in this application have the following advantages: In this application, the spatial domain feature extraction module extracts features from the face image to be detected in the spatial domain, uncovers forgery traces, and, combined with the "global-local" attention fusion module, further outputs features with stronger forgery detection capabilities; the frequency domain feature extraction module, with the help of discrete wavelet transform and SE attention mechanism, efficiently extracts possible forgery traces in the high-frequency domain of the image; the frequency domain perception module utilizes the high-frequency information of the frequency domain guidance features to enhance the spatial domain feature representation capability and accurately discover potential forgery traces. This application avoids simple classification of a single spatial domain or a single frequency domain, and makes full use of the high-frequency information contained in the HH, HL, and LH subbands. On the one hand, it guides the feature generation in the spatial domain, and on the other hand, it corrects the information in the LL subband for use in the next stage of frequency domain feature extraction, thereby achieving robust and efficient face forgery detection. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a face forgery detection method based on frequency domain features according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a face forgery detection model according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the spatial feature extraction module according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of the "global-local" attention fusion module according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the frequency domain feature extraction module according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the frequency domain sensing module according to an embodiment of this application. Detailed Implementation

[0026] The present application will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and should not be used to limit the scope of protection of the present application.

[0027] In the description of this application, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0028] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0029] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0030] Example 1: This example provides a face forgery detection method based on frequency domain features, such as... Figure 1 As shown, it includes:

[0031] Acquire the face image to be detected;

[0032] Input the face image to be detected into the trained detection model to obtain the face forgery detection result;

[0033] Among them, such as Figure 2 As shown, the detection model includes multiple ( ) connected in sequence. There are 10 spatial-frequency fusion units and a classifier. Each spatial-frequency fusion unit includes a spatial feature extraction module, a frequency domain feature extraction module, and a frequency domain sensing module. The processing includes:

[0034] In the first spatial-frequency fusion unit, the spatial domain feature extraction module extracts features from the face image to be detected in the spatial domain to obtain the first spatial domain feature; the frequency domain feature extraction module obtains the first frequency domain guiding feature and the first corrected low-frequency-low-frequency sub-band based on the face image to be detected; and the frequency domain perception module generates the first frequency domain attention feature based on the first frequency domain guiding feature, and fuses the first frequency domain attention feature with the first spatial domain feature to obtain the first fused feature.

[0035] In the t-th space-frequency fusion unit, the spatial domain feature extraction module extracts features from the (t-1)-th fused feature image in the spatial domain to obtain the t-th spatial domain feature; the frequency domain feature extraction module obtains the t-th frequency domain guiding feature and the t-th corrected low-frequency-low-frequency subband based on the (t-1)-th fused feature; the frequency domain perception module generates the t-th frequency domain attention feature based on the t-th frequency domain guiding feature, and fuses the t-th frequency domain attention feature with the t-th spatial domain feature to obtain the t-th fused feature; where... , This represents the total number of space-frequency fusion units;

[0036] The output of the last space-frequency fusion unit is the first The fused features are input into the classifier as classification features to obtain the face forgery detection results.

[0037] In some embodiments, the input to the spatial feature extraction module is ,in These represent the feature height, feature width, and number of basic feature channels input to the t-th spatial feature extraction module, respectively; when At that time, input the face image to be detected. ;when At that time, input the fused feature at the (t-1)th time. .

[0038] like Figure 3 As shown, the spatial feature extraction module includes a first downsampling module consisting of a convolutional layer, a normalization layer, an activation layer, a zero-padding layer, a convolutional layer, a normalization layer, an activation layer, a pooling layer, a normalization layer, a convolutional layer, a pooling layer, a normalization layer, and a convolutional layer, as well as a global-local attention fusion module.

[0039] Specifically, the first downsampling module downsamples the input of the spatial feature extraction module to obtain high-dimensional features. .

[0040] like Figure 4 As shown, the "global-local" attention fusion module performs size and channel number alignment operations on the input of the spatial feature extraction module (through pooling layers and convolutional layers) to obtain aligned low-dimensional features. Aligned low-dimensional features With high-dimensional features After channel concatenation, convolutional layers, pooling layers, and activation layers, attention features are obtained; the aligned low-dimensional features are then processed... High-dimensional features and attention characteristics The spatial features obtained by fusing through the attention fusion module are represented as follows:

[0041]

[0042] In the formula, Indicates spatial characteristics, This represents the aligned low-dimensional features. Representing high-dimensional features, Indicating attentional characteristics, This indicates element-wise multiplication.

[0043] In some embodiments, such as Figure 5 As shown, the input to the frequency domain feature extraction module is ,in These represent the feature height, feature width, and number of basic feature channels input to the t-th spatial feature extraction module, respectively; when At that time, input the face image to be detected. ;when At that time, input the (t-1)th corrected low-frequency subband. ;

[0044] The input of the frequency domain feature extraction module is subjected to discrete wavelet transform, decomposing it into low-frequency-low-frequency sub-bands, high-frequency-high-frequency sub-bands, high-frequency-low-frequency sub-bands, and low-frequency-high-frequency sub-bands. The high-frequency-high-frequency sub-bands are downsampled, squeezed, and activated by the SE attention mechanism, and then added to the high-frequency-low-frequency sub-bands and low-frequency-high-frequency sub-bands that have been processed by deformable convolution to obtain frequency domain guided features. The frequency domain guided features are processed by convolutional layers and activation layers to generate low-frequency corrected features. The low-frequency-low-frequency sub-bands are corrected using the low-frequency-low-frequency corrected features (guiding the low-frequency-low-frequency sub-bands to focus on the parts that are more likely to have forgery traces) to obtain the corrected low-frequency-low-frequency sub-bands.

[0045] Furthermore, the discrete wavelet transform is expressed as:

[0046] ;

[0047] ;

[0048] Wherein, LL represents low-frequency to low-frequency sub-band, LH represents low-frequency to high-frequency sub-band, HL represents high-frequency to low-frequency sub-band, and HH represents high-frequency to high-frequency sub-band. , , , These represent low-frequency-low-frequency subband filters, low-frequency-high-frequency subband filters, high-frequency-low-frequency subband filters, and high-frequency-high-frequency subband filters, respectively. This represents the convolution operation. This represents the input to the frequency domain feature extraction module.

[0049] The squeeze and activation SE attention mechanism consists of a pooling layer, a fully connected layer, an activation layer, another fully connected layer, and an activation layer connected in sequence. The pooling layer compresses the input features into a shape... The vector, where The number of feature channels is represented; then, through two channel-aware structures consisting of fully connected layers and activation layers, channel attention features containing channel weight coefficients are obtained to realize key channel feature perception; finally, the channel attention features are multiplied with the input features to output the final features with channel attention.

[0050] In some embodiments, such as Figure 6 As shown, in the frequency domain perception module, the frequency domain guided features are processed by a second downsampling module consisting of a convolutional layer, a normalization layer, and an activation layer to generate frequency domain attention features. After the frequency domain attention features and spatial features are concatenated, they are processed by a convolutional layer, a pooling layer, and an activation layer to generate fused features.

[0051] In some embodiments, the classifier includes one fully connected layer; the input size of the fully connected layer is the size of the classification feature; the output size of the fully connected layer is 2, representing the probability of predicting whether the face image is real or fake.

[0052] Furthermore, in some embodiments, the training method of the detection model includes the following steps:

[0053] Obtain a training set, which includes real and face image data generated by existing forgery techniques, and corresponding real labels, wherein the real labels take values ​​of 0 and 1, where 0 represents a real face and 1 represents a forged face;

[0054] In this embodiment, the training set can be obtained by directly downloading a public dataset, which may include FaceForensics++ (FF++), DeepFake Detection Challenge (DFDC), etc.

[0055] The real and face image data generated by existing forgery techniques are input into a pre-built detection model to obtain the model prediction results;

[0056] The cross-entropy loss function is calculated based on the model prediction results of the face image and the corresponding ground truth label.

[0057] In this embodiment, the cross-entropy loss function The expression is as follows:

[0058]

[0059] In the formula, The true label representing the input face image. This represents the model's prediction result for the input face image.

[0060] The detection model is iteratively updated and trained using the gradient descent method, and the detection model with the smallest cross-entropy loss function is taken as the trained detection model.

[0061] In this embodiment, based on the Adam optimizer, for each training batch, the cross-entropy loss function is first calculated using forward propagation. Then, backpropagation is used to calculate the partial derivatives of each weight parameter, and finally, common gradient descent methods (such as stochastic gradient descent) are used to update the weight parameters. This process is repeated until the cross-entropy loss function reaches its minimum, resulting in a trained detection model.

[0062] Example 2: Based on Example 1, this example provides a face forgery detection device guided by frequency domain features, including a processor and a storage medium;

[0063] The storage medium is used to store instructions;

[0064] The processor is configured to operate according to the instructions to execute the method according to Embodiment 1.

[0065] Example 3: Based on Example 1, this example provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Example 1.

[0066] Example 4: Based on Example 1, this example provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in Example 1.

[0067] Example 5: Based on Example 1, this example provides a computer program product, including a computer program that, when executed by a processor, implements the method described in Example 1.

[0068] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A face forgery detection method based on frequency domain features, characterized in that, include: Acquire the face image to be detected; Input the face image to be detected into the trained detection model to obtain the face forgery detection result; The detection model includes multiple spatial-frequency fusion units and a classifier connected in sequence. Each spatial-frequency fusion unit includes a spatial feature extraction module, a frequency domain feature extraction module, and a frequency domain sensing module. The processing includes: In the first spatial-frequency fusion unit, the spatial domain feature extraction module extracts features from the face image to be detected in the spatial domain to obtain the first spatial domain feature; the frequency domain feature extraction module obtains the first frequency domain guiding feature and the first corrected low-frequency-low-frequency sub-band based on the face image to be detected; and the frequency domain perception module generates the first frequency domain attention feature based on the first frequency domain guiding feature, and fuses the first frequency domain attention feature with the first spatial domain feature to obtain the first fused feature. In the t-th space-frequency fusion unit, the spatial domain feature extraction module extracts features from the (t-1)-th fused feature image in the spatial domain to obtain the t-th spatial domain feature; the frequency domain feature extraction module obtains the t-th frequency domain guiding feature and the t-th corrected low-frequency-low-frequency subband based on the (t-1)-th fused feature; the frequency domain perception module generates the t-th frequency domain attention feature based on the t-th frequency domain guiding feature, and fuses the t-th frequency domain attention feature with the t-th spatial domain feature to obtain the t-th fused feature; where... , This represents the total number of space-frequency fusion units; The output of the last space-frequency fusion unit is the first The fused features are input into the classifier as classification features to obtain the face forgery detection results; The input to the frequency domain feature extraction module is... ,in These represent the feature height, feature width, and number of basic feature channels input to the t-th frequency domain feature extraction module, respectively. The total number of space-frequency fusion units; when At that time, input the face image to be detected. ;when At that time, input the (t-1)th corrected low-frequency sub-band. ; The input of the frequency domain feature extraction module is subjected to discrete wavelet transform, decomposing it into low-frequency-low-frequency sub-band, high-frequency-high-frequency sub-band, high-frequency-low-frequency sub-band, and low-frequency-high-frequency sub-band. The high-frequency-high-frequency sub-band is downsampled, squeezed, and activated by an attention mechanism, and then added to the high-frequency-low-frequency sub-band and the low-frequency-high-frequency sub-band, which have been processed by deformable convolution, respectively, to obtain the frequency domain guided features. The frequency domain guided features are processed by convolutional layers and activation layers to generate low-frequency correction features. The low-frequency-low-frequency sub-band is then corrected using the low-frequency correction features to obtain the corrected low-frequency-low-frequency sub-band.

2. The method according to claim 1, characterized in that, The input to the spatial feature extraction module is ,in These represent the feature height, feature width, and number of basic feature channels input to the t-th spatial feature extraction module, respectively. The total number of space-frequency fusion units; when At that time, input the face image to be detected. ;when At time t-1, input the fused features ; The spatial feature extraction module includes a first downsampling module and a "global-local" attention fusion module. The first downsampling module downsamples the input of the spatial feature extraction module to obtain high-dimensional features. The "global-local" attention fusion module performs size and channel number alignment operations on the input of the spatial feature extraction module to obtain aligned low-dimensional features. The aligned low-dimensional features and high-dimensional features are then concatenated through channels to obtain attention features. The aligned low-dimensional features, high-dimensional features, and attention features are fused by the attention fusion module to obtain spatial features.

3. The method according to claim 2, characterized in that, The aligned low-dimensional features, high-dimensional features, and attention features are fused by the attention fusion module to obtain the spatial domain features, represented as: ， In the formula, Indicates spatial characteristics, This represents the aligned low-dimensional features. Representing high-dimensional features, Indicating attentional characteristics, This indicates element-wise multiplication.

4. The method according to claim 1, characterized in that, The data processing procedure of the frequency domain sensing module includes: The frequency domain guided features are processed by a second downsampling module consisting of convolutional layers, normalization layers, and activation layers to generate frequency domain attention features. After channel concatenation of the frequency domain attention features and spatial features, they are further processed by convolutional layers, pooling layers, and activation layers to generate fused features.

5. The method according to claim 1, characterized in that, The classifier contains one fully connected layer.

6. The method according to claim 5, characterized in that, The input size of the fully connected layer is the size of the classification feature; the output size of the fully connected layer is 2, which respectively represent the probability of predicting whether the face image is real or fake.

7. A face forgery detection device based on frequency domain features, characterized in that, Including processor and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 6.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 6.

Images