Face deepfake detection system and method based on image reconstruction

Abstract

The present application relates to a kind of face depth forgery detection system and method based on image reconstruction, belong to image processing technical field, solve the generalization of lower forgery detection in prior art, the modeling effect of the distribution of real image in high-dimensional space space and high-dimensional frequency domain space is poor.The present application passes through space reconstruction module, utilize the way of distillation learning, effectively utilize the prior distribution information of real face, so that network better models the commonality of real face in space domain.In addition, by frequency domain reconstruction module, the commonality of real image in frequency domain space is fully modeled, and the detection effect of space reconstruction module is improved.In addition, by fine-grained space-frequency conversion module, fine-grained space-frequency conversion is realized, and after channel rearrangement, the network better captures the frequency domain features of different frequency bands.The present application improves the generalization of forgery detection, and the modeling of the distribution of real image in high-dimensional space space and high-dimensional frequency domain space is sufficient.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a face deepfake detection system and method based on image reconstruction. Background Technology

[0002] Deepfake technology for faces includes face replacement, expression modification, and attribute editing. With the support of deep learning technology and the gradual improvement of synthesis technology, the difficulty of obtaining such forgeries has been greatly reduced. More and more fake images have appeared on the Internet, and the proliferation of these false information has had a great negative impact on the politics and security of society.

[0003] Currently, several methods exist for detecting deepfake faces. Generally, these methods are trained in a supervised manner on specific forgery datasets, achieving good detection results against specific forgery methods. However, with the development of deepfake technology, more and more new forgery methods have emerged, and supervised learning-based forgery detection techniques perform poorly on these unknown forgery methods, highlighting the need to improve the generalization ability of forgery detection. Furthermore, existing supervised learning-based forgery detection methods suffer from two main problems: first, these methods do not fully utilize the prior distribution information of real face images, failing to effectively model the distribution of real images in the high-dimensional spatial domain; second, these methods sometimes fail to fully or reasonably utilize frequency domain information, also failing to effectively model the distribution of real images in the high-dimensional frequency domain.

[0004] In summary, existing technologies suffer from low generalization in forgery detection and poor modeling of the distribution of real images in high-dimensional spatial and frequency domains. Summary of the Invention

[0005] In view of the above problems, the present invention provides a face deepfake detection system and method based on image reconstruction, which solves the problems of low generalization of forgery detection and poor modeling effect for the distribution of real images in high-dimensional spatial and high-dimensional frequency domains in the prior art.

[0006] This invention provides a face deepfake detection system based on image reconstruction, comprising: a spatial domain reconstruction module, a frequency domain reconstruction module, and a spatial-frequency domain feature fusion module, wherein...

[0007] The spatial domain reconstruction module is a convolutional neural network, used to extract the features to be distilled and intermediate features of the face image to be tested, as well as high-dimensional spatial domain features; it is also used to obtain the reconstructed spatial domain image based on the fused features.

[0008] The frequency domain reconstruction module is a convolutional neural network, used to extract the original frequency domain features of the face image to be tested, and process them to obtain high-dimensional frequency domain features; it is also used to obtain reconstructed frequency domain features based on fused features;

[0009] The spatial-frequency domain feature fusion module is used to fuse the high-dimensional spatial features and the high-dimensional frequency domain features to obtain fused features.

[0010] Furthermore, the spatial domain reconstruction module includes a teacher network module, which is a network pre-trained for a face recognition task. This network is used to process the first original spatial domain features of the face image to be tested, and to obtain multiple different levels of features to be distilled from the face image to be tested.

[0011] Furthermore, the spatial domain reconstruction module also includes a spatial domain encoder module. The spatial domain encoder module is a convolutional neural network with a deep separable convolutional network as its backbone. It is used to process the first original spatial domain features of the face image to be tested, and obtain multiple intermediate features at different levels of the face image to be tested, as well as the encoded high-dimensional spatial domain features.

[0012] Furthermore, the spatial domain reconstruction module also includes a spatial domain decoder module, which processes the fused features output by the spatial-frequency domain feature fusion module to obtain a reconstructed spatial domain image.

[0013] Furthermore, the frequency domain reconstruction module includes a fine-grained space-frequency conversion module, which is used to convert the second original spatial domain features from RGB space to YCbCr space, extract the luminance channel to obtain luminance features, process the luminance features using a sliding window method to obtain the frequency domain features without channel rearrangement, and obtain the original frequency domain features after channel rearrangement.

[0014] Furthermore, the frequency domain reconstruction module also includes a frequency domain encoder module, which processes the original frequency domain features to obtain the high-dimensional frequency domain features encoded from the image under test.

[0015] Furthermore, the frequency domain reconstruction module also includes a frequency domain decoder module, which processes the fused features output by the spatial frequency domain feature fusion module to obtain reconstructed frequency domain features.

[0016] This invention also provides a face deepfake detection method based on image reconstruction, implemented based on the image reconstruction-based face deepfake detection system, comprising the following steps, wherein steps S1 to S11 are the training phase, step S12 is the testing phase, and step S13 is the usage phase:

[0017] S1: Construct a dataset of real face images as the training set, and construct a dataset including real face images and fake face images as the test set; both the training set and the test set have image-level annotations.

[0018] S2: Scale the real face images in the training set obtained in step S1 to obtain the first original spatial feature F. ori_spa The data is input into the teacher network module for processing to obtain n different levels of features to be distilled from the real face image. The size of the feature to be distilled at the kth level is H. k ×W k ×C k ;

[0019] S3: Scale the real face image obtained in step S1 to obtain the first original spatial feature F. ori_spa The data is input into the spatial encoder module for processing to obtain n different levels of intermediate features from the real face image. and the encoded high-dimensional spatial features F spatial The size of the intermediate feature at the k-th level is H. k ×W k ×C k ;

[0020] S4: Calculate and obtain the distillation characteristics obtained in step S2. and the intermediate features obtained in step S3 Distillation loss between ditill ;

[0021] S5: Scale the real face image obtained in step S1 to obtain the second original spatial domain feature, and input it into the fine-grained space-frequency conversion module for processing to obtain the original frequency domain feature F. ori_fre ;

[0022] S6: Obtain the original frequency domain features F obtained in step S5. ori_fre The input is processed by the frequency domain encoder module to obtain the encoded high-dimensional frequency domain feature Ff. requency ;

[0023] S7: Obtain the high-dimensional spatial features F obtained in step S3. spatial And the high-dimensional frequency domain feature F obtained in step S6 frequency The input is fed into the spatial frequency domain feature fusion module for processing, and the fused feature F is obtained through a self-attention mechanism.

[0024] S8: Obtain the fused feature F obtained in step S7, input it into the spatial domain decoder module for processing, and obtain the reconstructed spatial domain image F. recon_spa ;

[0025] S9: Obtain the fused features obtained in step S7, input them into the frequency domain decoder module for processing, and obtain the reconstructed frequency domain features F. recon_fre ;

[0026] S10: The first original spatial feature F obtained based on step S2 ori_spa The reconstructed spatial features F obtained in step S8 recon_spa The original frequency domain features F obtained in step S5 ori_fre The reconstructed frequency domain features F obtained in step S9 recon_fre and the distillation loss obtained in step S4 ditill The spatial reconstruction loss is calculated. spa Frequency domain reconstruction loss fre The final total loss is obtained. total This serves as the basis for determining the authenticity of an image;

[0027] S11: Repeat steps S2 through S8 until the total loss is reached. total Convergence, complete training and save the trained spatial encoder module, frequency encoder module, spatial-frequency feature fusion module, spatial decoder module and frequency decoder module;

[0028] S12: In the testing phase, replace the images in the training set with the images in the test set, and execute steps S2 to S10 to complete the verification and testing of the image reconstruction-based deep face forgery detection method for forged face images.

[0029] S13: In the usage phase, the image of the face to be tested is replaced with the image in the test set, and steps S2 to S10 are executed to complete the deep face forgery detection based on image reconstruction for the face image to be tested.

[0030] Furthermore, the real face image obtained in step S1 is scaled to obtain a second original spatial domain feature, which is then input into a fine-grained space-frequency conversion module for processing to obtain the original frequency domain feature F. ori_fre ,include:

[0031] S5-1: Scale the real face image obtained in step S1 to obtain the second original spatial domain features;

[0032] S5-2: Transform the second original spatial feature from RGB space to YCbCr space, and extract the pixel value of the luminance channel, i.e., the luminance feature F. y ;

[0033] S5-3: Obtain the brightness feature F obtained in step S5-2 y A sliding window approach is used to analyze the brightness feature F. y The processing is performed with a window size of l*l. Then, DCT transformation is performed on each image block of size l*l after window processing, with a sliding step size of s, to obtain the frequency domain features without channel rearrangement.

[0034] S5-4: Obtain the un-channel rearranged frequency domain features obtained in step S5-3, and rearrange them according to the frequency band size so that features in the same frequency band are in the same channel after rearrangement, thus obtaining the original frequency domain features F. ori_fre .

[0035] Furthermore, the distillation loss Loss ditill The calculation is performed as follows:

[0036]

[0037] The spatial domain reconstruction loss, frequency domain reconstruction loss, and total loss are calculated as follows:

[0038]

[0039]

[0040] Loss total =αLoss distii +βLoss spa +γLoss fre ;

[0041] in, This represents the value of the original spatial feature at coordinate positions i and j. This represents the value of the reconstructed spatial feature at coordinate positions i and j. This represents the value of the original frequency domain feature at coordinate positions i and j. This represents the value of the reconstructed frequency domain feature at coordinate positions i and j; α, β, and γ are loss balancing factors used to balance the weights among the various losses.

[0042] Compared with the prior art, the present invention has at least the following beneficial effects:

[0043] (1) The spatial reconstruction module of the present invention utilizes distillation learning to effectively utilize the prior distribution information of real face data, enabling the network to better model the commonalities of real faces in the spatial domain.

[0044] (2) The frequency domain reconstruction module of the present invention can fully model the commonalities of real images in the frequency domain space, and assist the spatial domain module in improving the detection effect.

[0045] (3) The fine-grained space-frequency conversion module of the present invention realizes fine-grained space-frequency conversion, and through channel rearrangement, the network can better capture the frequency domain characteristics of different frequency bands.

[0046] (4) The face deep forgery detection system and method based on image reconstruction invented in this invention regards forgery detection as an anomaly detection task. Only real face images are used in the training stage, which can better model the high-dimensional features of real face images, so as not to overfit to certain specific forgery methods and improve the generalization of the model. Attached Figure Description

[0047] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0048] Figure 1 This is a schematic diagram of a face depth forgery detection system based on image reconstruction according to an embodiment of the present invention.

[0049] Figure 2 This is a flowchart of the training process of a face depth forgery detection method based on image reconstruction according to an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the spatial reconstruction module structure and process in an image reconstruction-based face depth forgery detection system according to an embodiment of the present invention. Detailed Implementation

[0051] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0052] Example 1

[0053] A specific embodiment of the present invention, such as Figure 1 As shown, a face deepfake detection system based on image reconstruction is disclosed, including: a spatial domain reconstruction module, a frequency domain reconstruction module, and a spatial-frequency domain feature fusion module; wherein,

[0054] The spatial domain reconstruction module includes a teacher network module, a spatial encoder module, and a spatial decoder module; the frequency domain reconstruction module includes a fine-grained space-frequency conversion module, a frequency encoder module, and a frequency decoder module.

[0055] Spatial domain reconstruction module, such as Figure 3As shown, the model constructs the distribution of the test face image in a high-dimensional spatial feature space and effectively utilizes prior information about the distribution of real face data through distillation learning. Specifically, the spatial reconstruction module is used to extract the distillation features and intermediate features of the test face image at multiple different levels, as well as the encoded high-dimensional spatial features, based on the first original spatial features of the test face image; it is also used to process the fused features output by the spatial-frequency domain feature fusion module to obtain the reconstructed spatial image.

[0056] The teacher network module uses a network pre-trained on large-scale face recognition tasks to extract features from the test face image at different levels. These features are then used to guide the training of the spatial encoder through subsequent distillation learning. Specifically, the teacher network module processes the first raw spatial features of the test face image to obtain n different levels of features to be distilled from the test face image. The size of the feature to be distilled at the kth level is H. k ×W k ×C k Where H represents the image height, W represents the image width, and C represents the number of image channels. The teacher network uses a convolutional neural network to extract features from the input image, mainly utilizing convolutional layers for upsampling and local feature extraction, and using the ReLU activation function to improve the network's nonlinear fitting ability.

[0057] The spatial encoder module, also known as the student network module, learns the features extracted by the teacher network module through distillation learning. It effectively utilizes the prior distribution information of real face data contained in the teacher network to encode the test face image, obtaining its high-dimensional spatial features. Specifically, the spatial encoder module processes the first raw spatial features of the test face image to obtain n intermediate features at different levels. and the encoded high-dimensional spatial features F spatial The size of the intermediate feature at the k-th level is H. k ×W k ×C k Similar to the teacher network, the spatial encoder also mainly uses convolutional layers and ReLU activation functions for feature extraction, but the number of convolutional kernels used is fixed and small, and the number of convolutional layers is also small. This is to reduce the model's parameters and improve the speed of training and inference.

[0058] The spatial domain decoder module reconstructs the face image under test based on the fused features output by the spatial-frequency domain feature fusion module, obtaining the reconstructed face image. Specifically, the spatial domain decoder module processes the fused features output by the spatial-frequency domain feature fusion module to obtain the reconstructed spatial domain image. The spatial domain decoder module mainly uses deconvolution layers to progressively upsample the input features, ultimately obtaining a reconstructed spatial domain image of the same size as the image under test.

[0059] The frequency domain reconstruction module transforms the test face image from the spatial domain to the frequency domain, modeling the distribution of the test face image in the high-dimensional frequency domain feature space. Specifically, the frequency domain reconstruction module is used to convert the second original spatial domain features of the test face image into original frequency domain features, which are then processed to obtain the high-dimensional frequency domain features encoded by the image; it is also used to process the fused features output by the spatial-frequency domain feature fusion module to obtain the reconstructed frequency domain features.

[0060] The fine-grained space-frequency conversion module transforms the face image under test from the spatial domain to the frequency domain, obtaining the original frequency domain feature map. Specifically, the module transforms the second original spatial domain features from RGB space to YCbCr space, where chroma and luminance are decoupled. Then, it extracts the luminance channel to obtain luminance features. A sliding window is used to process the luminance features, resulting in un-channel rearranged frequency domain features. After channel rearrangement, the original frequency domain features are obtained. During the space-frequency domain conversion using the sliding window, a fixed window size k is used to perform a Discrete Fourier Transform on each k×k block to obtain the frequency domain features. Channel rearrangement places features in the same frequency band into the same channel after transformation, ensuring a spatial correspondence between the obtained original frequency domain features and the image under test, which is beneficial for subsequent feature extraction using convolutional neural networks.

[0061] The frequency domain encoder module encodes the original frequency domain features output by the fine-grained spatial-frequency domain transformation module into a high-dimensional frequency domain feature space. Specifically, the frequency domain encoder module processes the original frequency domain features to obtain the encoded high-dimensional frequency domain features of the image under test. The frequency domain encoder uses a convolutional neural network to extract features from the input image, primarily utilizing convolutional layers for upsampling and local feature extraction, and employing the ReLU activation function to enhance the network's nonlinear fitting capability.

[0062] The frequency domain decoder module reconstructs the original frequency domain feature map based on the fused features output by the spatial frequency domain feature fusion module, resulting in a reconstructed frequency domain feature map. Specifically, the frequency domain decoder module processes the fused features output by the spatial frequency domain feature fusion module to obtain the reconstructed frequency domain features. The frequency domain decoder module primarily uses deconvolution layers to progressively upsample the input features, ultimately obtaining reconstructed frequency domain features of the same size as the original frequency domain features.

[0063] The spatial-frequency domain feature fusion module is used to fuse the high-dimensional spatial features extracted by the spatial encoder module and the high-dimensional frequency features extracted by the frequency encoder module to obtain fused features. The spatial-frequency domain feature fusion module mainly uses a self-attention mechanism for adaptive feature fusion.

[0064] In an optional implementation, both the spatial domain reconstruction module and the frequency domain reconstruction module employ convolutional neural networks.

[0065] In an optional implementation, the teacher network module can select 3-5 different levels of features to be distilled. To shorten training time and reduce computational load, three levels of features to be distilled can be selected, with sizes of 64×64, 32×32, and 16×16, respectively.

[0066] In an optional implementation, the face image extracted by the teacher network module, spatial encoder module, and fine-grained spatial-frequency domain conversion module can be a square size between 256×256 and 512×512 pixels. To ensure extraction quality and reduce computation, 256×256 pixels can be selected.

[0067] Compared with existing technologies, the image reconstruction-based deepfake face detection system provided by the present invention effectively utilizes prior distribution information of real face data through a spatial domain reconstruction module using distillation learning, enabling the network to better model the commonalities of real faces in the spatial domain. Furthermore, the frequency domain reconstruction module can fully model the commonalities of real images in the frequency domain, assisting the spatial domain reconstruction module in improving detection performance. Additionally, a fine-grained space-frequency conversion module achieves fine-grained space-frequency conversion, and channel rearrangement allows the network to better capture frequency domain features of different frequency bands.

[0068] Example 2

[0069] A specific embodiment of the present invention, such as Figure 2 As shown, a face deepfake detection method based on image reconstruction is disclosed, which is implemented based on the detection system in Embodiment 1.

[0070] Specifically, it includes the following steps S1 to S12, where steps S1 to S11 are the training phase, step S12 is the testing phase, and step S13 is the usage phase:

[0071] S1: Construct a dataset containing only real face images as the training set, and construct a dataset containing real face images and fake face images synthesized using deepfake technology as the test set; both the training set and the test set have image-level annotations, in which 1 represents a fake face image and 0 represents a real face image.

[0072] S2: Scale the real face images in the training set obtained in step S1 to obtain the first original spatial feature F. ori_spa The data is input into the teacher network module for processing to obtain n different levels of features to be distilled from the real face image. k = 1, 2, 3...n; where the size of the distillation feature of the k-th level is H. k ×W k ×C k Optionally, the number of levels of the feature to be distilled can be set to 3.

[0073] S3: Scale the real face image obtained in step S1 to obtain the first original spatial feature F. ori_spa The data is input into the spatial encoder module for processing to obtain n different levels of intermediate features from the real face image. and the encoded high-dimensional spatial features F spatial The size of the intermediate feature at the k-th level is H. k ×W k ×C k Intermediate features refer to the outputs of each intermediate layer of the neural network, and encoded high-dimensional spatial features refer to the outputs of the last layer of the neural network. Optionally, the number of layers of the intermediate features to be distilled can be set to 3.

[0074] S4: Calculate and obtain the distillation characteristics obtained in step S2. and the intermediate features obtained in step S3 Distillation loss between ditill .

[0075] S5: Scale the real face image obtained in step S1 to obtain the second original spatial domain feature, and input it into the fine-grained space-frequency conversion module for processing to obtain the original frequency domain feature F. ori_fre Specifically, this includes the following steps S5-1 to S5-4. It is worth noting that the scaled real face image here is the same image as the real face images in steps S2 and S3, but the scaled second original spatial feature and the first original spatial feature F... ori_spa They differ in size.

[0076] S5-1: Scale the real face image obtained in step S1 to obtain the second original spatial features.

[0077] S5-2: The second original spatial feature is transformed from the RGB space to the YCbCr space where chroma and luminance are decoupled. Then, only the luminance channel, i.e., the pixel value of the Y channel, is extracted as the subsequent input, i.e., the luminance feature F. y .

[0078] S5-3: Obtain the brightness feature F obtained in step S5-2 y A sliding window approach is used to analyze the brightness feature F. y The processing is performed with a window size of l*l. Then, a DCT (Discrete Cosine Transform) is performed on each image block of size l*l after the window processing, with a sliding step size of s, to obtain the frequency domain features without channel rearrangement. Optionally, the sliding window size l is set to 8 and the sliding step size s is set to 2 to obtain better results.

[0079] S5-4: Obtain the un-channel rearranged frequency domain features obtained in step S5-3, and rearrange them according to the bandwidth size so that features in the same bandwidth are in the same channel after rearrangement, thus obtaining the rearranged frequency domain features, i.e., the original frequency domain features F. ori_fre .

[0080] S6: Obtain the original frequency domain features F obtained in step S5-4 ori_fre The input is processed by the frequency domain encoder module to obtain the encoded high-dimensional frequency domain feature F. frequency .

[0081] S7: Obtain the high-dimensional spatial features F obtained in step S3. spatial And the high-dimensional frequency domain feature F obtained in step S6 frequency The input is fed into the spatial frequency domain feature fusion module for processing, and the fused feature F is obtained through the self-attention mechanism.

[0082] S8: Obtain the fused feature F obtained in step S7, input it into the spatial domain decoder module for processing, and obtain the reconstructed spatial domain image F. recon_spa .

[0083] S9: Obtain the fused features obtained in step S7, input them into the frequency domain decoder module for processing, and obtain the reconstructed frequency domain features F. recon_fre .

[0084] S10: The first original spatial feature F obtained based on step S2 ori_spa The reconstructed spatial features F obtained in step S8 recon_spa The original frequency domain features F obtained in step S5 ori_fre The reconstructed frequency domain features F obtained in step S9 recon_fre and the distillation loss obtained in step S4 ditill The spatial reconstruction loss is calculated. spa Frequency domain reconstruction loss fre The final total loss is obtained. total This serves as the basis for determining the authenticity of an image;

[0085] S11: Repeat steps S2 through S8 until the total loss is reached.total Convergence, complete training and save the trained spatial encoder module, frequency encoder module, spatial-frequency feature fusion module, spatial decoder module and frequency decoder module;

[0086] S12: In the testing phase, replace the images in the training set with the images in the test set, execute steps S2 to S10, calculate the relevant detection indicators for the testing phase based on the results of step S10, and complete the verification and testing of the face depth forgery detection method based on image reconstruction for forged face images; among them, the relevant detection indicators for the testing phase include the AUC (Area Under Curve) indicator.

[0087] S13: In the usage phase, the image of the face to be tested is replaced with the image in the test set, and steps S2 to S10 are executed. Based on the result of step S10, the relevant detection indicators for the usage phase are calculated to complete the face depth forgery detection based on image reconstruction for the face image to be tested; among which, the relevant detection indicators for the usage phase include the AUC indicator.

[0088] In an optional implementation, the network selected for the teacher network module in step S2 is a convolutional neural network with a 50-layer convolutional network backbone that has been pre-trained on the widerface face detection dataset, and the spatial encoder module in step S3 is a convolutional neural network with a depth-separable convolutional network backbone.

[0089] In an optional implementation, the distillation loss in step S4 is... ditill The calculation is performed using the following formula:

[0090]

[0091] In an optional implementation, the spatial domain reconstruction loss and frequency domain reconstruction loss in step S10 are both obtained using the MSE (Mean Squared Error) loss function, and the total loss is obtained by weighted summation of the various losses.

[0092]

[0093]

[0094] Loss total =αLoss distii +βLoss spa +γLoss fre

[0095] in, This represents the value of the original spatial feature at coordinate positions i and j. This represents the value of the reconstructed spatial feature at coordinate positions i and j. This represents the value of the original frequency domain feature at coordinate positions i and j. This represents the value of the reconstructed frequency domain feature at coordinate positions i and j; α, β, and γ are loss balancing factors used to balance the weights among the various losses.

[0096] The following details the verification process of the image reconstruction-based deepfake detection method for faces provided according to an embodiment of the present invention, including training and testing. To verify the effectiveness and practicality of the present invention, a subset of real faces from the FaceForensic++ dataset (Face Forensics++ dataset) was used as the training set, comprising 30,000 images. The model was trained according to steps S1-S11 described above, using Adam (Adaptive Momentum Estimation Optimization Algorithm) as the optimizer, with a learning rate set to 0.0005. A total of 100 iterations were performed, and the model from the last iteration was saved as the final result.

[0097] The model was tested using FaceForensic++ (Face Forensics++ dataset). The test set contained 750 real face images and 760 fake face images. The IDs (identities) of the real face images were the same as those in the training set, and the attacked IDs (identities) of the fake images were the same as those in the training set. The trained model was tested according to step S12 above. The AUC index reached 86.01%, which indicates that a high level of results was obtained by the method provided by the embodiment of the present invention, demonstrating that the present invention is efficient and feasible.

[0098] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A face deepfake detection system based on image reconstruction, characterized in that, include: The module comprises a spatial domain reconstruction module, a frequency domain reconstruction module, and a spatial-frequency domain feature fusion module. The spatial domain reconstruction module is a convolutional neural network, used to extract the features to be distilled and intermediate features of the face image to be tested, as well as high-dimensional spatial domain features; it is also used to obtain the reconstructed spatial domain image based on the fused features. The frequency domain reconstruction module is a convolutional neural network, used to extract the original frequency domain features of the face image to be tested, and process them to obtain high-dimensional frequency domain features; it is also used to obtain reconstructed frequency domain features based on fused features; The spatial-frequency domain feature fusion module is used to fuse the high-dimensional spatial features and the high-dimensional frequency domain features to obtain fused features; The spatial domain reconstruction module includes: a teacher network module, which is a network pre-trained for a face recognition task, used to process the first original spatial domain features of the face image to be tested, and obtain multiple different levels of features to be distilled from the face image; a spatial domain encoder module, which is a convolutional neural network with a deep separable convolutional network as its backbone, used to process the first original spatial domain features of the face image to be tested, and obtain multiple different levels of intermediate features and encoded high-dimensional spatial domain features from the face image; and a spatial domain decoder module, used to process the fused features output by the spatial-frequency domain feature fusion module to obtain the reconstructed spatial domain image. The frequency domain reconstruction module includes: a fine-grained space-frequency conversion module, used to convert the second original spatial domain features from RGB space to YCbCr space, and extract the luminance channel to obtain luminance features. The luminance features are processed using a sliding window method to obtain the frequency domain features without channel rearrangement. After channel rearrangement, the original frequency domain features are obtained; a frequency domain encoder module, used to process the original frequency domain features to obtain the high-dimensional frequency domain features encoded from the image under test; and a frequency domain decoder module, used to process the fused features output by the space-frequency domain feature fusion module to obtain the reconstructed frequency domain features.

2. A method for detecting deepfake faces based on image reconstruction, characterized in that, The face deepfake detection system based on image reconstruction as described in claim 1 includes the following steps, wherein steps S1 to S11 are the training phase, step S12 is the testing phase, and step S13 is the usage phase: S1: Construct a dataset of real face images as the training set, and construct a dataset including real face images and fake face images as the test set; both the training set and the test set have image-level annotations. S2: Scale the real face images in the training set obtained in step S1 to obtain the first original spatial features. The image is input into the teacher network module for processing to obtain the real face image. Distillation characteristics at different levels Among them, the first k The size of the distillation feature at each level is ; S3: Scale the real face image obtained in step S1 to obtain the first original spatial features. The image is input into the spatial encoder module for processing to obtain the real face image. Intermediate features at different levels and the encoded high-dimensional spatial features Among them, the first k The size of the intermediate features at each level is ; S4: Calculate and obtain the distillation characteristics obtained in step S2. and the intermediate features obtained in step S3 Distillation loss between ; S5: Scale the real face image obtained in step S1 to obtain the second original spatial domain features, and input them into the fine-grained space-frequency conversion module for processing to obtain the original frequency domain features. ; S6: Obtain the original frequency domain features obtained in step S5 The input is processed by the frequency domain encoder module to obtain the encoded high-dimensional frequency domain features. ; S7: Obtain the high-dimensional spatial features obtained in step S3 and the high-dimensional frequency domain features obtained in step S6 The input is fed into the spatial-frequency domain feature fusion module for processing, and the fused features are obtained through a self-attention mechanism. ; S8: Obtain the fusion features obtained in step S7 The image is input into the spatial domain decoder module for processing to obtain the reconstructed spatial domain image. ; S9: Obtain the fused features obtained in step S7, input them into the frequency domain decoder module for processing, and obtain the reconstructed frequency domain features. ; S10: The first original spatial features obtained based on step S2 The reconstructed spatial features obtained in step S8 The original frequency domain features obtained in step S5 The reconstructed frequency domain features obtained in step S9 and the distillation loss obtained in step S4 The spatial reconstruction loss was calculated. and frequency domain reconstruction loss The final total loss was obtained. This serves as the basis for determining the authenticity of an image; S11: Repeat steps S2 to S8 until the total loss is reached. Convergence, complete training and save the trained spatial encoder module, frequency encoder module, spatial-frequency feature fusion module, spatial decoder module and frequency decoder module; S12: In the testing phase, replace the images in the training set with the images in the test set, and execute steps S2 to S10 to complete the verification and testing of the image reconstruction-based deep face forgery detection method for forged face images. S13: In the usage phase, the image of the face to be tested is replaced with the image in the test set, and steps S2 to S10 are executed to complete the deep face forgery detection based on image reconstruction for the face image to be tested.

3. The face depth forgery detection method based on image reconstruction according to claim 2, characterized in that, The real face image obtained in step S1 is scaled to obtain the second original spatial domain feature, which is then input into the fine-grained space-frequency conversion module for processing to obtain the original frequency domain feature. ,include: S5-1: Scale the real face image obtained in step S1 to obtain the second original spatial domain features; S5-2: Transform the second original spatial feature from RGB space to YCbCr space, and extract the pixel value of the luminance channel, i.e., the luminance feature. ; S5-3: Obtain the brightness features obtained in step S5-2 A sliding window method is used to analyze the brightness features. Processing is performed; the window size is... Then, the size of the window after processing is... Perform DCT transformation on each image patch with a sliding step size of . This yields the frequency domain characteristics of the un-channel rearranged channel. S5-4: Obtain the un-channel rearranged frequency domain features obtained in step S5-3, and rearrange them according to the frequency band size so that features in the same frequency band are in the same channel after rearrangement, thus obtaining the original frequency domain features. .

4. The face deepfake detection method based on image reconstruction according to claim 3, characterized in that, distillation loss The calculation is performed as follows: ； The spatial domain reconstruction loss, frequency domain reconstruction loss, and total loss are calculated as follows: ； ； ； in, Represents the original spatial features at coordinate positions The value, Indicates the reconstructed spatial features at coordinate positions The value, Represents the original frequency domain features at coordinate positions The value, Indicates the reconstructed frequency domain features at coordinate positions The value; This is the loss balancing factor, used to balance the weights among various losses.

Images