Face forgery detection method, system, device and medium based on frequency domain multi-scale

Abstract

The application discloses a face forgery detection method and system based on frequency domain multi-scale, equipment and medium, which can obtain images of different frequency bands through convolution, downsampling, upsampling through bilinear interpolation and Gaussian kernel convolution operation, so that the face forgery detection still has good generalization performance when facing different scenes; through extracting the features of images of different frequency bands, the obtained convolution features include different frequency band information; through pooling the convolution features in different scales, multi-scale features are obtained, which helps to capture more discriminative features and retain more rich texture information, solves the problem that the scales of artifacts generated by the forgery method are different and single-scale features may not capture all the required spatial information; finally, through training, a face forgery detection network is obtained, which combines the advantages of images of different frequency bands and multi-scale features, greatly improving the generalization ability, accuracy and performance of the face forgery detection method.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of face anti-counterfeiting technology, and in particular to a face forgery detection method and system based on frequency domain multi-scale, a device and a medium. BACKGROUND

[0002] With the development of artificial intelligence, face forgery technology can create very realistic fake face images and videos and is of various types. When these technologies are used for malicious purposes, they will cause many social hazards. In order to deal with these threats, more and more research has been put into the field of face forgery detection. Nowadays, most existing face forgery detection methods focus on intra-database detection, that is, the forgery method of training and testing samples is the same. Even if some methods use information from other domains and achieve good robustness, these methods are often affected by overfitting, resulting in that the detection method has good effect only when it detects forgery images related to training samples. In real life, the detector often faces samples whose forgery method is unknown, that is, the cross-database scene. The lack of generalization ability of face forgery detection technology leads to the inability to meet the detection of increasingly updated and unknown forgery methods. In the cross-database scene of training and testing samples synthesized by different algorithms, most face forgery detection methods are difficult to achieve satisfactory performance. SUMMARY

[0003] The present application aims to at least solve one of the problems in the prior art. To this end, the present application proposes a face forgery detection method and system based on frequency domain multi-scale, a device and a medium, which uses frequency domain information as a clue and combines multi-scale angles to capture artifacts, greatly improves the generalization ability of the face forgery detection method, and has high accuracy and performance in the cross-database scene of face forgery detection.

[0004] In a first aspect, an embodiment of the present application provides a face forgery detection method based on frequency domain multi-scale, which comprises:

[0005] obtaining a face image;

[0006] inputting the face image into a preset face forgery detection network to obtain the authenticity of the face image output by the face forgery detection network; the face forgery detection network is trained by the following method:

[0007] convolve the face image by a Gaussian kernel to obtain a convolution result of the face image; down-sample the convolution result of the face image to obtain a first image with a size of one quarter of the face image; up-sample the first image to a second image with the size of the face image by bilinear interpolation; convolve the second image by the Gaussian kernel to obtain a third image; obtain a fourth image by taking a residual between the third image and the face image; convolve the third image by the Gaussian kernel to obtain a convolution result of the third image; down-sample the convolution result of the third image to obtain a fifth image with a size of one quarter of the third image; up-sample the fifth image to a sixth image with the size of the third image by bilinear interpolation; convolve the sixth image by the Gaussian kernel to obtain a seventh image; obtain an eighth image by taking a residual between the seventh image and the fourth image;

[0008] extract features of the face image, the fourth image, the seventh image and the eighth image by a convolutional neural network to obtain convolutional features;

[0009] pool the convolutional features in different scales to obtain multi-scale features;

[0010] input the multi-scale features after adaptive average pooling to a preset classifier to obtain an output result of the classifier.

[0011] The method according to the embodiment of the application has at least the following beneficial effects:

[0012] The fourth image, the seventh image and the eighth image are obtained by convolution, down-sampling, up-sampling by bilinear interpolation and Gaussian kernel convolution, the face image is decomposed into images of different frequency bands, the information represented by different frequency bands is different, the low frequency corresponds to semantic information, the high frequency corresponds to texture information and details, and the different frequency band information enables the face forgery detection to still have good generalization performance when facing different scenes; the features of the face image, the fourth image, the seventh image and the eighth image are extracted, so that the obtained convolutional features include different frequency band information; the multi-scale features are obtained by pooling the convolutional features in different scales, which helps to capture more discriminative features and retain more rich texture information, solves the problem that the scale of artifacts generated by the forgery method is not the same and single-scale features may not capture all the required spatial information; finally, the face forgery detection network is obtained by training, the advantages of images of different frequency bands and multi-scale features are combined, the generalization ability, accuracy and performance of the face forgery detection method are greatly improved, and the trained face forgery detection network has high robustness.

[0013] According to some embodiments of the present application, the feature extraction of the face image, the fourth image, the seventh image and the eighth image by the convolutional neural network comprises:

[0014] The fourth image, the seventh image and the eighth image are merged to obtain a first merged image;

[0015] The first merged image is convolved and merged with the face image to obtain a second merged image;

[0016] The second merged image is extracted by the convolutional neural network.

[0017] According to some embodiments of the present application, the pooling of the convolutional features at different scales to obtain multi-scale features comprises:

[0018] The convolutional features are convolved and reduced in dimension by multiple dilated convolutions with different dilation rates to obtain multiple dilated features;

[0019] Each dilated feature is adaptively averaged pooled at different scales to obtain a pooled feature with different scales corresponding to each dilated feature;

[0020] Each pooled feature with different scales corresponding to each dilated feature is bilinearly interpolated upsampled and merged to obtain a merged feature corresponding to each dilated feature; the resolution of the merged feature is the same as that of the convolutional feature;

[0021] The merged features of all dilated features are merged and convolved to obtain the multi-scale features.

[0022] According to some embodiments of the present application, after the multi-scale features are input into a preset classifier to obtain an output result of the classifier, the training method of the face forgery detection network further comprises:

[0023] The loss of the output result is calculated by a cross-entropy loss function;

[0024] The loss of the output result is backpropagated to update the weight of the face forgery detection network; the training of the face forgery detection network is completed until the loss of the output result reaches a loss threshold or the number of iterations reaches a cycle threshold.

[0025] According to some embodiments of the present application, the calculation formula of the loss of the output result comprises:

[0026]

[0027] wherein, represents the result predicted by the model, loss representing an output result, y is set to 1 if a sample category is false, and is set to 0 otherwise.

[0028] According to some embodiments of the present application, the convolutional neural network uses an Xception network.

[0029] According to some embodiments of the present application, the Gaussian kernel is:

[0030]

[0031] In a second aspect, embodiments of the present application provide a face forgery detection system based on frequency domain multi-scale, which comprises:

[0032] An image acquisition module is configured to acquire a face image.

[0033] A face forgery detection network training module is configured to input the face image into a preset face forgery detection network to obtain the authenticity of the face image output by the face forgery detection network; the face forgery detection network training module comprises:

[0034] A frequency domain decomposition submodule is configured to convolve the face image by a Gaussian kernel to obtain a convolution result of the face image; downsample the convolution result of the face image to obtain a first image with a size of one quarter of the face image; upsample the first image by bilinear interpolation to restore it to a second image with the size of the face image; convolve the second image by the Gaussian kernel to obtain a third image; obtain a fourth image by taking a residual between the third image and the face image; convolve the third image by the Gaussian kernel to obtain a convolution result of the third image; downsample the convolution result of the third image to obtain a fifth image with a size of one quarter of the third image; upsample the fifth image by bilinear interpolation to restore it to a sixth image with the size of the third image; convolve the sixth image by the Gaussian kernel to obtain a seventh image; obtain an eighth image by taking a residual between the seventh image and the fourth image.

[0035] A feature extraction submodule is configured to extract features from the face image, the fourth image, the seventh image, the eighth image, and a convolutional neural network to obtain convolutional features.

[0036] A multi-scale feature submodule is configured to pool the convolutional features at different scales to obtain multi-scale features.

[0037] A classifier submodule is configured to input the multi-scale features after adaptive average pooling into a preset classifier to obtain an output result of the classifier.

[0038] In a third aspect, embodiments of the present application provide an electronic device, comprising at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the face forgery detection method based on frequency domain multi-scale as described in the first aspect.

[0039] In a fourth aspect, embodiments of the present application provide a computer storage medium, which stores computer executable instructions for causing a computer to perform the face forgery detection method based on frequency domain multi-scale as described in the first aspect.

[0040] It should be noted that the beneficial effects between the second to fourth aspects of the present application and the prior art are the same as those of the face forgery detection method based on frequency domain multi-scale of the first aspect, which will not be described here.

[0041] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0043] Figure 1 is a flowchart of a face forgery detection method based on frequency domain multi-scale provided by an embodiment of the present application;

[0044] Figure 2 is a flowchart of training of a face forgery detection network provided by an embodiment of the present application;

[0045] Figure 3 is a flowchart of feature extraction of a face image, a fourth image, a seventh image and an eighth image by a convolutional neural network provided by an embodiment of the present application;

[0046] Figure 4 is a flowchart of pooling of convolutional features to obtain multi-scale features at different scales provided by an embodiment of the present application;

[0047] Figure 5 is a flowchart of a training step included in the training method of the face forgery detection network provided by an embodiment of the present application;

[0048] Figure 6 is a schematic diagram of a face forgery detection method based on frequency domain multi-scale provided by an embodiment of the present application;

[0049] Figure 7 is a schematic diagram of obtaining the fourth image, the seventh image and the eighth image provided by an embodiment of the present application;

[0050] Figure 8 is a schematic diagram of visualizing the fourth image, the seventh image and the eighth image provided by an embodiment of the present application;

[0051] Figure 9 is a schematic diagram of a specific implementation of a face forgery detection method based on frequency domain multi-scale provided by an embodiment of the present application;

[0052] Figure 10 is a structural diagram of a face forgery detection system based on frequency domain multi-scale provided by an embodiment of the present application;

[0053] Figure 11 is a structural diagram of a sub-module under a face forgery detection network module provided by an embodiment of the present application;

[0054] Figure 12 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0056] In the description of the present application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features or the sequence of indicated technical features.

[0057] In the description of the present application, it should be understood that the orientation description, such as up, down, etc., is based on the orientation or position relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element indicated must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.

[0058] In the description of the present application, it should be noted that, unless otherwise explicitly limited, the words such as setting, installing, connecting, etc. should be broadly understood, and those skilled in the art can reasonably determine the specific meaning of the above words in the present application in combination with the specific content of the technical solution.

[0059] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the following described embodiments are part of the embodiments of the present application, not all embodiments.

[0060] Referring to Figure 1 and Figure 2 In some embodiments of the present application, a face forgery detection method based on frequency domain multi-scale is provided, comprising:

[0061] Step S100, acquiring a face image.

[0062] Step S200, inputting the face image into a preset face forgery detection network to obtain the authenticity of the face image output by the face forgery detection network; the face forgery detection network is trained in the following way:

[0063] Step S210, convolving the face image through a Gaussian kernel to obtain a convolution result of the face image; down-sampling the convolution result of the face image to obtain a first image with a size of one-fourth of the face image; up-sampling the first image through bilinear interpolation to restore it to a second image with the size of the face image; convolving the second image through a Gaussian kernel to obtain a third image; obtaining a fourth image through a residual between the third image and the face image; convolving the third image through a Gaussian kernel to obtain a convolution result of the third image; down-sampling the convolution result of the third image to obtain a fifth image with a size of one-fourth of the third image; up-sampling the fifth image through bilinear interpolation to restore it to a sixth image with the size of the third image; convolving the sixth image through a Gaussian kernel to obtain a seventh image; obtaining an eighth image through a residual between the seventh image and the fourth image.

[0064] Step S220, extracting features of the face image, the fourth image, the seventh image and the eighth image through a convolutional neural network to obtain convolutional features.

[0065] Step S230, pooling the convolutional features in different scales to obtain multi-scale features.

[0066] Step S240, inputting the multi-scale features into a preset classifier after adaptive average pooling to obtain an output result of the classifier.

[0067] The fourth image, the seventh image and the eighth image are obtained through the convolution, downsampling, upsampling through bilinear interpolation and Gaussian kernel convolution of step S210, the face image is decomposed into images of different frequency bands, the information represented by different frequency bands is different, the low frequency corresponds to semantic information, the high frequency corresponds to texture information and details, and different frequency band information enables the face forgery detection to still have good generalization performance when facing different scenes; the features of the face image, the fourth image, the seventh image and the eighth image are extracted through step S220, so that the obtained convolution features include different frequency band information; the convolution features are pooled in different scales through step S230 to obtain multi-scale features, which is helpful to capture more discriminative features and retain more rich texture information, and solves the problems that the scales of artifacts generated by the forgery method are different and single-scale features may not capture all the required spatial information; finally, the face forgery detection network is trained through step S240, the advantages of images of different frequency bands and multi-scale features are combined, the generalization ability, accuracy and performance of the face forgery detection method are greatly improved, and the trained face forgery detection network has high robustness.

[0068] Referring to Figure 3 and referring to Figure 8 In some embodiments of the present application, the face image, the fourth image, the seventh image and the eighth image are subjected to feature extraction by a convolutional neural network, comprising:

[0069] Step S221, merging the fourth image, the seventh image and the eighth image to obtain a first merged image.

[0070] Step S222, merging the first merged image after convolution with the face image to obtain a second merged image.

[0071] Step S223, feature extraction on the second merged image by a convolutional neural network.

[0072] It should be noted that, in order to distinguish the fourth image, the seventh image and the eighth image, as shown in Figure 8 , the present embodiment will provide better observation for the grayscale and visualization of the fourth image, the seventh image and the eighth image, so as to facilitate the distinction.

[0073] The fourth image, the seventh image and the eighth image are merged, so that the images of different frequency bands are fused on the same image, which facilitates the subsequent feature extraction and improves the generalization performance.

[0074] Referring to Figure 4 In some embodiments of the present application, the convolution features are pooled in different scales to obtain multi-scale features, comprising:

[0075] Step S231, performing convolution on the convolution feature through the dilated convolution of multiple different expansion rates and reducing dimension to obtain multiple expansion features.

[0076] Step S232, performing adaptive average pooling of different scales on each expansion feature to obtain the pooling feature with different scales corresponding to each expansion feature.

[0077] Step S233, performing bilinear interpolation upsampling on the pooling feature with different scales corresponding to each expansion feature and then merging to obtain the merged feature corresponding to each expansion feature; the resolution of the merged feature is the same as that of the convolution feature.

[0078] Step S234, merging and convolving the merged features of all expansion features to obtain the multi-scale feature.

[0079] By performing convolution on the convolution feature through the dilated convolution of different expansion rates and reducing dimension, multiple branch expansion features are obtained, adaptive average pooling of different scales is performed on each branch to obtain the pooling feature with different scales, which can improve the robustness of the multi-scale feature, and finally the merged features of all expansion features are merged and convolved to obtain the multi-scale feature, which can help to capture more discriminative features and retain more rich texture information, and solve the problem that the scale of the artifact generated by the forgery method is not the same and the single-scale feature may not capture all the required spatial information.

[0080] Reference Figure 5 In some embodiments of the present application, after the multi-scale feature is input into the preset classifier to obtain the output result of the classifier, the training method of the face forgery detection network further comprises:

[0081] Step S250, calculating the loss of the output result through the cross-entropy loss function.

[0082] Step S260, updating the weight of the face forgery detection network through the loss of the output result; and iteratively until the loss of the output result reaches the loss threshold or the number of iterations reaches the cycle threshold to obtain the trained face forgery detection network.

[0083] The preset face forgery detection network is trained through multi-scale features, the training is stopped after meeting the preset condition, the trained face forgery detection network is obtained, the face forgery detection network can be fully converged, good performance can be provided for face forgery detection, the weight of the face forgery detection network is updated through loss back propagation of the output result, so that the output of the network is as close to the true value as possible, the back propagation algorithm is to derive the gradient through the chain rule from the last loss function to each layer of the network in front, and then the gradient descent algorithm is used to update the parameters, so that the output of the loss function of the model is as small as possible, and the loss function is optimized, and the accuracy of face image anti-counterfeiting is ensured.

[0084] In some embodiments of the application, the loss of the output result is calculated by the following formula:

[0085]

[0086] Wherein, The model prediction result is represented by y, The loss of the output result is represented by y, and y is set to 1 if the sample category is false, otherwise it is set to 0.

[0087] The loss of the output result is calculated to ensure that the loss function of the model is optimized, so that the output of the network is as close to the true value as possible.

[0088] In some embodiments of the application, the convolutional neural network uses an Xception network.

[0089] The Xception network simplifies the convolutional neural network, so that the convergence process is faster and the accuracy is higher, and the performance of the convolutional neural network is greatly improved.

[0090] In some embodiments of the application, the Gaussian kernel is:

[0091]

[0092] It should be noted that the Gaussian kernel is center-symmetric in each direction, and no horizontal and vertical trace lines will be generated during filtering, which can better filter.

[0093] The Gaussian kernel suitable for face images is selected for better filtering, so that the results obtained by convolution have good effects and the image error generated during filtering is reduced.

[0094] Referring to Figure 6 , Figure 7 and Figure 9 , in order to facilitate the understanding of those skilled in the art, one specific embodiment of the application provides a face forgery detection method based on frequency domain multi-scale, comprising:

[0095] The first step is to perform frame extraction on the video to be detected, and then extract the face image. The face image is resized to 299x299 input network. The model includes training and inference stages. In the training process, the face image is first input into the frequency domain decomposition module, and the face image is convolved through the Gaussian kernel to obtain the convolution result of the face image; the convolution result of the face image is down-sampled to obtain a first image with a quarter of the size of the face image; the first image is up-sampled to a second image with the size of the face image through bilinear interpolation; the second image is convolved through the Gaussian kernel to obtain a third image; the third image and the face image are combined through the residual to obtain a fourth image; the third image is convolved through the Gaussian kernel to obtain the convolution result of the third image; the convolution result of the third image is down-sampled to obtain a fifth image with a quarter of the size of the third image; the fifth image is up-sampled to a sixth image with the size of the third image through bilinear interpolation; the sixth image is convolved through the Gaussian kernel to obtain a seventh image; the seventh image and the fourth image are combined through the residual to obtain an eighth image; three images with different frequency bands, the fourth image I1, the eighth image I2 and the seventh image I3 are obtained, and the three images with different frequency bands, the fourth image I1, the eighth image I2 and the seventh image I3 are visualized, as shown in Figure 8 The upper images respectively represent the fourth image I1, the eighth image I2 and the seventh image I3, and the lower images respectively represent the visualized fourth image I1, the eighth image I2 and the seventh image I3.

[0096] The first step is to combine the images with different frequency bands after convolution and input them into the convolutional neural network for feature extraction. The convolutional neural network used is the Xception network, and only the input stream and the output stream are retained. Then, the extracted features are input into the multi-scale pooling module, and the obtained results are input into the convolutional neural network for feature extraction after the frequency domain decomposition module. Referring to Figure 9 , the specific process of the multi-scale pooling feature module is as follows: first, an expansion convolution with a set of different expansion rates d∈{1, 2, 3} is used to perform convolution on the input feature F and reduce the dimension to obtain features F1, F2, F3. Then, for F1, F2, F3, regarded as three branches, taking F1 as an example, a plurality of adaptive average pooling with different scales are applied to decompose the feature map into different sub-regions to obtain pooling features with different scales. The pooling features are bilinearly up-sampled to maintain the same resolution size as the input features. Then, the features are combined with F1 to obtain the results of a single branch. The results of the three branches are combined, and finally a 1x1 convolution is performed to obtain the required multi-scale features. The specific structure diagram of the multi-scale pooling module is as follows: Figure 9As shown in the figure, where F represents the input feature, d1, d2, d3 represent different expansion rates, and the three different size cubes after adaptive average pooling represent the pooling features obtained after pooling with different pooling kernels. Cat represents the merging operation in the channel. The omitted parts of the upper and lower branches are consistent with the middle branch.

[0097] Third step, the multi-scale features are adaptively averaged and pooled into a fully connected layer to output the results. The loss is calculated using the cross-entropy loss function, as follows:

[0098]

[0099] If the sample category is false, y is set to 1, otherwise it is set to 0. The model predicts the results. Then, the weights are updated by backpropagation. When the set conditions are met, the training is stopped, and the trained network is obtained. In the inference stage, the image to be detected is input into the trained model to obtain the classification result.

[0100] Reference Figure 10 and Figure 11 One embodiment of the present application also provides a face forgery detection system based on frequency domain multi-scale, comprising an image acquisition module 1100, a face forgery detection network training module 1200, a frequency domain decomposition submodule 1210, a feature extraction submodule 1220, a multi-scale feature submodule 1230 and a classifier submodule 1240, wherein:

[0101] The image acquisition module 1100 is used to acquire a face image.

[0102] The face forgery detection network training module 1200 is used to input the face image into a pre-set face forgery detection network to obtain the authenticity of the face image output by the face forgery detection network; the face forgery detection network training module comprises:

[0103] The frequency domain decomposition submodule 1210 is used to convolve the face image through a Gaussian kernel to obtain a convolution result of the face image; down-samples the convolution result of the face image to obtain a first image with a size of one quarter of the face image; up-samples the first image through bilinear interpolation to restore it to a second image with the size of the face image; convolves the second image through a Gaussian kernel to obtain a third image; obtains a fourth image through a residual between the third image and the face image; convolves the third image through a Gaussian kernel to obtain a convolution result of the third image; down-samples the convolution result of the third image to obtain a fifth image with a size of one quarter of the third image; up-samples the fifth image through bilinear interpolation to restore it to a sixth image with the size of the third image; convolves the sixth image through a Gaussian kernel to obtain a seventh image; obtains an eighth image through a residual between the seventh image and the fourth image.

[0104] The feature extraction submodule 1220 is configured to perform feature extraction on the face image, the fourth image, the seventh image, the eighth image, and the convolutional neural network to obtain convolutional features.

[0105] The multi-scale feature submodule 1230 is configured to perform pooling of different scales on the convolutional features to obtain multi-scale features.

[0106] The classifier submodule 1240 is configured to input the multi-scale features after adaptive average pooling into a preset classifier to obtain an output result of the classifier.

[0107] It should be noted that, since the one kind of face forgery detection system based on frequency domain multi-scale in the embodiment and the one kind of face forgery detection method based on frequency domain multi-scale described above are based on the same inventive concept, the corresponding content in the method embodiment is also applicable to the device embodiment, and will not be described in detail here.

[0108] Reference Figure 12 Another embodiment of the present application also provides an electronic device 6000, which can be any type of smart terminal, such as a mobile phone, a tablet computer, a personal computer, etc.

[0109] Specifically, the electronic device 6000 includes one or more control processors 6001 and a memory 6002, Figure 12 For example, the control processor 6001 and the memory 6002 can be connected through a bus or other means, Figure 12 For example, the connection is through a bus.

[0110] The memory 6002 is a kind of non-transient computer readable storage medium, which can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the electronic device in the embodiment of the present application.

[0111] The control processor 6001 performs various functional applications and data processing of the one kind of face forgery detection method based on frequency domain multi-scale by running the non-transient software programs, instructions and modules stored in the memory 6002, that is, implements the one kind of face forgery detection method based on frequency domain multi-scale in the above-mentioned method embodiment.

[0112] The memory 6002 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required by at least one function, and the like. The data storage area can store data created by the use of the method for face forgery detection based on frequency domain multi-scale, and the like. In addition, the memory 6002 can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 6002 can optionally include a memory that is remotely located with respect to the control processor 6001, and these remote memories can be connected to the electronic device 6000 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0113] When one or more modules stored in the memory 6002 are executed by the one or more control processors 6001, the method for face forgery detection based on frequency domain multi-scale in the above-described method embodiments is executed, such as the method steps of the above-described method for face forgery detection based on frequency domain multi-scale. Figures 1 to 5

[0114] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include a high-speed random access memory, and can further include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory that is remotely located with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0115] It should be noted that, since the electronic device in the present embodiment and the above-described method for face forgery detection based on frequency domain multi-scale are based on the same inventive concept, the corresponding contents in the method embodiments are also applicable to the present device embodiment, which will not be described in detail here.

[0116] An embodiment of the present application also provides a computer-readable storage medium storing computer-executable instructions for performing the method for face forgery detection based on frequency domain multi-scale of the above-described embodiments.

[0117] It should be noted that, since the computer-readable storage medium in the present embodiment and the above-described method for face forgery detection based on frequency domain multi-scale are based on the same inventive concept, the corresponding contents in the method embodiments are also applicable to the present device embodiment, which will not be described in detail here.

[0118] ​As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, etc. in the above-disclosed methods can be embodied in software, firmware, hardware, and any suitable combination thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves or other transport mechanisms, and includes any information delivery media.

[0119] In the description of the specification, reference to terms such as "one embodiment", "some embodiments", "certain embodiments", "example", "specific example" or "some examples" means that a particular feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. Descriptions of the above terms in the specification do not necessarily refer to the same embodiment or example. Moreover, the particular features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0120] Although embodiments of the application have been shown and described, it is to be understood that various modifications, substitutions, replacements and variations can be made to these embodiments without departing from the principles and spirit of the application, the scope of which is defined by the claims and their equivalents.

Claims

1. A face forgery detection method based on frequency domain multi-scale, characterized in that, The face forgery detection method based on frequency domain multi-scale includes: Obtaining a face image; Inputting the face image into a preset face forgery detection network to obtain the authenticity of the face image output by the face forgery detection network; the face forgery detection network is trained by the following method: Convolve the face image by a Gaussian kernel to obtain a convolution result of the face image; down-sample the convolution result of the face image to obtain a first image with a size of one quarter of the face image; up-sample the first image by bilinear interpolation to restore it to a second image with the size of the face image; convolve the second image by the Gaussian kernel to obtain a third image; obtain a fourth image by taking the difference between the third image and the face image; convolve the third image by the Gaussian kernel to obtain a convolution result of the third image; down-sample the convolution result of the third image to obtain a fifth image with a size of one quarter of the third image; up-sample the fifth image by bilinear interpolation to restore it to a sixth image with the size of the third image; convolve the sixth image by the Gaussian kernel to obtain a seventh image; obtain an eighth image by taking the difference between the seventh image and the fourth image; Extract features of the face image, the fourth image, the seventh image and the eighth image by a convolutional neural network to obtain convolutional features; Pool the convolutional features at different scales to obtain multi-scale features; Input the multi-scale features after adaptive average pooling into a preset classifier to obtain the output result of the classifier.

2. The face forgery detection method based on frequency domain multiscale according to claim 1, characterized in that, The feature extraction of the face image, the fourth image, the seventh image and the eighth image by the convolutional neural network includes: Merge the fourth image, the seventh image and the eighth image to obtain a first merged image; Merge the first merged image after convolution with the face image to obtain a second merged image; Extract features of the second merged image by the convolutional neural network.

3. The method of claim 1, wherein the method is based on frequency domain multiscale face forgery detection. The pooling of the convolutional features at different scales to obtain multi-scale features includes: Perform convolution and dimension reduction on the convolutional features by multiple dilated convolutions with different dilation rates to obtain multiple dilated features; Perform adaptive average pooling at different scales on each dilated feature to obtain a pooled feature with different scales corresponding to each dilated feature; Merge the pooled features with different scales corresponding to each dilated feature after bilinear interpolation up-sampling to obtain a merged feature corresponding to each dilated feature; the resolution of the merged feature is the same as that of the convolutional feature; Merge and convolve the merged features of all dilated features to obtain the multi-scale features.

4. The method of claim 1, wherein the method is based on frequency domain multiscale face forgery detection. After the multi-scale features are input into the preset classifier to obtain the output result of the classifier, the training method of the face forgery detection network further includes: Calculate the loss of the output result by a cross-entropy loss function; The loss of the output result is back propagated to update the weight value of the face forgery detection network; and the iteration is repeated until the loss of the output result reaches a loss threshold or the number of iterations reaches a cycle threshold to obtain the trained face forgery detection network.

5. The face forgery detection method based on frequency domain multiscale according to claim 4, characterized in that, The loss of the output result is back propagated to update the weight value of the face forgery detection network; and the iteration is repeated until the loss of the output result reaches a loss threshold or the number of iterations reaches a cycle threshold to obtain the trained face forgery detection network. wherein, represents the result predicted by the model, represents the loss of the output result, y is set to 1 if the sample class is false, otherwise it is set to 0.

6. The face forgery detection method based on frequency domain multiscale according to any one of claims 1 or 2, characterized in that, The convolutional neural network uses an Xception network.

7. The method of claim 1, wherein the method is based on frequency domain multiscale face forgery detection. The Gaussian kernel is:

8. A frequency domain multi-scale based face forgery detection system characterized in that, The face forgery detection system based on the frequency domain multi-scale includes: An image acquisition module is configured to acquire a face image. A face forgery detection network training module is configured to input the face image into a preset face forgery detection network to obtain the authenticity of the face image output by the face forgery detection network; the face forgery detection network training module includes: A frequency domain decomposition submodule is configured to convolve the face image by a Gaussian kernel to obtain a convolution result of the face image; downsample the convolution result of the face image to obtain a first image with a size of one quarter of the face image; upsample the first image by bilinear interpolation to restore it to a second image with the size of the face image; convolve the second image by the Gaussian kernel to obtain a third image; obtain a fourth image by taking the difference between the third image and the face image; convolve the third image by the Gaussian kernel to obtain a convolution result of the third image; downsample the convolution result of the third image to obtain a fifth image with a size of one quarter of the third image; upsample the fifth image by bilinear interpolation to restore it to a sixth image with the size of the third image; convolve the sixth image by the Gaussian kernel to obtain a seventh image; obtain an eighth image by taking the difference between the seventh image and the fourth image; A feature extraction submodule is configured to extract features from the face image, the fourth image, the seventh image, the eighth image, and a convolutional neural network to obtain convolutional features. A multi-scale feature submodule is configured to pool the convolutional features at different scales to obtain multi-scale features. A classifier submodule is configured to input the multi-scale features after adaptive average pooling to a preset classifier to obtain an output result of the classifier.

9. An electronic device, comprising: The memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the face forgery detection method based on the frequency domain multi-scale according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the face forgery detection method based on the frequency domain multi-scale according to any one of claims 1 to 7.

Images