A method and apparatus for detecting a fake image based on polynomial evaluation

Abstract

The application discloses a kind of based on polynomial evaluation's counterfeit image detection method and device, including selecting respectively with image domain feature clue, frequency domain feature clue, time sequence feature clue, local feature clue and biological feature clue as the basis for determination of multi-class counterfeit image detection model, and respectively training and testing to determine the equal error rate and true and false classification accuracy of each model, and equal error rate is used as threshold value;With each kind of counterfeit image detection model, the counterfeit image to be detected is detected to obtain each kind of detection probability;For each kind of detection probability, according to the corresponding threshold value, after the detection probability is calibrated, the true and false classification accuracy is used as weight, the calibrated detection probability of all models is polynomial fusion, and the fusion result is used as the final counterfeit image detection result.The method and device detect counterfeit image from multiple angles and calibrate to improve the accuracy of counterfeit image detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of deep fake detection, and particularly relates to a fake image detection method and device based on polynomial evaluation. BACKGROUND

[0002] In recent years, with the promotion of deep learning technology, face swapping fake technology has developed rapidly and achieved a face swapping effect that is difficult to distinguish from reality. Although face swapping fake has a positive effect in the field of film and television entertainment, it has brought serious security risks in public opinion safety. Malicious attackers can use face swapping technology to create and spread false videos, damage personal reputation, and destroy news authenticity. Face swapping technology can also attack existing face authentication systems and cause harm to personal information and property security.

[0003] To address the security risks caused by the misuse of face swapping technology and other technologies, research on deep fake detection technology has received continuous attention in recent years. Existing technologies have proposed various fake face detection algorithms and achieved high true and false classification accuracy on related data sets. However, existing deep fake detection algorithms are based on a single feature clue to determine the authenticity of deep fake images. When faced with DeepFake fake images and videos circulating on the network, deep fake detection algorithms using a single feature clue are prone to misjudgment and have poor generalization. Therefore, there is a lack of models in existing deep fake detection algorithms that can determine the authenticity of images from multiple angles and multiple feature clues.

[0004] Existing DeepFake detection technologies mostly extract feature clues from images and videos, train binary classifiers, and identify DeepFake fake videos. Common DeepFake detection methods, such as WM (Wining Models), F3Net, LocalRelation, and Xception, use a single feature clue to detect fake videos. Although the WM model combines three network models to comprehensively determine the credibility of the image to be detected, neither EfficientNet nor Xception considers other detection features when extracting fake features from the image domain. F3Net only uses frequency domain features, LocalRelation only uses local features, and Xception only uses image domain features. However, relying on a single feature clue to measure the credibility of a video can lead to one-sidedness and poor generalization.

[0005] A few existing detection methods combine different feature clues together, but the proportion of the results obtained by each method in the final comprehensive reliability is determined by training in a certain dataset, and the weight is determined by experiment without theoretical support. Moreover, the weight value obtained in this way is not ideal in cross-dataset. Moreover, the threshold for image authenticity judgment by different feature clues is not the same, and the authenticity probability directly obtained by the model needs to be calibrated. SUMMARY

[0006] In view of the above, the purpose of the present application is to provide a method and device for detecting fake images based on polynomial evaluation, which can detect fake images from multiple angles and calibrate them to improve the accuracy of fake image detection.

[0007] To achieve the above-mentioned purpose of the application, the method for detecting fake images based on polynomial evaluation provided by the embodiment comprises the following steps:

[0008] Selecting multiple fake image detection models based on image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues respectively, and training and testing each model to determine the equal error rate and authenticity classification accuracy of each model, and taking the equal error rate as the threshold;

[0009] Using each type of fake image detection model to detect the fake image to be detected to obtain each type of detection probability;

[0010] For each type of detection probability, the detection probability is calibrated according to the corresponding threshold, and the authenticity classification accuracy is taken as the weight, and the calibrated detection probabilities of all models are polynomially fused, and the fusion result is taken as the final fake image detection result.

[0011] Preferably, the first type of fake image detection model based on image domain feature clues comprises a WM model and an Xception model.

[0012] The second type of fake image detection model based on frequency domain feature clues comprises a F3Net model.

[0013] The third type of fake image detection model based on local feature clues comprises a Local Relation model.

[0014] Preferably, the equal error rate of each model is the point at which the rejection error rate and the error acceptance rate are equal in the model training process.

[0015] Preferably, the detection probability is calibrated according to the corresponding threshold by using the following formula:

[0016]

[0017] wherein p i denotes the detection probability output by the i-th fake image detection model, and i denotes the equal error rate corresponding to the i-th fake image detection model, and p i denotes the calibrated detection probability.

[0018] Preferably, the true-false classification accuracy of each model is equal to the ACC of the model on the test set.

[0019] Preferably, the calibrated detection probabilities of all models are polynomially fused to obtain the final fake image detection result by using the following formula:

[0020]

[0021] wherein p denotes the final fake image detection result, and w i denotes the true-false classification accuracy of the i-th fake image detection model, and p i denotes the calibrated detection probability of the i-th fake image detection model.

[0022] To achieve the above-mentioned purposes, the embodiments further provide a fake image detection device based on polynomial evaluation, comprising a model construction module, a detection module, a calibration evaluation module,

[0023] The model construction module is configured to select multi-class fake image detection models respectively based on image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues, and to train and test each model to determine the equal error rate and the true-false classification accuracy of each model, and to take the equal error rate as a threshold.

[0024] The detection module is configured to detect the to-be-detected fake image by using each type of fake image detection model to obtain each type of detection probability.

[0025] The calibration evaluation module is configured to, for each type of detection probability, calibrate the detection probability according to the corresponding threshold, take the true-false classification accuracy as a weight, polynomially fuse the calibrated detection probabilities of all models, and take the fusion result as the final fake image detection result.

[0026] To achieve the above-mentioned purposes, the embodiments further provide a fake image detection device based on polynomial evaluation, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned fake image detection method based on polynomial evaluation when executing the computer program.

[0027] To achieve the above-mentioned object of the application, the embodiment further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above-mentioned method for detecting a fake image based on polynomial evaluation.

[0028] Compared with the prior art, the application has at least the following beneficial effects:

[0029] The detection results of the multi-class fake image detection model based on the image domain feature clues, the frequency domain feature clues, the time sequence feature clues, the local feature clues and the biological feature clues are combined to judge the authenticity of the to-be-detected fake image from multiple angles and multiple features, the judgment basis is more comprehensive, the equal error rate is introduced as a threshold to calibrate the detection probability obtained by the model, and the error caused by different thresholds of different models is reduced, and the fake image detection accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0031] Figure 1 is a flowchart of the method for detecting a fake image based on polynomial evaluation provided by the embodiment;

[0032] Figure 2 is a flowchart of the method for detecting a fake image based on polynomial evaluation provided by the embodiment;

[0033] Figure 3 is a structural schematic diagram of the device for detecting a fake image based on polynomial evaluation provided by the embodiment. DETAILED DESCRIPTION

[0034] In order to make the object, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and do not limit the protection scope of the present application.

[0035] The inventive concept of the present application is that, in view of the problem of one-sidedness and insufficient generalization of existing fake image detection models in image credibility measurement, the present application provides a fake image detection method and device based on polynomial evaluation, which takes the equal error rate of the model as a threshold, calibrates the direct output of the model, is similar to the normalization of experimental data, prevents the detection method of a single feature clue from having a higher weight in the final credibility measurement, and at the same time, the performance in the training data set is used to determine the weight of the model, the better the performance, the higher the weight, and the poorer the performance, the less the weight, so as to solve the shortcoming that the best weight distribution is obtained in a single data set but the cross-data set effect is not ideal.

[0036] As shown in Figure 1 The fake image detection method based on polynomial evaluation provided by the embodiment comprises the following steps:

[0037] Step 1, selecting multi-class fake image detection models taking image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues as the basis for judgment respectively, and training each model to determine the equal error rate and the true and false classification accuracy of each model, and taking the equal error rate as a threshold.

[0038] Many DeepFake fake methods will leave fake features in the image domain, frequency domain, etc. The DeepFake detection method also starts from these features, extracts image features through a feature extractor. The extracted features are used to train a classifier to obtain a DeepFake detection model. The features commonly used for DeepFake detection include image domain features, frequency domain features, local features, time sequence features and biological features, etc.

[0039] In the embodiment, the WM model and the Xception model taking the image domain feature clues as the basis for judgment, the F3Net model taking the frequency domain feature clues as the basis for judgment, and the Local Relation model taking the local feature clues as the basis for judgment are selected as the multi-class fake image detection models, and the FF++ data set is used to train each multi-class fake image detection model respectively, and the equal error rate of each model is obtained. The equal error rate is the point at which the rejection error rate and the error acceptance rate are equal in the model training process. In the training process, the small batch self-distillation method is also used to improve the robustness of the model.

[0040] In the embodiment, the performance of the model in the test set is also tested, and the ACC of each fake image detection model in the test set is taken as the true and false classification accuracy.

[0041] Step 2, using each type of fake image detection model to detect the fake image to be detected to obtain each type of detection probability.

[0042] During testing, the forged image to be detected is input into each type of forged image detection model, and the detection probability of determining whether the input image is a real image or a forged image is obtained through forward reasoning.

[0043] Step 3: For each type of detection probability, after calibrating the detection probability according to the corresponding threshold, use the accuracy of true and false classification as the weight to perform a weighted summation of the calibrated detection probabilities of all models, and use the summation result as the final fake image detection result.

[0044] In the embodiments, such as Figure 2 As shown, based on the equal error rate as the threshold θ, the detection probabilities obtained from the four forgery detection models are calibrated using threshold θ as a benchmark. This is because the thresholds for judging real and forged images differ in different forgery detection models. For example, in the Local Relation model, the output value for real images is around 0.7, while the output value for forged images is above 0.85. If all four models use 0.5 as the threshold to determine the authenticity of images, then the Local Relation model will contribute nothing to the multinomial fusion. Therefore, it is necessary to calibrate the output detection probabilities of each forgery detection model first.

[0045] In forged image detection models, the decision threshold can be adjusted to balance high security and ease of use. When the false rejection rate (FRR) is greater than the false acceptance rate (FCR), the possibility of false acceptance is reduced, improving system security. Conversely, when the FCR is greater than the FRR, the user experience is appropriately improved, making the system easier for users to use. The equal error rate (EER) is the point where the FRR equals the FRR, balancing system security and ease of use. Therefore, this invention uses the equal error rate as the threshold θ for calibrating the output of each model, with the specific calibration formula as follows:

[0046]

[0047] Where, p i Let θ represent the detection probability output by the i-th forged image detection model. i p represents the equal error rate corresponding to the i-th fake image detection model. i 'Indicates the detection probability after calibration.

[0048] When θ i When p is 0.5, i The value is the same as the original value. When θ i When p > 0.5, after recalibrating the formula, the originally large p will be reduced. i Appropriately reduce, similarly when θ iWhen <0.5, the detection probability output by each model is appropriately increased after re-calibration of the formula, so that the size of the detection probability output by each model is close to each other, similar to the normalization of data preprocessing, and the influence of the output of different models on the calculation of the comprehensive confidence is reduced.

[0049] After the output detection probability of each model is calibrated, the final comprehensive detection probability is calculated by polynomial fusion, which is used as the basis for judging the final fake image detection result. Specifically, the embodiment can use the true-false classification accuracy as the weight to perform weighted summation fusion on the calibrated detection probability of all models, and the specific formula is:

[0050]

[0051] where p represents the final fake image detection result, w i represents the true-false classification accuracy of the i-th fake image detection model, p i ’ represents the calibrated detection probability of the i-th fake image detection model.

[0052] The four fake image detection models based on different feature clues are combined, the performance ACC of the model is used to determine the weight of the detection probability output by each fake image detection model in the final result, so that the comprehensive confidence obtained by combining multiple models can more accurately detect the authenticity of the fake image.

[0053] The above embodiment provides a fake image detection method based on polynomial evaluation, which combines WM, F3Net, Local Relation and Xception four models from multiple angles and multiple feature clues, judges the authenticity of the to-be-detected image from the image domain feature clue, the frequency domain feature clue and the local feature clue, solves the problem of one-sidedness of the existing method; using multiple angles and multiple feature clues, different fake trace features are extracted through multiple models to improve the generalization of the model; using the equal error rate of the model to calibrate the result directly obtained by the model, similar to normalization, reducing the proportion of the result obtained by different feature clues in the final comprehensive confidence because of the different determination thresholds; the weight of each model is determined by ACC in the test set. The weight of the model with better performance is higher, and the weight of the model with poor performance is smaller.

[0054] Based on the same inventive concept, the embodiment also provides a fake image detection device based on polynomial evaluation, as shown in Figure 3 which includes a model construction module, a detection module, a calibration and evaluation module,

[0055] The model construction module is configured to select multiple types of fake image detection models respectively taking image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues as the basis for judgment, and to train and test each model to determine the equal error rate and the true-false classification accuracy of each model, and to use the equal error rate as the threshold value.

[0056] It should be noted that the above embodiment provides a fake image detection device based on polynomial evaluation. When performing fake image detection based on polynomial evaluation, the above-mentioned functional modules are divided for example. The above-mentioned functions can be completed by different functional modules according to the needs, i.e. the internal structure of the terminal or server is divided into different functional modules to complete all or part of the above-described functions. In addition, the fake image detection device based on polynomial evaluation provided by the above embodiment and the fake image detection method based on polynomial evaluation belong to the same concept, and the specific implementation process is described in detail in the fake image detection method based on polynomial evaluation. Here, it is not repeated.

[0057] The embodiment further provides a fake image detection device based on polynomial evaluation, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned fake image detection method based on polynomial evaluation is realized, which comprises the following steps:

[0058] Step 1, selecting multiple types of fake image detection models respectively taking image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues as the basis for judgment, and training and testing each model to determine the equal error rate and the true-false classification accuracy of each model, and using the equal error rate as the threshold value;

[0059] Step 2, using each type of fake image detection model to detect the fake image to be detected to obtain the detection probability of each type;

[0060] Step 3, for each detection probability, calibrating the detection probability according to the corresponding threshold value, using the true-false classification accuracy as the weight, and performing polynomial fusion on the calibrated detection probability of all models, and using the fusion result as the final fake image detection result.

[0061] In practical applications, the computer memory can be a proximal volatile memory such as RAM, a non-volatile memory such as ROM, FLASH, a floppy disk, a mechanical hard disk, etc., and a remote storage cloud. The computer processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), i.e., the steps of the method for detecting a fake image based on polynomial evaluation can be implemented by these processors.

[0062] The embodiments also provide a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method for detecting a fake image based on polynomial evaluation, including the following steps:

[0063] Step 1, selecting a multi-class fake image detection model based on image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues, and biological feature clues respectively, and training and testing each model to determine the equal error rate and the true-false classification accuracy of each model, and taking the equal error rate as the threshold;

[0064] Step 2, detecting the to-be-detected fake image by using each type of fake image detection model to obtain each type of detection probability;

[0065] Step 3, for each type of detection probability, calibrating the detection probability according to the corresponding threshold, and taking the true-false classification accuracy as the weight, and performing polynomial fusion on the calibrated detection probabilities of all models, and taking the fusion result as the final fake image detection result.

[0066] The computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0067] The above detailed description of the specific embodiments of the present application has explained the technical solutions and beneficial effects of the present application. It should be understood that the above description is only the most preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, supplements, and equivalent replacements made within the principle range of the present application should be included in the protection scope of the present application.

Claims

1. A method for detecting a fake image based on polynomial evaluation, characterized by, The method comprises the following steps: selecting multiple types of fake image detection models taking image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues as judgment basis respectively, wherein the first type of fake image detection model taking image domain feature clues as judgment basis comprises a WM model and an Xception model; the second type of fake image detection model taking frequency domain feature clues as judgment basis comprises an F3Net model; the third type of fake image detection model taking local feature clues as judgment basis comprises a Local Relation model; determining the equal error rate and the true-false classification accuracy of each model by training and testing respectively, and taking the equal error rate as the threshold, wherein the equal error rate of each model is the point at which the rejection error rate and the false acceptance rate are equal in the model training process; the true-false classification accuracy of each model is equal to the ACC of the model in the test set; detecting the to-be-detected fake image by using each type of fake image detection model to obtain each type of detection probability; for each type of detection probability, the detection probability is calibrated according to the corresponding threshold by using the following formula: wherein p i represents the detection probability output by the i-th fake image detection model, θ i represents the equal error rate corresponding to the i-th fake image detection model, p i ’ represents the detection probability after calibration; then, the calibrated detection probabilities of all models are polynomially fused by taking the true-false classification accuracy as the weight, and the fusion result is taken as the final fake image detection result. 2.The method of claim 1, wherein, the calibrated detection probabilities of all models are polynomially fused by using the following formula to obtain the final fake image detection result: where p represents the final fake image detection result, w i represents the true-false classification accuracy of the i-th fake image detection model, p i ’ represents the calibrated detection probability of the i-th fake image detection model.

3. A forged image detection device based on polynomial evaluation, characterized by, The method comprises a model construction module, a detection module, a calibration and evaluation module, the model construction module is used to select multiple types of fake image detection models taking image domain feature clues, frequency domain feature clues, time sequence feature clues, local feature clues and biological feature clues as judgment basis respectively, wherein the first type of fake image detection model taking image domain feature clues as judgment basis comprises a WM model and an Xception model; the second type of fake image detection model taking frequency domain feature clues as judgment basis comprises an F3Net model; the third type of fake image detection model taking local feature clues as judgment basis comprises a Local Relation model; the equal error rate and the true-false classification accuracy of each model are determined by training and testing respectively, and the equal error rate is taken as the threshold, wherein the equal error rate of each model is the point at which the rejection error rate and the false acceptance rate are equal in the model training process; the true-false classification accuracy of each model is equal to the ACC of the model in the test set; the detection module is used to detect the to-be-detected fake image by using each type of fake image detection model to obtain each type of detection probability; the calibration and evaluation module is used to calibrate the detection probability according to the corresponding threshold by using the following formula for each type of detection probability: wherein p i represents the detection probability output by the i-th fake image detection model, θ i represents the equal error rate corresponding to the i-th fake image detection model, p i ’ represents the detection probability after calibration; then, the calibrated detection probabilities of all models are polynomially fused by taking the true-false classification accuracy as the weight, and the fusion result is taken as the final fake image detection result.

4. A forged image detection apparatus based on polynomial evaluation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the fake image detection method based on polynomial evaluation of claim 1 or 2.

5. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the steps of the fake image detection method based on polynomial evaluation of claim 1 or 2.

Images