A deep fake image detection method based on two-stage learning

Abstract

The present invention provides a deep fake image detection method based on dual-stage learning, which relates to the field of image processing technology. The method comprises the following steps: S1, obtaining a first global feature of a first true and false image pair, obtaining a first universal fake feature information based on the first global feature, detecting an image based on the first universal fake feature information and updating the parameters of the initial detection model; S2, obtaining a second global feature of a second true and false image pair, obtaining a second universal fake feature information, a specific fake method, gender and race based on the second global feature, calculating feature classification loss, and updating the parameters of the detection model based on the feature classification loss. The deep fake image detection method based on dual-stage learning provided by the present invention can effectively detect deep fake images and improve the detection success rate of unknown deep fake image technology.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a two-stage training deep learning forged image detection method. Background Art

[0002] As a user's unique identity information, face has been widely used in various scenarios. However, with the development of computer technology, some technologies have been maliciously used to forge face images. In particular, with the development of deep learning technology, the method of using deep learning technology to forge faces is called deep forgery. Some forged face images are difficult for the human eye to distinguish. These forged face images are widely used to generate false information, guide public opinion, and lead to adverse consequences.

[0003] Among the various existing deep fake image detection methods, the recognition of known deep fake technologies is relatively mature, but it is difficult to recognize unknown fake technologies. Therefore, in order to solve the above problems, the present invention proposes a deep fake image detection method based on dual-stage learning. Summary of the invention

[0004] The purpose of the present invention is to provide a deep fake image detection method based on two-stage learning, which improves the success rate of deep fake image detection and improves the success rate of identifying deep fake images in the prior art.

[0005] The present invention provides a deep fake image detection method based on two-stage learning, which adopts the following technical solution:

[0006] In the first stage, a first true and false image pair is input into an initial detection model, a first global feature of the first true and false image pair is obtained based on an encoder, first universal forged feature information is obtained based on the first global feature, an image is detected based on the first universal forged feature information, and parameters of the initial detection model are updated to obtain a disentangled forged image detection model;

[0007] In the second stage, a second true and false image pair is input into the disentangled forged image detection model, a second global feature of the second true and false image pair is obtained based on the encoder, second universal forgery feature information, specific forgery method, gender and race are obtained based on the second global feature, feature classification loss of the second universal forgery feature information, specific forgery method, gender and race is calculated, and parameters of the disentangled forged image detection model are updated based on the feature classification loss.

[0008] As described above, the deep fake image detection method based on two-stage learning, wherein obtaining the first global feature of the first true and false image pair based on the encoder includes: mapping the RGB information of the first true and false image pair to a latent space based on the encoder.

[0009] As described above, the deep fake image detection method based on two-stage learning, wherein the process of obtaining the first universal fake feature information based on the first global feature includes: extracting the first universal fake feature information in the first global feature based on a first universal disguise feature extractor.

[0010] The deep forged image detection method based on two-stage learning as described above, wherein the image is detected based on the first universal forged feature information and the parameters of the initial detection model are updated to obtain a disentangled forged image detection model, comprising:

[0011] Based on the first universal forged feature classification head, the first universal forged feature information is processed to obtain a first detection result, and the binary cross entropy loss of the first detection result and the true and false labels of the first true and false image pair is calculated, and the parameters of the initial detection model are updated based on the binary cross entropy loss to obtain a disentangled forged image detection model; the parameters of the initial detection model include parameters of the first universal camouflage feature extractor and parameters of the first universal forged feature classification head.

[0012] The deep fake image detection method based on two-stage learning as described above, wherein obtaining the second universal fake feature information, the specific fake method, the gender and the race based on the second global feature comprises:

[0013] Acquire the second universal forged feature information based on the universal camouflage feature extractor, and process the second universal forged feature information based on the second universal forged feature classification head to obtain a second detection result;

[0014] Acquire specific forgery method information based on a specific forgery method feature extractor, and process the specific forgery method information based on a specific forgery method classification head to obtain multiple specific forgery method probabilities;

[0015] Acquire gender information based on a gender feature extractor, and obtain gender by processing the gender information based on a gender classification head;

[0016] The racial information is obtained based on the racial feature extractor, and the racial category is obtained by processing the racial information based on the racial classification head.

[0017] In the above-mentioned deep forgery image detection method based on two-stage learning, the process of calculating the feature classification loss of the second general forgery feature information, specific forgery method, gender and race includes:

[0018] The feature classification loss of the second universal forged feature information and gender is calculated using binary cross entropy loss, and the feature classification loss of the specific forgery method and race is calculated using multi-classification cross entropy loss.

[0019] The deep fake image detection method based on two-stage learning as described above, wherein the feature classification loss of the second universal fake feature information and gender is calculated using binary cross entropy loss, comprises:

[0020] The feature classification loss formula for calculating the second universal forgery feature information is as follows:

[0021] ,

[0022] in, is the universal forgery feature loss, N is the total number of samples of true and false image pairs, is the true and false label of the second true and false image pair, is the second detection result;

[0023] The formula for calculating the feature classification loss of the gender is as follows:

[0024] ,

[0025] Among them, the is the gender feature loss, N is the total number of samples of true and false image pairs, is the gender label of the second true and false image pair, To detect gender.

[0026] The deep fake image detection method based on two-stage learning as described above, wherein the feature classification loss of the specific fake method and race is calculated using multi-classification cross entropy loss, including:

[0027] The formula for calculating the feature classification loss for the specific forgery method is as follows:

[0028] ,

[0029] in, is the feature loss of a specific forgery method, N is the total number of samples of true and false image pairs, For the type of forgery method, is the specific forgery method label of the second true and false image pair, The probability of detecting multiple specific counterfeiting methods;

[0030] The formula for calculating the feature classification loss of the race is as follows:

[0031] ,

[0032] in, is the ethnic feature loss, N is the total number of samples of true and false image pairs, For racial types, is the ethnicity label of the second true and false image pair, The racial category to be tested.

[0033] The deep fake image detection method based on two-stage learning as described above, wherein updating the parameters of the disentangled fake image detection model based on the feature classification loss includes:

[0034] Calculating a latent space contrast loss based on the feature classification loss, and updating parameters of the disentangled forged image detection model based on the latent space contrast loss;

[0035] The formula for calculating the latent space contrast loss is as follows:

[0036] ,

[0037] in, is the latent space contrast loss, is the model learning rate, , They are the contrast losses calculated using the real image as the anchor and the fake image as the anchor, respectively.

[0038] In the deep fake image detection method based on two-stage learning as described above, the parameters of the disentangled fake image detection model are updated based on the latent space contrast loss, and the formula used is as follows:

[0039] ,

[0040] in, To disentangle all parameters of the forged image detection model, is the model learning rate, is the gradient obtained by backpropagation of the loss function.

[0041] The beneficial effects of the deep fake image detection method based on two-stage learning proposed in the present invention are:

[0042] 1. In the first stage of the present invention, the parameters of the initial detection model are updated by inputting true and false image pairs for training, thereby obtaining a detection model with prior knowledge.

[0043] 2. The second stage of the present invention can effectively detect the authenticity, forgery method, gender and race of the image, and iteratively update the model parameters to improve the detection success rate of unknown images. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Flowchart of a deep fake image detection method based on dual-stage learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0046] The embodiment of the present invention provides a deep fake image detection method based on two-stage learning, see Figure 1 ,include:

[0047] S1. Input a first true and false image pair into an initial detection model, obtain a first global feature of the first true and false image pair based on an encoder, obtain first universal forged feature information based on the first global feature, detect an image based on the first universal forged feature information and update the parameters of the initial detection model to obtain a disentangled forged image detection model;

[0048] S2. Input a second true and false image pair into the disentangled forged image detection model, obtain a second global feature of the second true and false image pair based on the encoder, obtain second universal forgery feature information, specific forgery method, gender and race based on the second global feature, calculate the feature classification loss of the second universal forgery feature information, specific forgery method, gender and race, and update the parameters of the disentangled forged image detection model based on the feature classification loss.

[0049] In some embodiments, the process of executing step S1 includes:

[0050] S1.1. Input a first true-false image pair into an initial detection model, obtain a first global feature of the first true-false image pair based on an encoder, and obtain first universal forgery feature information based on the first global feature;

[0051] S1.2. Detect an image based on the first universal forgery feature information and update parameters of the initial detection model to obtain a disentangled forgery image detection model.

[0052] Specifically, when executing step S1.1, it includes:

[0053] S1.1.1, inputting the first true and false image pair into the initial detection model;

[0054] S1.1.2. Obtaining a first global feature of the first true and false image pair based on the encoder;

[0055] S1.1.3. Obtain first universal forgery feature information based on the first global feature.

[0056] Specifically, when executing step S1.1.2, obtaining the first global feature of the first true and false image pair based on the encoder includes: mapping RGB information of the first true and false image pair to a latent space based on the encoder.

[0057] Furthermore, when executing step S1.1.3, obtaining the first universal forged feature information based on the first global feature includes: extracting the first universal forged feature information in the first global feature based on a first universal camouflage feature extractor.

[0058] In fact, the Efficinet-b4 network used by the encoder uses the extracted last layer features as the extracted first global features with a shape of (1, 1024), and then the first universal forged feature information with a shape of (1, 256) is extracted by the first universal forged feature extractor.

[0059] Specifically, when executing step S1.2, detecting an image based on the first universal forgery feature information and updating the parameters of the initial detection model to obtain a disentangled forgery image detection model, it includes:

[0060] Based on the first universal forged feature classification head, the first universal forged feature information is processed to obtain a first detection result, and the binary cross entropy loss of the first detection result and the true and false labels of the first true and false image pair is calculated, and the parameters of the initial detection model are updated based on the binary cross entropy loss to obtain a disentangled forged image detection model; the parameters of the initial detection model include parameters of the first universal camouflage feature extractor and parameters of the first universal forged feature classification head.

[0061] In fact, by updating the parameters of the initial detection model, the initial detection model can learn the prior knowledge of how to distinguish between true and false images, so that the second stage of the disentangled forged image detection model can be trained to detect true and false images more accurately.

[0062] In some embodiments, the process of executing step S2 includes:

[0063] S2.1. Inputting a second true-false image pair into the disentangled forged image detection model, and obtaining a second global feature of the second true-false image pair based on the encoder;

[0064] S2.2, obtaining second universal forgery feature information, specific forgery method, gender and race based on the second global feature;

[0065] S2.3, calculating the feature classification loss of the second general forgery feature information, specific forgery method, gender and race;

[0066] S2.4. Update the parameters of the disentangled forged image detection model based on the feature classification loss.

[0067] Specifically, when executing step S2.2, obtaining the second universal forgery feature information, the specific forgery method, the gender and the race based on the second global feature includes:

[0068] Acquire the second universal forged feature information based on the universal camouflage feature extractor, and process the second universal forged feature information based on the second universal forged feature classification head to obtain a second detection result;

[0069] Acquire specific forgery method information based on a specific forgery method feature extractor, and process the specific forgery method information based on a specific forgery method classification head to obtain multiple specific forgery method probabilities;

[0070] Acquire gender information based on a gender feature extractor, and obtain gender by processing the gender information based on a gender classification head;

[0071] The racial information is obtained based on the racial feature extractor, and the racial category is obtained by processing the racial information based on the racial classification head.

[0072] In practice, the disentangled forged image detection model is usually defined as , represents all the parameters in the disentangled forged image detection model, where the parameters of the encoder are , the parameters of each feature extractor are , the classification head parameters are .

[0073] Furthermore, for the general forgery feature information detection task, the final output of the model is the probability of true or false, and the corresponding labels are 0 for true images and 1 for fake images; for specific forgery method detection, the final output of the model is the probability of the forgery method of the fake image in the training set, and the corresponding labels are 0 for true images, 1 for forgery method 1, 2 for forgery method 2, and so on; for the gender detection module, the final output of the model is the probability of male or female, and the corresponding labels are 1 for male and 0 for female; for the race detection module, the final output of the model is the probability of the person in the picture being white, black, or yellow, and the corresponding labels are 0, 1, and 2 respectively.

[0074] Specifically, when executing step S2.3, calculating the feature classification loss of the second general forgery feature information, specific forgery method, gender and race includes:

[0075] S2.3.1, the second general forged feature information, gender feature classification loss is calculated using binary cross entropy loss;

[0076] S2.3.2. The feature classification loss of the specific forgery method and race is calculated using multi-classification cross entropy loss.

[0077] Specifically, when executing step S2.3.1, the feature classification loss of the second universal forged feature information and gender is calculated using binary cross entropy loss, including:

[0078] The feature classification loss formula for calculating the second universal forgery feature information is as follows:

[0079] ,

[0080] Among them, the is the universal forgery feature loss, N is the total number of samples of true and false image pairs, is the true and false label of the second true and false image pair, is the second detection result;

[0081] The formula for calculating the feature classification loss of the gender is as follows:

[0082] ,

[0083] Among them, the is the gender feature loss, N is the total number of samples of true and false image pairs, is the gender label of the second true and false image pair, To detect gender.

[0084] Specifically, when executing step S2.3.2, the feature classification loss of the specific forgery method and race is calculated using multi-classification cross entropy loss, including:

[0085] The formula for calculating the feature classification loss for the specific forgery method is as follows:

[0086] ,

[0087] Among them, the is the feature loss of a specific forgery method, N is the total number of samples of true and false image pairs, is the specific forgery method label of the second true and false image pair, The probability of detecting multiple specific counterfeiting methods;

[0088] The formula for calculating the feature classification loss of the race is as follows:

[0089] ,

[0090] Among them, the is the ethnic feature loss, N is the total number of samples of true and false image pairs, is the ethnic label of the second true and false image pair, The racial category to be tested.

[0091] Specifically, when executing step S2.4, updating the parameters of the disentangled forged image detection model based on the feature classification loss includes:

[0092] S2.4.1. Calculate latent space contrast loss based on the feature classification loss;

[0093] S2.4.2. Update the parameters of the disentangled forged image detection model based on the latent space contrast loss.

[0094] Specifically, when executing step S2.4.1 and calculating the latent space contrast loss based on the feature classification loss, the formula for calculating the latent space contrast loss is as follows:

[0095] ,

[0096] Among them, the is the latent space contrast loss, is the model learning rate, , They are the contrast losses calculated using the real image as the anchor and the fake image as the anchor, respectively.

[0097] Specifically, when executing step S2.4.2, the parameters of the disentangled forged image detection model are updated based on the latent space contrast loss, and the updating is performed using the following formula:

[0098] ,

[0099] in, To disentangle all parameters of the forged image detection model, is the model learning rate, is the gradient obtained by backpropagation of the loss function.

[0100] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. A deep fake image detection method based on two-stage learning, characterized in that: The following steps are involved: Inputting a first true and false image pair into an initial detection model, obtaining a first global feature of the first true and false image pair based on an encoder, obtaining first universal forged feature information based on the first global feature, processing the first universal forged feature information based on a first universal forged feature classification head to obtain a first detection result, and calculating a binary cross entropy loss between the first detection result and the true and false labels of the first true and false image pair, and updating the parameters of the initial detection model based on the binary cross entropy loss to obtain a disentangled forged image detection model; the parameters of the initial detection model include parameters of a first universal camouflage feature extractor and parameters of a first universal forged feature classification head; Input a second true and false image pair into the disentangled forged image detection model, obtain a second global feature of the second true and false image pair based on the encoder, obtain second universal forgery feature information, specific forgery method, gender, and race based on the second global feature, calculate feature classification loss of the second universal forgery feature information, specific forgery method, gender, and race, wherein the feature classification loss of the second universal forgery feature information and gender is calculated using a binary cross entropy loss, and the feature classification loss of the specific forgery method and race is calculated using a multi-classification cross entropy loss; Calculating a latent space contrast loss based on the feature classification loss, and updating parameters of the disentangled forged image detection model based on the latent space contrast loss; The formula for calculating the latent space contrast loss is as follows: ； in, is the latent space contrast loss, is the model learning rate, They are general forgery feature loss, gender feature loss, specific forgery method feature loss, and ethnic feature loss. , They are the contrast losses calculated using the real image as the anchor and the fake image as the anchor, respectively.

2. A deep fake image detection method based on two-stage learning according to claim 1, characterized in that: Acquiring a first global feature of the first true and false image pair based on the encoder includes: mapping RGB information of the first true and false image pair to a latent space based on the encoder.

3. A deep fake image detection method based on two-stage learning according to claim 1, characterized in that: The process of acquiring the first universal forged feature information based on the first global feature includes: extracting the first universal forged feature information in the first global feature based on a first universal forged feature extractor.

4. According to claim 1, a deep fake image detection method based on two-stage learning is characterized in that: Acquiring second universal forgery feature information, specific forgery method, gender, and race based on the second global feature includes: Acquire the second universal forged feature information based on the universal camouflage feature extractor, and process the second universal forged feature information based on the second universal forged feature classification head to obtain a second detection result; Acquire specific forgery method information based on a specific forgery method feature extractor, and process the specific forgery method information based on a specific forgery method classification head to obtain multiple specific forgery method probabilities; Acquire gender information based on a gender feature extractor, and obtain gender by processing the gender information based on a gender classification head; The racial information is obtained based on the racial feature extractor, and the racial category is obtained by processing the racial information based on the racial classification head.

5. According to claim 1, a deep fake image detection method based on dual-stage learning is characterized in that: The second general forged feature information, gender feature classification loss is calculated using binary cross entropy loss, including: The feature classification loss formula for calculating the second universal forgery feature information is as follows: ； in, is the universal forgery feature loss, N is the total number of samples of true and false image pairs, is the true and false label of the second true and false image pair, is the second test result; The formula for calculating the feature classification loss of the gender is as follows: ； in, is the gender feature loss, N is the total number of samples of true and false image pairs, is the gender label of the second true and false image pair, To detect gender.

6. A deep fake image detection method based on two-stage learning according to claim 1, characterized in that: The feature classification loss for the specific forgery method and race is calculated using multi-classification cross entropy loss, including: The formula for calculating the feature classification loss for the specific forgery method is as follows: ； in, is the feature loss of a specific forgery method, N is the total number of samples of true and false image pairs, For the type of forgery method, is the specific forgery method label of the second true and false image pair, The probability of detecting multiple specific counterfeiting methods; The formula for calculating the feature classification loss of the race is as follows: ； in, is the ethnic feature loss, N is the total number of samples of true and false image pairs, For racial types, is the ethnicity label of the second true and false image pair, The racial category to be tested.

7. A deep fake image detection method based on two-stage learning according to claim 1, characterized in that: The parameters of the disentangled forged image detection model are updated based on the latent space contrast loss, using the following formula: ； in, To disentangle all parameters of the forged image detection model, is the model learning rate, is the gradient obtained by backpropagation of the loss function.

Images