A method and device for detecting a fake face-swap image based on an identity recognition probability distribution

Abstract

The application discloses a kind of based on identity recognition probability distribution's fake face swapping image detection method and device, construct real image and corresponding identity label, identity label is carried out smoothing operation to construct smooth identity label;Real image is used to carry out multiple rounds of identity label-based supervision learning to identity recognition model to optimize parameter;Copy current parameter optimization identity recognition model as copy model, based on the pixel area of greatest contribution in the prediction result of copy model in real image is shielded to construct shielded image;Shielded image is used to carry out multiple rounds of smooth identity label-based supervision learning to current parameter optimization identity recognition model to optimize parameter;Until training ends, the maximum prediction probability value output by identity recognition model is compared with threshold value, when maximum prediction probability value is less than threshold value, it is detected as fake face swapping image.This method prevents attack while improving the detection accuracy and generalization of fake face swapping image.

Description

Technical Field

[0001] This invention belongs to the field of deepfake detection technology, specifically relating to a method and apparatus for detecting deepfake face-swapping images based on identity recognition probability distribution. Background Technology

[0002] In recent years, with the development of deep generative technologies such as Generative Adversarial Networks (GANs), the generation effect of face-swapping images has become more realistic and the generation process has become simpler. As a result, the threat to public opinion security has become more serious, and there is an urgent need to study reliable methods for detecting fake face-swapping images.

[0003] Most existing methods for detecting fake face-swapping images model it as a binary classification problem, collecting a large number of fake face-swapping images and real face images to train a deep neural network for classification, which is then used to detect the authenticity of the image to be tested. Examples include a deep fake image detection method using joint multi-scale features disclosed in patent document CN115100128A, and a face forgery detection method disclosed in patent document CN115240243A. These methods are prone to overfitting to the fake face-swapping images in the training set, and their accuracy decreases when detecting images generated by face-swapping methods not included in the training set. Therefore, they struggle to cope with rapidly evolving face-swapping techniques.

[0004] Another type of method improves the generalization of detection methods across different face-swapping techniques by studying and extracting common forgery traces caused by the face-swapping forgery process, such as detecting face blending boundary traces. However, these traces are easily destroyed by image compression, blurring and other operations, so this type of method is difficult to use for detecting face-swapping images whose quality has been reduced after being spread through the network.

[0005] Furthermore, existing research on detecting fake face-swapped images rarely addresses gray-box attack scenarios where face-swappers are aware of and attempt to bypass detection methods, thus leaving them vulnerable to being bypassed by face-swappers. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide a method and apparatus for detecting fake face-swapping images based on identity recognition probability distribution, which improves the detection accuracy of fake face-swapping images while defending against attacks.

[0007] To achieve the above-mentioned objectives, an embodiment provides a method for detecting forged face-swapping images based on identity recognition probability distribution, comprising the following steps:

[0008] Step 1: Construct real images and corresponding identity tags, and perform a smoothing operation on the identity tags to construct smooth identity tags;

[0009] Step 2: Use real images to perform multiple rounds of supervised learning based on identity labels on the identity recognition model to optimize the parameters;

[0010] Step 3: Copy the current parameter-optimized identity recognition model as the copy model, and occlude the pixel region that contributes the most to the prediction results of the copy model based on the real image to construct the occluded image;

[0011] Step 4: Use occluded images to perform multiple rounds of supervised learning based on smooth identity labels on the current parameter-optimized identity recognition model to optimize the parameters;

[0012] Step 5: Repeat steps 3 and 4 until training is complete. Extract the final optimized identity recognition model for detecting fake face-swap images, including: comparing the maximum predicted probability value output by the identity recognition model with a threshold. If the maximum predicted probability value is less than the threshold, it is detected as a fake face-swap image.

[0013] Preferably, the smoothing operation refers to combining the unique hot code of the identity tag with a uniform distribution, and the smoothed identity tag vector is represented as:

[0014]

[0015] Among them, y s,c Let α represent the smoothed identity label vector value corresponding to category c, α represent the label smoothing hyperparameter, C represent the total number of identity label categories, and c = target means that category c is the identity label corresponding to the real image.

[0016] Preferably, the hyperparameter α for label smoothing is 0.5.

[0017] Preferably, the identity recognition model is used to predict the output identity recognition probability distribution based on the input image. When real images are used to perform multiple rounds of supervised learning based on identity labels on the current identity recognition model, the cross-entropy between the predicted output identity recognition probability distribution and the one-hot code encoding of the identity label is used as the loss function l. i , represented as:

[0018]

[0019] Among them, z i This represents the probability distribution vector of the i-th image output by the identity recognition model. Indicate z i The identity label y corresponding to the real image i The probability value, z i,c Indicate z i The probability value of identity label c, where C represents the total number of identity label categories, exp() represents the exponential function e, and log() represents the logarithmic function.

[0020] Preferably, the step of constructing an occluded image by occluding the pixel region that contributes the most to the prediction result of the copy model based on the real image includes:

[0021] The probability distribution of identity recognition in the copy model of the real image is calculated, and the probability values ​​corresponding to the identity labels are extracted from the probability distribution and backpropagated to calculate the gradient. Then, the N target pixels with the largest gradient values ​​are selected, and a rectangular region centered on each target pixel is extracted. The pixel values ​​of the N rectangular regions are set to zero to achieve occlusion, and the occluded image is obtained.

[0022] Preferably, when using occluded images to perform multiple rounds of supervised learning based on smooth identity labels on the current parameter-optimized identity recognition model, the cross-entropy between the predicted output identity recognition probability distribution and the smooth identity label vector is used as the loss function l. i ', is represented as:

[0023]

[0024] Among them, z i Let z represent the probability distribution vector of the i-th image output in the identity recognition model. i,c Indicate z i The probability value of identity label c, z i,j Indicate z i The probability value of identity label j, where C represents the total number of identity label categories, exp() represents the exponential function e, log() represents the logarithmic function, and y s,c This represents the smoothed identity label vector value corresponding to category c.

[0025] Preferably, the identity recognition model includes a feature extraction unit for image feature extraction and a recognition unit for predicting the identity recognition probability distribution;

[0026] The feature extraction unit uses the ArcFace model, and the recognition unit uses a fully connected network.

[0027] To achieve the above-mentioned objectives, an embodiment provides a fake face-swapping image detection device based on identity recognition probability distribution, including a tag processing module, a pre-optimization module, an occlusion image construction module, a re-optimization module, and a detection module;

[0028] The label processing module is used to construct real images and corresponding identity labels, and to perform smoothing operations on the identity labels to construct smooth identity labels.

[0029] The pre-optimization module is used to perform multiple rounds of supervised learning based on identity labels on the identity recognition model using real images to optimize the parameters;

[0030] The occlusion image construction module is used to copy the identity recognition model optimized by the current parameters as a copy model, and to occlude the pixel region that contributes the most to the classification result of the copy model based on the real image to construct an occlusion image;

[0031] The re-optimization module is used to perform multiple rounds of supervised learning based on smooth identity labels on the identity recognition model with the current parameters optimized using occluded images in order to optimize the parameters;

[0032] The detection module is used to extract the final parameter optimized identity recognition model for the detection of fake face-swap images, including: comparing the maximum predicted probability value output by the identity recognition model with a threshold, and detecting a fake face-swap image when the maximum predicted probability value is less than the threshold.

[0033] To achieve the above-mentioned objectives, the embodiments also provide a forged face-swapping image detection device based on identity recognition probability distribution, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned forged face-swapping image detection method based on identity recognition probability distribution.

[0034] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0035] This invention leverages the shared essence of falsified images by integrating the identity features of two different identities involved in the face swap. It only requires training the identity recognition model with real images, eliminating the need to train a true / false classification model with existing falsified images, thus exhibiting good generalization to different face swap methods. Furthermore, the identity features are not easily destroyed by image operations such as compression, resulting in good generalization to images of varying quality. Additionally, the designed training strategy expands the attention area of ​​the identity recognition model, thereby enhancing its robustness against gray-box attacks.

[0036] This invention specifically addresses scenarios where a set of protected individuals needs to be detected in forged images. It trains an identity recognition model using the real faces of the protected individuals and performs authenticity detection based on the difference in the identity recognition probability distribution between the face-swapped forged image and the real image. Furthermore, it addresses gray-box attack scenarios where forged image generators are aware of the detection method but lack the specific model parameters. This invention leverages the property that gray-box attacks rely on similar model attention regions. By occluding the region in the image that contributes most to the model's classification and then retraining, the attention region of the identity recognition model is expanded, thereby reducing the success rate of gray-box attacks. Simultaneously, smoothing labels are used in the loss calculation of the occluded image to prevent overfitting. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of a method for detecting fake face-swapping images based on identity recognition probability distribution provided in the embodiment;

[0039] Figure 2 This is a diagram illustrating the training process of the identity recognition model provided in the embodiment;

[0040] Figure 3 This is a flowchart of the detection process for forged face-swapped images using an identity recognition model, provided in the embodiment.

[0041] Figure 4 This is a schematic diagram of the structure of the forged face-swapping image detection device based on identity recognition probability distribution provided in the embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0043] The embodiment provides a method for detecting forged face-swapping images based on identity recognition probability distribution. The core innovation is to use the identity features of the forged face-swapping image to fuse the common feature of the two different identities involved in the face-swapping for detection, thereby achieving face-swapping detection generalization that exceeds that of existing methods. In addition, an identity recognition model training strategy based on attention-based image occlusion and label smoothing is designed for defense against gray-box attack scenarios. It achieves good defense performance in gray-box scenarios where face-swapping generators know the detection method and attempt to bypass it.

[0044] The probability distribution output by the identity recognition model represents the common features of identity fusion in forged face-swapping images, thereby distinguishing between forged face-swapping images and real face images. Specifically, the identity recognition model has higher uncertainty regarding the identity of forged face-swapping images than real face images, and therefore the maximum probability value of the probability distribution is lower than that of real face images. The training of the identity recognition model incorporates two training strategies: attention-based image occlusion and label smoothing, to detect forged face-swapping images subjected to gray-box attacks.

[0045] like Figure 1 As shown in the embodiment, the method for detecting forged face-swapping images based on identity recognition probability distribution includes the following steps:

[0046] Step 1: Construct real images and corresponding identity labels, and perform a smoothing operation on the identity labels to construct smooth identity labels.

[0047] In this embodiment, the real images to be detected as forgeries and their corresponding identity labels are obtained. Specifically, the CelebDF dataset for forgery face-swapping is used as an example. It contains 59 real videos with different identities and forged videos obtained by swapping the faces of any two of these identities. For each identity, 10 frames are randomly selected from all the frames obtained from its real videos, resulting in 590 real face images as sample data for the identity recognition model.

[0048] Label smoothing combines one-hot labels with uniform distribution. Its purpose is to ensure that the identity recognition model pays attention to the entire face region as evenly as possible, preventing overfitting to the remaining face portion after occlusion. The smoothed identity label vector is represented as follows:

[0049]

[0050] Among them, y s,c Let y represent the smoothed identity label vector value corresponding to category c, α represent the label smoothing hyperparameter, preferably 0.5, C represent the total number of identity label categories, which is 59 in this example, and c = y means that category c is the identity label y corresponding to the real image.

[0051] Step 2: Use real images to perform multiple rounds of supervised learning based on identity labels on the identity recognition model to optimize the parameters.

[0052] In this embodiment, the identity recognition model f consists of a feature extraction unit for image feature extraction and a recognition unit for predicting the probability distribution of identity recognition. The identity recognition model f needs to be optimized before it is applied.

[0053] The feature extraction unit encodes the input face image into an identity feature vector. After pre-training on a large number of real face datasets, the extracted identity feature vector has the property that the feature distance between faces with the same identity is close and the feature distance between faces with different identities is far. The recognition unit has the same input dimension as the output feature dimension of the feature extraction unit, and the output dimension is the same as the number of people in the set of protected persons that need to be detected forged images.

[0054] Specifically, the feature extraction unit uses the open-source model ArcFace as an example. Its network structure is IR_SE50, and it is pre-trained on a large-scale face recognition dataset using the ArcFace loss function. The pre-training parameters are publicly available. The input is a 112×112 aligned face image, and the output is a 512-dimensional identity feature vector. The recognition unit uses an FC layer connected to the output of the last layer of ArcFace. The input is a 512-dimensional identity feature vector, and the output dimension is the identity recognition probability distribution of the number of identity label categories. In this example, the output dimension is 59.

[0055] Before training the identity recognition model using real images, the real images are processed, including: using MTCNN to identify face regions, and performing affine transformation on the image to align the face according to the coordinates of the face feature points, cropping the face region by a certain scaling ratio and scaling the size to the input size of ArcFace (112×112); loading the pre-trained parameters of ArcFace, and randomly initializing the parameters of the FC layer.

[0056] When pre-training the identity recognition model for multiple rounds (e.g., 10 rounds), the ArcFace parameters are fixed, only the parameters of the fully connected (FC) layers are updated, and the cross-entropy loss is calculated using the one-hot encoding of the identity labels of real images and the predicted identity recognition probability distribution. i :

[0057]

[0058] Among them, z i This indicates that the i-th training sample outputs a 59-dimensional identity recognition probability distribution vector after passing through the FC layer. Indicate z i The identity label y corresponding to the real image i The probability value, z i,c Indicate z i The probability value of identity label c, where C represents the total number of identity label categories, which is 59, exp() represents the exponential function e, and log() represents the logarithmic function.

[0059] Step 3: Copy the current parameter-optimized identity recognition model as a copy model, and occlude the pixel region that contributes the most to the prediction results of the copy model based on the real image to construct an occluded image.

[0060] In the embodiments, such as Figure 2 As shown, after multiple rounds of parameter optimization of the identity recognition model, a copy of the currently optimized identity recognition model is used as a copy model f'. This copy model f' is used to calculate the occlusion region based on the gradient. Specifically, the real image is input into the copy model f', and the occlusion is constructed based on the pixel region of the real image that contributes the most to the prediction result of the copy model, including:

[0061] Calculate the output identity recognition probability distribution of the real image at the FC layer of the copy model, and extract the identity label y from the identity recognition probability distribution. i Corresponding probability value After backpropagation to calculate the gradient, the N target pixels with the largest gradient values ​​are selected, and a rectangular region centered on each target pixel is extracted. The pixel values ​​of the N rectangular regions are set to zero to achieve occlusion, resulting in an occluded image. Preferably, N is 5. When extracting the rectangular region, two integers between 25 and 35 can be randomly selected as the length and width to obtain a rectangular region.

[0062] Step 4: Use occluded images to perform multiple rounds of supervised learning based on smooth identity labels on the current parameter-optimized identity recognition model to optimize the parameters.

[0063] In this embodiment, when performing multi-round (e.g., 5 rounds) supervised learning based on smooth identity labels on the identity recognition model optimized with the occluded image, the cross-entropy loss l between the predicted output identity recognition probability distribution and the smooth identity label vector is calculated. i The system then backpropagates to update the identity recognition model parameters, optimizing the parameters of the FC layer and the higher-level parameters of the ArcFace model. In this embodiment, the parameters of the 20th convolutional module and below in ArcFace are fixed, while the parameters above the 20th convolutional module are updated. These parameters can be changed according to the actual situation. The cross-entropy loss is... i ' is represented as:

[0064]

[0065] Among them, z i,j Indicate z i The probability value of identity label j in the middle.

[0066] Step 5: Repeat steps 3 and 4 until training is complete. Extract the final optimized identity recognition model for detecting fake face-swap images, including: comparing the maximum predicted probability value output by the identity recognition model with a threshold. If the maximum predicted probability value is less than the threshold, it is detected as a fake face-swap image.

[0067] After step 4, where the identity recognition model is trained multiple times using the occluded image, the copy model is updated with the parameters of the new identity recognition model. This involves extracting the identity recognition model optimized in step 4 as the copy model, and repeating steps 3 and 4 multiple times until the iteration ends. The iteration termination condition can be the loss function of the identity recognition model. i 'convergence.

[0068] After training, the final optimized identity recognition model is used to detect forged face-swapped images. Based on the property that face-swapped images fuse two different identities used for face swapping, the uncertainty of the identity recognition model regarding the identity of face-swapped images is higher than that of real face images. This results in the maximum probability value of face-swapped images being lower than that of real face images. Therefore, forged face-swapped images are identified based on the magnitude of the maximum probability value in the identity recognition probability distribution. Figure 3 As shown, it includes:

[0069] First, the image to be tested is input into the trained identity recognition model to obtain the logits vector output by the FC layer. The identity recognition probability distribution is then calculated using Softmax, as shown in the formula:

[0070]

[0071] Where x represents the image to be tested, and p(x) c Let f(x) represent the probability that the input image is labeled c, exp() represents the exponential function e, and f(x) represent the logits vector output by the identity recognition model. c and f(x) j These represent the values ​​at indices c and j in the logits vector, respectively.

[0072] Then, the maximum probability value p is identified from the identity recognition probability distribution output by the identity recognition model. max This can be expressed as a formula:

[0073]

[0074] Finally, the maximum predicted probability value is compared with a threshold. If the maximum predicted probability value is less than the threshold, it is detected as a fake face-swapped image; otherwise, it is a real image. The threshold is obtained through testing: real and fake test samples are collected, input into the trained identity recognition model, the maximum probability value in the identity recognition probability distribution is obtained, and the optimal threshold for real / fake classification is determined based on the maximum probability value and the real labels of the test samples.

[0075] To test the effectiveness of this invention, tests were conducted on a partial face-swapping forgery dataset following the steps outlined in the specific implementation examples described above. AUC (Area Under Curve) was used as the evaluation metric; a higher AUC (closer to 1.0) indicates better detection performance. Table 1 shows the test results on three datasets: FaceForensics++, Celeb-DF, and DeeperForensics-1.0. FaceForensics++ only includes three face-swapping forgery components: DeepFakes, FaceSwap, and FaceShifter, excluding facial animation forgery. Furthermore, FaceForensics++ incorporates varying degrees of compression.

[0076] Table 1. Authenticity Detection Test Results

[0077]

[0078] As can be seen from the test results in Table 1, the detection results of the present invention are all above 95%, reaching 98.9% and 97.9% under different compression rates of the FaceForensics++ face-swapping fake dataset, demonstrating high detection performance.

[0079] To test the effectiveness of this invention, a gray-box attack defense test was conducted on the CelebDF face-swapping dataset. The Attack Success Rate (ASR) was used as the evaluation metric; the closer the ASR is to 0, the better the defense effect.

[0080] Specifically, a gray-box attack refers to an attacker who knows the detection method but cannot obtain the specific parameters and / or training dataset of the identity recognition model used for detection. Therefore, the attacker uses other available models and / or data to modify the face-swapped forged image, hoping to be recognized as a real image by the detection method. As an example, the gray-box attack implementation method simulated in this instance is as follows: After the face-swapped forged image is generated, the attacker obtains a publicly available pre-trained facial identity feature extraction model, such as ArcFace. The face-swapped forged image and the real image of the target person are input into ArcFace respectively, resulting in two 512-dimensional identity feature vectors. Adversarial noise is added to the face-swapped forged image using the projective gradient descent method, reducing the cosine distance between the identity feature vector obtained from the noisy face-swapped image through ArcFace and the identity feature vector of the target real image. Table 2 shows the ASR after performing the above attack on the forged image in the CelebDF dataset. For comparison, it shows the ASR of the identity recognition model obtained in step 2 with ArcFace fixed and only the FC layer trained, without adopting the training strategies described in steps 3 to 4 of this invention.

[0081] Table 2 Results of graybox attack resistance tests

[0082] method ASR (%) AUC Do not adopt the training strategies described in steps (3-3) to (3-5). 97.11％ 0.9933 Complete steps of the invention 39.07％ 0.9617

[0083] As can be seen from the test results in Table 2, the training strategy described in steps 3 to 4 of this invention reduced ASR from 97% to 39% while maintaining a high AUC, resulting in good defense performance.

[0084] Based on the same inventive concept, the embodiments also provide a forged face-swapping image detection device based on identity recognition probability distribution, such as... Figure 4 As shown, it includes a label processing module, a pre-optimization module, an occluded image construction module, a re-optimization module, and a detection module;

[0085] The system comprises the following modules: a label processing module for constructing real images and corresponding identity labels, and smoothing the identity labels to create smooth labels; a pre-optimization module for performing multiple rounds of supervised learning based on identity labels on the identity recognition model using real images to optimize parameters; an occlusion image construction module for copying the currently optimized identity recognition model as a copy model, and occluding the pixel regions that contribute the most to the classification results of the copy model based on real images to construct occlusion images; a re-optimization module for performing multiple rounds of supervised learning based on smoothed identity labels on the currently optimized identity recognition model using occlusion images to optimize parameters; and a detection module for extracting the final optimized identity recognition model for detecting fake face-swapping images, including comparing the maximum predicted probability value output by the identity recognition model with a threshold, and detecting a fake face-swapping image if the maximum predicted probability value is less than the threshold.

[0086] It should be noted that the forged face-swapping image detection device provided in the above embodiments should be illustrated using the above-described functional module division as an example when performing forged face-swapping image detection. The functions can be assigned to different functional modules as needed, that is, the internal structure of the terminal or server can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the forged face-swapping image detection device and the forged face-swapping image detection method embodiments provided in the above embodiments belong to the same concept; their specific implementation process is detailed in the forged face-swapping image detection method embodiments, and will not be repeated here.

[0087] The forged face-swapping image detection device provided in the above embodiments applies identity features in forged face-swapping image detection. It utilizes the common feature that face-swapping images fuse two different identities used for face-swapping, and distinguishes between forged face-swapping images and real face images through the probability distribution output by the identity recognition model. Furthermore, it incorporates anti-grey-box attack design in the training strategy of the identity recognition model. This invention obtains real images of the set of people to be detected forged images and their corresponding identity labels as a training set; obtains a pre-trained face identity feature extraction model, and adds a fully connected layer at the end as the identity recognition model; trains the identity recognition model using a strategy combining attention-based image occlusion and label smoothing; inputs the images to be detected (real or fake) into the identity recognition model to obtain the identity recognition probability distribution; and identifies the forged face-swapping image based on the maximum probability value in the identity recognition probability distribution. Since the face-swapping image fuses two different identities used for face-swapping, the uncertainty of the identity recognition model for the identity of the face-swapping image is higher than that of the real face image, therefore the maximum probability value of the face-swapping image is lower than that of the real face image. The training steps of this invention use only real images and do not rely on existing fake face-swapping images for training, thus achieving good generalization to different face-swapping methods; at the same time, identity features are not easily destroyed by image operations such as compression, thus achieving good generalization to images of different qualities; and the designed training strategy expands the attention area of ​​the identity recognition model, thus enhancing its robustness to gray-box attacks.

[0088] Based on the same inventive concept, the embodiment also provides a forged face-swapping image detection device based on identity recognition probability distribution, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned forged face-swapping image detection method, including the following steps:

[0089] Step 1: Construct real images and corresponding identity tags, and perform a smoothing operation on the identity tags to construct smooth identity tags;

[0090] Step 2: Use real images to perform multiple rounds of supervised learning based on identity labels on the identity recognition model to optimize the parameters;

[0091] Step 3: Copy the current parameter-optimized identity recognition model as the copy model, and occlude the pixel region that contributes the most to the prediction results of the copy model based on the real image to construct the occluded image;

[0092] Step 4: Use occluded images to perform multiple rounds of supervised learning based on smooth identity labels on the current parameter-optimized identity recognition model to optimize the parameters;

[0093] Step 5: Repeat steps 3 and 4 until training is complete. Extract the final optimized identity recognition model for detecting fake face-swap images, including: comparing the maximum predicted probability value output by the identity recognition model with a threshold. If the maximum predicted probability value is less than the threshold, it is detected as a fake face-swap image.

[0094] In practical applications, the memory can be volatile memory located at the local end, such as RAM; it can also be non-volatile memory, such as ROM, FLASH, floppy disks, hard disks, etc.; or it can be a remote storage cloud. The processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field-programmable gate array (FPGA). That is, the steps of the forged face-swapping image detection method based on identity recognition probability distribution can be implemented through these processors.

[0095] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting forged face-swap images based on identity recognition probability distribution, characterized in that, Includes the following steps: Step 1: Construct real images and corresponding identity tags. Perform a smoothing operation on the identity tags to construct smooth identity tags. The smoothing operation refers to combining the one-hot code of the identity tag with a uniform distribution. The smooth identity tag vector is represented as: Among them, y s,c Let α represent the smoothed identity label vector value corresponding to category c, α represent the label smoothing hyperparameter, C represent the total number of identity label categories, and c = target means that category c is the identity label corresponding to the real image; Step 2: Use real images to perform multiple rounds of supervised learning based on identity labels on the identity recognition model to optimize the parameters; Step 3: Copy the current parameter-optimized identity recognition model as the copy model, and occlude the pixel region that contributes the most to the prediction results of the copy model based on the real image to construct the occluded image; Step 4: Use occluded images to perform multiple rounds of supervised learning based on smooth identity labels on the current parameter-optimized identity recognition model to optimize the parameters; Step 5: Repeat steps 3 and 4 until training is complete. Extract the final optimized identity recognition model for detecting fake face-swap images, including: comparing the maximum predicted probability value output by the identity recognition model with a threshold. If the maximum predicted probability value is less than the threshold, it is detected as a fake face-swap image.

2. The method for detecting forged face-swapping images based on identity recognition probability distribution according to claim 1, characterized in that, The hyperparameter α for label smoothing is set to 0.

5.

3. The method for detecting forged face-swapping images based on identity recognition probability distribution according to claim 1, characterized in that, The identity recognition model is used to predict the output identity recognition probability distribution based on the input image. When real images are used to perform multiple rounds of supervised learning based on identity labels on the current identity recognition model, the cross-entropy between the predicted output identity recognition probability distribution and the one-hot code encoding of the identity label is used as the loss function l. i , represented as: Among them, z i This represents the probability distribution vector of the i-th image output by the identity recognition model. Indicate z i The identity label y corresponding to the real image i The probability value, z i,c Indicate z i The probability value of identity label c, where C represents the total number of identity label categories, exp() represents the exponential function e, and log() represents the logarithmic function.

4. The method for detecting forged face-swapping images based on identity recognition probability distribution according to claim 1, characterized in that, The method of constructing an occluded image by occluding the pixel region that contributes the most to the prediction results of the copy model based on the real image includes: The probability distribution of identity recognition in the copy model of the real image is calculated, and the probability values ​​corresponding to the identity labels are extracted from the probability distribution and backpropagated to calculate the gradient. Then, the N target pixels with the largest gradient values ​​are selected, and a rectangular region centered on each target pixel is extracted. The pixel values ​​of the N rectangular regions are set to zero to achieve occlusion, and the occluded image is obtained.

5. The method for detecting forged face-swapping images based on identity recognition probability distribution according to claim 1, characterized in that, When using occluded images to perform multiple rounds of supervised learning based on smooth identity labels on the current parameter-optimized identity recognition model, the cross-entropy between the predicted output identity recognition probability distribution and the smooth identity label vector is used as the loss function l. i ', is represented as: Among them, z i Let z represent the probability distribution vector of the i-th image output in the identity recognition model. i,c Indicate z i The probability value of identity label c, z i,j Indicate z i The probability value of identity label j, where C represents the total number of identity label categories, exp() represents the exponential function e, log() represents the logarithmic function y s,c This represents the smoothed identity label vector value corresponding to category c.

6. The method for detecting forged face-swapping images based on identity recognition probability distribution according to claim 1, characterized in that, The identity recognition model includes a feature extraction unit for image feature extraction and a recognition unit for predicting the identity recognition probability distribution.

7. A device for detecting forged face-swapping images based on identity recognition probability distribution, characterized in that, It includes a label processing module, a pre-optimization module, an occluded image construction module, a re-optimization module, and a detection module; The label processing module is used to construct real images and corresponding identity labels, and to perform a smoothing operation on the identity labels to construct smooth identity labels. The smoothing operation refers to combining the one-hot code label of the identity label with a uniform distribution. The smooth identity label vector is represented as: Among them, y s,c Let α represent the smoothed identity label vector value corresponding to category c, α represent the label smoothing hyperparameter, C represent the total number of identity label categories, and c = target means that category c is the identity label corresponding to the real image; The pre-optimization module is used to perform multiple rounds of supervised learning based on identity labels on the identity recognition model using real images to optimize the parameters; The occlusion image construction module is used to copy the identity recognition model optimized by the current parameters as a copy model, and to occlude the pixel region that contributes the most to the classification result of the copy model based on the real image to construct an occlusion image; The re-optimization module is used to perform multiple rounds of supervised learning based on smooth identity labels on the identity recognition model with the current parameters optimized using occluded images in order to optimize the parameters; The detection module is used to extract the final parameter optimized identity recognition model for the detection of fake face-swap images, including: comparing the maximum predicted probability value output by the identity recognition model with a threshold, and detecting a fake face-swap image when the maximum predicted probability value is less than the threshold.

8. A device for detecting forged face-swapping images based on identity recognition probability distribution, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for detecting fake face-swapping images based on identity recognition probability distribution as described in any one of claims 1-7.

Images