Method for mutual generation of fake data for deepfake authentication model

Abstract

The application discloses a method for generating fake data for a DeepFake authentication model, which comprises the following steps: performing face detection on an input image to extract a face image; generating a face contour based on the input face image; searching for a face contour in a face database that matches the face contour to a threshold; extracting an inner region of the face contour of the input face image and an outer region of the matched face contour in the face database; and fusing the inner region of the face contour of the input face image after smoothing and compression with the outer region of the matched face contour in the face database to generate fake face data with artifact information. The application can realize the rapid generation of fake face data and meet the training needs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of DEEPFAKE forgery identification model training, in particular to a mutual generation method of forgery data for a DeepFake identification forgery model. BACKGROUND

[0002] In the process of DEEPFAKE forgery identification model training, data is crucial, and the increase in data quantity and data variety can greatly improve the performance of the model. Data can be divided into forgery data and real data. In the actual training process, real data can be easily obtained through video and media website crawlers, but the generation of forgery data is difficult.

[0003] The difficulty of generating forgery data through existing generation algorithms mainly includes:

[0004] a) Poor stability of generation algorithm performance: Currently, the generation algorithm based on GAN network has poor stability in the process of generating a large number of faces, most of which are not usable and need to be selected by hand. In actual process, the proportion of high-quality usable faces in the total generated faces is not more than 20%.

[0005] b) Low efficiency of generation algorithm: In the actual training process, the demand for data is large, with a magnitude of hundreds of thousands to millions, or even tens of millions. The generation efficiency of the generation algorithm based on GAN network is too low. If ten thousand faces are generated, without considering the selection of faces later, a single 8-card GPU server still needs several weeks of time.

[0006] c) High requirement of generation algorithm on computing resources: The generation algorithm based on GAN network has high requirements on the memory and GPU performance of the server in the training and forward process, and the server for generating the algorithm requires a large amount of investment. Therefore, small enterprises, small laboratories and individual users are difficult to generate a large amount of data. SUMMARY

[0007] The purpose of the present application is to solve the technical defects in the prior art and provide a mutual generation method of forgery data for a DeepFake identification forgery model, which can realize the rapid generation of face forgery data.

[0008] The technical solution adopted by the present application to achieve the purpose is:

[0009] A mutual generation method of forgery data for a DeepFake identification forgery model, comprising

[0010] detecting the face of the input image and extracting the input face image;

[0011] generating a face contour according to the input face image;

[0012] searching for a face contour in the face database that matches the face contour to a threshold;

[0013] extracting an inner region of the face contour of the input face image and an outer region of the matched face contour in the face database;

[0014] filling the inner region of the face contour of the input face image into the outer region of the matched face contour in the face database for fusion;

[0015] performing a smoothing operation on the pixels around the face contour of the fused image to generate a fake face data with artifact information.

[0016] Preferably, a Gaussian smoothing algorithm is used to perform the smoothing operation on the pixels around the face contour of the fused image.

[0017] Preferably, the Hu moment contour matching algorithm is used to search for a face contour in the face database that matches the face contour to a threshold.

[0018] Preferably, the step of generating a face contour according to the input face image is extracting key points from the face image, and then forming a face contour based on the key points.

[0019] Preferably, a face key point detection algorithm is used to extract face key points from the input face image, and then the key points of the edges are sequentially connected in order to form a face contour.

[0020] Preferably, a contour extraction method is used to extract the inner region of the face contour of the input face image and the outer region of the matched face contour in the face database;

[0021] The extraction step of the contour extraction method is:

[0022] performing pure 0 pixel filling on the face contour to be processed to obtain a face contour Mask;

[0023] performing traversal processing on the face image to be processed based on the face contour Mask to obtain the outer region and the inner region of the face contour.

[0024] Preferably, before the step of extracting the input face image, the method further comprises the step of:

[0025] decoding the input data with correct format to uniformly convert the data format into an image or an image sequence, and for the image sequence, inputting each image for face detection to extract the input face image.

[0026] Preferably, the face detection on the input image is based on a face detection algorithm, and the face image extracted from the input is achieved by detecting the face and extracting the face image within the face detection frame.

[0027] The input data includes video data, streaming media data, image sequences, and camera sequences.

[0028] The method for generating fake data for a DeepFake authentication model can quickly generate fake face data, meeting the training needs. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The flowchart of the method for generating fake data for a DeepFake authentication model.

[0030] Figure 2 The process diagram of the method for generating fake data for a DeepFake authentication model.

[0031] Figure 3 The diagram of extracting the inner and outer regions of the face contour for fusion to form a fused face.

[0032] Figure 4 The diagram of extracting the inner and outer regions of the face contour after extracting the face contour for fusion to form a fused face.

[0033] Figure 5 The diagram of performing Gaussian filtering on the fused face. DETAILED DESCRIPTION

[0034] The application will be further described below in conjunction with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and do not limit the application.

[0035] As shown in Figure 1 The method for generating fake data for a DeepFake authentication model includes the following steps:

[0036] Face detection on the input image to extract the input face image;

[0037] Generating a face contour based on the input face image;

[0038] Searching for a face contour in the face database that matches the face contour to a threshold value;

[0039] Extracting the inner region of the face contour of the input face image and the outer region of the matched face contour in the face database;

[0040] filling the inner region of the face contour of the input face image into the outer region of the face contour matched by the face database to perform fusion;

[0041] performing smoothing operation on the peripheral pixels of the face contour of the fused image to generate the fake face data with artifact information.

[0042] In the embodiment of the present application, by filling the inner region of the face contour of the input face image into the outer region of the face contour matched by the face database, the inner region of the face contour of the input face image is fused with the outer region of the face contour matched by the face database, and the peripheral pixels of the face contour of the fused image are smoothed, so that the face data for training can be quickly generated.

[0043] In the embodiment of the present application, before the step of performing face detection on the input image and extracting the input face image, the following steps are further included:

[0044] The input data in correct format is decoded, and the data format is uniformly converted into an image or an image sequence, and for the image sequence, face detection is performed on each image to extract the input face image.

[0045] By converting the input data into image data, subsequent face detection and face contour extraction processing are facilitated.

[0046] The face detection algorithm can be based on a face detection algorithm, and after detecting the face, the face image defined by the face detection frame is extracted.

[0047] The aforementioned face detection algorithm is a prior art, and any available face detection algorithm can be used to implement it, and thus further description is omitted.

[0048] In the embodiment of the present application, the input data to be processed can be various data including video data, streaming media data, image sequences, camera capture sequences, etc.

[0049] As an optional embodiment, when performing face contour matching search or lookup, an existing contour matching algorithm can be used, such as Hu moment contour matching algorithm, to search for a face contour in the face database that matches the face contour to a threshold.

[0050] As a preferred embodiment, in the present application, a Gaussian smoothing algorithm is used to perform smoothing operation on the peripheral pixels of the face contour of the fused image. When a Gaussian kernel filter is used for smoothing filtering, the filtering path is the face contour, i.e. the face contour, and specifically, refer to Figure 4 for smoothing processing along the filtering path in the filtering direction.

[0051] As an optional embodiment, the step of generating the face contour from the input face image is to extract the key points from the face image, and then form the face contour based on the key points.

[0052] In a specific implementation, a face key point detection algorithm can be used to extract the face key points from the input face image, and then sequentially connect the key points of the edges in order to form the face contour.

[0053] For example, a 68-point detection algorithm can be used to extract the face key points from the input face image, including the eyes, nose, mouth area, and the like, and then sequentially connect the key points of the edges in order to form the required face contour. Please refer to Figure 3 , Figure 3 As shown in the first, second, third, and fourth columns on the left side, the first column represents the input image, the second column represents the key point detection on the input image, the third column represents the face key points obtained, and the fourth column represents the face contour obtained based on the key points.

[0054] As an optional embodiment, when the inner region of the face contour of the input face image and the outer region of the face contour matched in the face database are extracted, a contour extraction method is used to implement.

[0055] Please refer to Figure 3 shown, wherein the extraction step of the contour extraction method is:

[0056] The inner region of the face contour of the face image to be processed is filled with pure 0 pixels to obtain a face contour Mask; the face image to be processed is processed based on the face contour Mask to obtain the outer region and the inner region of the face contour.

[0057] The face image to be processed can be an input image or a face image corresponding to the face contour obtained by searching in the database.

[0058] Specifically, please refer to Figure 3 shown in the fifth, sixth, seventh, and eighth columns on the left side, wherein the fifth column represents the effect diagram of placing the extracted face contour on the face, and from the effect diagram, it can be seen that the face contour fits well in the face. After obtaining the face contour, the face contour is filled with pure 0 pixels to obtain a contour Mask, as shown in the sixth column, and by using the face contour Mask, the outer region and the inner region of the contour can be quickly obtained through logical operations between pixels, as shown in the seventh and eighth columns. Figure 3

[0059] ​The contour extraction calculation process is as follows:

[0060] Let the input image be I input , the contour Mask image be I mask , the input image is equal in size to the contour, and the intersection symbol represents the process of logical operation of each pixel of two images, C=A∩B represents that the output image C is formed by traversing image B using image A, if the pixel value in image B is greater than 0, that is, True, the pixel value of image A is retained in the corresponding position of the output image C, if the pixel value in image B is 0, that is, False, 0 is filled in the corresponding position of the output image C.

[0061] Specifically, the contour inside region I in_contour and the contour outside region I out_contour can be represented as follows, respectively:

[0062] I in_contour =I input ∩(1-I mask )

[0063] I out_contour =I input ∩I mask .

[0064] In order to verify the effect of the method of the application, the training data set of the fake data is processed by the method of the application, a large amount of fake data is generated, the detection model is trained, and after the detection model is trained, it is tested on different test sets, and the test results are shown in the following table.

[0065] Mainly verify the performance of the detection model on the WildDeepfake, Celeb-DF, Kaggle DFDC three test data sets, that is, the detection accuracy of the detection model on the test set, as shown in Table 1, the percentage in the table represents the accuracy, randomly select 1000 real videos in the WildDeepfake, Celeb-DF, Kaggle DFDC three test sets, 1000 fake videos, decode, sample, get 30241 real images and 32416 fake images. After testing all the real images and fake images, the detection accuracy of the network for real and fake images is shown in the table.

[0066] Among them, the training data set is Deeper Forensics 1.0 data set, which contains 50000 real videos and 10000 fake videos, after decoding, 6432221 real images and 1345217 fake images are obtained.

[0067] In the following table, residual network + Resize represents that the fake data used for model training is generated by randomly scaling the fake data (i.e., fake images) to form data in which the number of fake images is the same as that of real images; residual network + Blur represents that the fake data used for model training is generated by randomly blurring fake images to form data in which the number of fake images is the same as that of real images; and residual network + Resize + Blur represents that the fake data used for model training is generated by randomly scaling and blurring fake images to form data in which the number of fake images is the same as that of real images, wherein the number of images generated by random scaling is the same as that by random blurring.

[0068] Residual network + Cross-generation represents that the fake data used for model training is generated by using a cross-generation method of facial contours to generate fake face data, so that the number of fake face images is the same as that of real images.

[0069]

[0070] Table 1

[0071] The fake detection model can be a residual network, and the residual network can be ResNet18 or ResNet50. The fake detection model uses ResNet18 / ResNet50 for binary classification, and the output number is 2. For the label of the training data, 0 represents a real image, and 1 represents a fake image. The output of the residual network is between 0 and 1. If the output value is less than 0.5, it represents that the network recognizes the input image as a real image. If the output value is greater than or equal to 0.5, it represents that the network recognizes the input image as a fake image.

[0072] It can be seen that the contour search method is used in the present application. After face fusion, smoothing filtering is performed near the contour. Meanwhile, similar face images in the database can be generated from each other. Therefore, the present application is suitable for face generation tasks with large data volume and high precision.

[0073] The basic principles and main features of the present application and the advantages of the present application are shown and described above. It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application, and any reference signs in the claims should not be regarded as limiting the involved claims.

[0074] Furthermore, it should be understood that although the specification is described in terms of embodiments, not every embodiment includes every feature described. The specification can include implicit combinations of explicitly mentioned features and / or explicit combinations of implicitely mentioned features. Each embodiment depends on the explicit combinations of features and / or the implicit combinations of features made specifically within that embodiment, and each such embodiment can be combined with every other such embodiment to create further embodiments.

Claims

1. A method for generating forged data for use in DeepFake authentication models, characterized in that, Including the following steps: Perform face detection on the input image and extract the input face image; Generate a facial outline based on the input facial image; Search the face database for face contours whose matching degree with the stated face contour meets the threshold; Extract the inner region of the face contour of the input face image and the outer region of the face contour matched in the face database; The inner region of the face contour of the input face image is filled into the outer region of the face contour matched by the face database for fusion. Smoothing operation is performed on the pixels surrounding the face contour in the fused image to generate fake face data with artifact information; The inner region of the face contour of the input face image and the outer region of the face contour matched in the face database are extracted using a contour extraction method. The extraction steps of the contour extraction method are as follows: Fill the inner region of the face contour to be processed with pure 0 pixels to obtain the face contour mask; The face image to be processed is traversed based on the face contour mask to obtain the outer and inner regions of the face contour.

2. The method for generating forged data for DeepFake detection models according to claim 1, characterized in that, A Gaussian smoothing algorithm is used to smooth the pixels around the face contour in the fused image.

3. The method for generating forged data for DeepFake detection models according to claim 1 or 2, characterized in that, The Hu-moment contour matching algorithm is used to search the face database for face contours that match the threshold.

4. The method for generating forged data for DeepFake detection models according to claim 3, characterized in that, The step of generating a face contour based on the input face image is to extract key points from the face image and then form a face contour based on the key points.

5. The method for generating forged data for DeepFake detection models according to claim 4, characterized in that, A facial landmark detection algorithm is used to extract facial landmarks from the input facial image, and then the landmarks on the edges are connected in sequence to form a facial outline.

6. The method for generating forged data for DeepFake detection models according to claim 1, characterized in that, Before the steps of performing face detection on the input image and extracting the input face image, the following steps are also included: Decode the correctly formatted input data and convert the data format into an image or image sequence. For image sequences, perform face detection on each input image and extract the input face image.

7. The method for generating forged data for DeepFake detection models according to claim 1 or 6, characterized in that, Face detection is performed on the input image, and the extraction of the input face image is achieved by using a face detection algorithm to detect faces and then extracting the face image defined by the face detection box.

8. The method for generating forged data for a DeepFake authentication model according to claim 7, characterized in that, Input data includes video data, streaming media data, image sequences, and camera capture sequences.

Images