Method and system for training false face detection depth model

Abstract

The invention discloses a method and system for training a false face detection model. The method comprises the following steps: collecting a real face image and a false face image, and calculating to obtain a false salient region through a gradient deep learning back propagation principle; randomly generating an erasure mask based on the counterfeit salient region; inputting the false face image and the erasing mask into an enhancement network to obtain an enhanced image; inputting the real face image and the enhanced image into a false face detection model, and performing adversarial training based on a first loss function; inputting the attack picture into an adversarial generator to obtain an adversarial enhanced picture; and inputting the adversarial enhanced picture into a false face detection model, and performing adversarial training based on a second loss function to obtain a false face detection depth model. According to the method, a deceptive enhanced image is obtained through an adversarial enhancement mode, and generalization performance and detection accuracy are further improved through improvement of a generative network architecture.

Description

Technical Field

[0001] The invention relates to a method for training a fake face detection depth model and also to a system for implementing the method, belonging to the technical field of digital image processing. Background Art

[0002] The development of deep learning technology, especially generative adversarial networks (GAN), has greatly promoted the progress of image synthesis technology. These technologies can synthesize large-scale, highly realistic, and detailed images, and their quality can even exceed that of real images. Deepfake technology is a typical application in this field. Given an original image and a target image or video, it can replace the face in the target with the face in the original image, and make it difficult to distinguish between the real and the fake. Nevertheless, the media synthesized by deepfake technology still have some high-frequency details and inconsistent information, which can be distinguished by deep network learning.

[0003] In the field of deep fake technology detection, it is usually constructed as a binary classification problem. That is, for the input medium, the network will give a score to determine whether the medium is real or synthetic. Most of the improvement work focuses on the optimization of deep network structure, with the aim of extracting more discriminative features to improve the accuracy of classification.

[0004] In the Chinese invention patent with the authorization announcement number CN111753729B, a fake face detection method is disclosed. The method predicts the face difference map corresponding to the target image by using a network model, and determines and outputs the fake face detection result of the target image based on the difference map. This method provides a new means for deep fake detection by analyzing the face difference of the image. However, the technical solution mainly modifies the deep learning network without changing the input content of the network. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide a method for training a deep model for fake face detection.

[0006] Another technical problem to be solved by the present invention is to provide a system for training a fake face detection depth model.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a method for training a fake face detection deep model is provided, comprising the following steps:

[0009] S1: Collect real face images and fake face images, and calculate the fake salient areas through the principle of gradient deep learning back propagation;

[0010] S2: Randomly generate erasure masks based on fake salient areas;

[0011] S3: Input the fake face image and the erasure mask into the enhancement network to obtain an enhanced image;

[0012] S4: input the real face image and the enhanced image into the fake face detection model, and perform adversarial training based on the first loss function until the loss defined by the first loss function no longer decreases, then stop the training and enter step S5;

[0013] S5: Input the fake face image into the image enhancement generator to obtain an adversarial enhanced image;

[0014] S6: Input the adversarial enhanced image into the fake face detection model, perform adversarial training based on the second loss function, and stop training until the loss defined by the second loss function no longer decreases, thereby obtaining a fake face detection deep model.

[0015] Preferably, the formula for calculating the fraud-significant area Map is:

[0016] Map=max(abs(▽ I ○ fake -▽ I ○ real ))

[0017] =max(▽ I (abs(○ fake -○ real )))

[0018] Among them, max(·) is the maximum value along the channel; abs(·) is the absolute value of each pixel; ▽ I ○ fake is the gradient of the probability that the input image is fake relative to the input image; I ○ real To find the gradient of the probability that the input image is real with respect to the input image.

[0019] Preferably, in step S2, the method of randomly generating an erasure mask based on the fraudulent significant area includes the following sub-steps:

[0020] S21: Input face image, image width, image height, fraud-pronounced area, number of erasures, erasure probability; maximum erasure width and maximum erasure height;

[0021] S22: Generate a probability value that conforms to the (0, 1) uniform distribution, and determine whether the probability value is less than the erasure probability; if the probability value is less than the erasure probability, proceed to step S23; if the probability value is greater than or equal to the erasure probability, proceed to step S3;

[0022] S23: Determine whether the count value is less than the erased number; if the count value is less than the erased number, proceed to step S24; if the count value is greater than or equal to the erased number, proceed to step S3;

[0023] S24: Determine whether the mask is blocked; if not, proceed to step S25; if blocked, proceed to step S3;

[0024] S25: randomly generate the upper boundary and the left boundary of the mask, subtract the maximum erasing height from the upper boundary of the mask to obtain the lower boundary of the mask, and subtract the maximum erasing width from the left boundary of the mask to obtain the boundary of the mask; take the mask (i, j) as the center, set the values ​​of all pixels in the upper, lower, left and right boundary areas to 1, and add 1 to the count value, and repeat steps S23 to S25;

[0025] Among them, i is the horizontal coordinate of a pixel in the mask image; j is the vertical coordinate of a pixel in the mask image.

[0026] Preferably, the first loss function is:

[0027]

[0028] in, To optimize the network parameters by maximizing the minimization loss function; I real For real pictures; p train (I real ) is the real image data distribution; D is the image enhancement discriminator; θ D I is the discriminant network; fake is a fake face image; p G (I fake ) is the fake face image data distribution; G is the image enhancement generator; θ G is an image enhancement generator.

[0029] Preferably, the second loss function is:

[0030]

[0031] Among them, y is the true label of the sample; Predict labels for samples for the network.

[0032] Preferably, the enhancement network in step S3 includes two networks, an image enhancement generator and an image enhancement discriminator, which synthesize enhanced images by generating adversarial methods.

[0033] According to a second aspect of an embodiment of the present invention, a system for training a deep model for fake face detection is provided, comprising a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the above method.

[0034] Compared with the prior art, the present invention has the following technical features: First, by collecting real face images and fake face images, and calculating the fake significant area using the principle of gradient deep learning back propagation, the present invention can accurately locate the sensitive area of ​​the face and provide key information for subsequent fake face detection. Secondly, based on the fake significant area, an erasing mask is randomly generated, and it is input into the enhancement network with the fake face image to obtain an enhanced image. This process obtains a more deceptive enhanced image by adversarial enhancement, which effectively improves the robustness of the model. Thirdly, the real face image and the enhanced image are input into the fake face detection model, and adversarial training is performed based on two different loss functions until the loss no longer decreases, thereby obtaining an optimized fake face detection deep model. Finally, the present invention further improves the generalization performance and detection accuracy by improving the generation network architecture, so that it performs better in the fake face detection task and can more accurately distinguish the representative forgeries of fake faces generated by different operation techniques. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A logical framework diagram of a training fake face detection depth model provided by an embodiment of the present invention;

[0036] Figure 2 A flow chart of a method for training a fake face detection depth model provided by an embodiment of the present invention;

[0037] Figure 3 A code logic diagram for randomly generating an erasure mask for a fraudulently significant area in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of an image enhancement generator in an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of a system for training a fake face detection depth model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] First embodiment

[0042] At the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR) held in 2021, Chengrui Wang and Weihong Deng published the paper "Representative Forgery Mining for Fake Face Detection" (full text: https: / / arxiv.org / abs / 2104.06609). The paper proposes an attention-based data augmentation framework to guide the detector to refine and expand its attention range. The specific method involves two steps: first, the gradient of the detector is used to generate an image-level forgery attention map to accurately locate the sensitive areas of the face; second, the Top-N sensitive areas of the face are intentionally occluded through the Suspicious Forgery Erase technique. This method can encourage the detector to dig deeper into previously ignored areas to obtain more representative forgery results. In addition, the method is simple and easy to use and can be easily integrated into various CNN models. Experiments show that the detector trained using this method can distinguish representative forgeries of fake faces generated by different manipulation techniques, and enable CNN-based detectors to achieve state-of-the-art performance without structural modifications.

[0043] like Figure 1 As shown, the embodiment of the present invention has been improved on the basis of the technical solution proposed in the above paper, mainly adding an adversarial enhancement link. Specifically, the technical solution of the embodiment of the present invention includes three key parts: generation of fake salient areas, adversarial enhancement process and conventional training. This improvement aims to improve the robustness of the model through adversarial training, so that it performs better in the fake face detection task.

[0044] like Figure 2 As shown, a method for training a fake face detection depth model provided by the first embodiment of the present invention comprises at least the following steps:

[0045] S1: Collect real face images and fake face images, and calculate the fake salient area through the principle of gradient deep learning back propagation; the formula for calculating the fake salient area Map is:

[0046]

[0047] Among them, max(·) is the maximum value along the channel; abs(·) is the absolute value of each pixel; is the gradient of the probability that the input image is fake with respect to the input image; To find the gradient of the probability that the input image is real with respect to the input image.

[0048] S2: Randomly generate an erasure mask based on the fake salient area.

[0049] S3: Input the fake face image and the erasure mask into the enhancement network to obtain the enhanced image.

[0050] S4: Input the real face image and the enhanced image into the fake face detection model, and perform adversarial training based on the first loss function until the loss defined by the first loss function no longer decreases, then the model stops training and enters step S5.

[0051] S5: Input the fake face image into the image enhancement generator to obtain an adversarial enhanced image.

[0052] S6: Input the adversarial enhanced image into the fake face detection model, perform adversarial training based on the second loss function, and stop training until the loss defined by the second loss function no longer decreases, thereby obtaining a fake face detection deep model.

[0053] Generating an erasure mask is a key step in the adversarial enhancement process. It is mainly divided into two stages. First, the gradient information of the deep learning model is used to determine the areas in the image that have the greatest impact on the discrimination results. These areas are called fake salient areas or fake salient maps, represented by Map. This step is implemented by the above formula (1), which involves calculating the gradient of the deep learning model output to identify the pixels that have the greatest impact on the model's decision. After obtaining the fake salient area Map, the second stage is entered, which is to generate an erasure mask based on the Map. The purpose of this process is to create a mask that will guide the deep network to enhance specific areas of the image without affecting other areas. Figure 3 As shown, the generation process of the erasure mask includes the following steps:

[0054] Initialize a mask matrix of all zeros with the same size as the input image, indicating that no area is selected for erasing in the initial state.

[0055] The random number generator determines whether to perform an erase operation, and this decision is based on a preset erase probability P. If the random number is less than or equal to P, the erase operation continues.

[0056] In the fake salient area map, the area with the greatest impact is selected as the candidate area for erasure. These areas are determined based on the value of the map, and the area with the largest value in the map is usually selected.

[0057] For each selected erase area, the size of the erase area (height H_t and width W_t) is randomly generated, which does not exceed the preset maximum erase size (H_max and W_max).

[0058] According to the generated size and selected areas, the erasing mask is updated and the values ​​of the corresponding areas are set to 1, indicating that these areas will be modified by the deep network in the subsequent enhancement process.

[0059] The above steps are repeated until a predetermined erasure number N is reached or the erasure mask has covered a sufficient area.

[0060] In this way, an erasure mask indicating which areas are erased is generated and used to guide the deep network to enhance specific areas of the image while keeping other areas unchanged.

[0061] In one embodiment of the present invention, the method of randomly generating an erasure mask based on the fraudulent salient area in step S2 includes the following sub-steps:

[0062] S21: Input face image I, image width W, image height H, fake significant area Map, number of erasures N, erasure probability P, maximum erasure width W max and the maximum erasing height H max .

[0063] S22: Generate a probability value that conforms to the (0, 1) uniform distribution, and determine whether the probability value is less than the erasure probability P; if the probability value is less than the erasure probability P, proceed to step S23; if the probability value is greater than or equal to the erasure probability P, proceed to step S3.

[0064] S23: Determine whether the count value is less than the erase number N; if the count value is less than the erase number N, proceed to step S24; if the count value is greater than or equal to the erase number N, proceed to step S3.

[0065] S24: Determine whether the mask is blocked; if not, proceed to step S25; if blocked, proceed to step S3.

[0066] S25: Randomly generate the upper boundary H of the mask t and mask left border W t , the maximum erasing height H max With the mask upper boundary H t Subtract and get the lower boundary H of the mask b , the maximum erase width W max With mask left border W t Subtract, and the surface has a boundary W r, take the mask (i, j) as the center, set the values ​​of all pixels in its upper, lower, left and right boundary areas to 1, add 1 to the count value, and repeat steps S23 to S25; where i is the horizontal coordinate of a pixel in the mask image; j is the vertical coordinate of a pixel in the mask image.

[0067] In one embodiment of the present invention, the enhancement network in step S3 includes two networks: an image enhancement generator and an image enhancement discriminator, which synthesize enhanced images in a generative adversarial manner.

[0068] like Figure 4 As shown in the figure, the input of the image enhancement generator contains two parts, and the final output is only the part replaced by the mask. Therefore, during the redesign of the generation network, masks will be added at different stages of the network to ensure that the learning focus of the network is on the mask part, rather than on other parts of the image. For further explanation of the enhancement network, please refer to the above-mentioned paper "Representative Forgery Mining for Fake Face Detection", which will not be repeated in this invention.

[0069] In one embodiment of the present invention, the first loss function in step S4 is:

[0070]

[0071] in, To optimize the network parameters by maximizing the minimization loss function; I real For real pictures; p train (I real ) is the real image data distribution; D is the image enhancement discriminator; θ D I is the discriminant network; fake is a fake face image; p G (I fake ) is the fake face image data distribution; G is the image enhancement generator; θ G is an image enhancement generator.

[0072] In one embodiment of the present invention, the second loss function in step S6 is:

[0073]

[0074] Among them, y is the true label of the sample; Predict labels for samples for the network.

[0075] In summary, the method for training a fake face detection deep model provided by the first embodiment of the present invention can effectively improve the accuracy and robustness of the fake face detection model by collecting real and fake face images, calculating fake salient areas, randomly generating erasure masks, enhancing network processing, and adversarial training based on two different loss functions. This method particularly enhances the recognition ability of the fake face detection deep model for face images synthesized by deep fake technology, enabling it to achieve higher performance standards in fake face detection tasks, while enhancing the generalization ability for various fake methods.

[0076] Second embodiment

[0077] Based on the above method, the second embodiment of the present invention provides a system for training a fake face detection model. Figure 5 As shown, the system includes one or more processors and a memory, wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the method in the above embodiment.

[0078] Wherein, the processor is used to control the overall operation of the system to complete all or part of the steps of the above method. The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory is used to store various types of data to support operations in the system, and these data may include, for example, instructions for any application or method for operating on the system, and application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, etc.

[0079] In an exemplary embodiment, the system can be implemented by a computer chip or entity, or by a product with a certain function, for executing the above method and achieving the same technical effect as the above method. Specifically, the computer can be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0080] In another exemplary embodiment, the present invention further provides a computer-readable storage medium including program instructions, which, when executed by a processor, implements the steps of the method in any of the above embodiments. For example, the computer-readable storage medium may be the above-mentioned memory including program instructions, which may be executed by a processor to complete the above-mentioned method and achieve the same technical effect as the above-mentioned method.

[0081] It should be noted that the above embodiments are only examples, and the technical solutions of the various embodiments can be combined, all within the protection scope of the present invention.

[0082] The above is a detailed description of the method and system for training a fake face detection depth model provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims

1. A method for training a fake face detection model, characterized in that The steps include: S1: Collect real face images and fake face images, and calculate the fake salient areas through the principle of gradient deep learning back propagation; S2: Randomly generate erasure masks based on fake salient areas; S3: Input the fake face image and the erasure mask into the enhancement network to obtain an enhanced image; S4: input the real face image and the enhanced image into the fake face detection model, and perform adversarial training based on the first loss function until the loss defined by the first loss function no longer decreases, then stop the training and enter step S5; S5: Input the fake face image into the image enhancement generator to obtain an adversarial enhanced image; S6: Input the adversarial enhanced image into the fake face detection model, perform adversarial training based on the second loss function, and stop training until the loss defined by the second loss function no longer decreases, thereby obtaining a fake face detection deep model.

2. The method according to claim 1, characterized in that The formula for calculating the fake significant area Map is: Map=max(abs(▽ I ○ fake -▽ I ○ real )) =max(▽ I (abs(○ fake -○ real ))) Among them, max(·) is the maximum value along the channel; abs(·) is the absolute value of each pixel; ▽ I ○ fake is the gradient of the probability that the input image is fake relative to the input image; I ○ real To find the gradient of the probability that the input image is real with respect to the input image.

3. The method according to claim 1, characterized in that In step S2, the method of randomly generating an erasure mask based on the fraudulent significant area includes the following sub-steps: S21: Input the face image, image width, image height, fake prominent area, number of erasures, erasure probability, maximum erasure width and maximum erasure height; S22: Generate a probability value that conforms to the (0, 1) uniform distribution, and determine whether the probability value is less than the erasure probability; if the probability value is less than the erasure probability, proceed to step S23; if the probability value is greater than or equal to the erasure probability, proceed to step S3; S23: Determine whether the count value is less than the erased quantity; If the count value is less than the erased number, the process proceeds to step S24; if the count value is greater than or equal to the erased number, the process proceeds to step S3; S24: Determine whether the mask is blocked; If it is not blocked, go to step S25; if it is blocked, go to step S3; S25: Randomly generate the upper boundary and the left boundary of the mask, subtract the maximum erasing height from the upper boundary of the mask to obtain the lower boundary of the mask, and subtract the maximum erasing width from the left boundary of the mask to obtain the boundary of the mask; center the mask at (i, j), set the values ​​of all pixels in the upper, lower, left and right boundary areas to 1, add 1 to the count value, and repeat steps S23 to S25; wherein i is the horizontal coordinate of a pixel in the mask image; j is the vertical coordinate of a pixel in the mask image.

4. The method according to claim 1, characterized in that The first loss function is: in, To optimize the network parameters by maximizing the minimization loss function; I real For real pictures; p train (I real ) is the real image data distribution; D is the image enhancement discriminator; θ D I is the discriminant network; fake is a fake face image; p G (I fake ) is the fake face image data distribution; G is the image enhancement generator; θ G is an image enhancement generator.

5. The method according to claim 1, characterized in that The second loss function is: Among them, y is the true label of the sample; Predict labels for samples for the network.

6. The method according to claim 1, characterized in that The enhancement network in step S3 includes two networks, an image enhancement generator and an image enhancement discriminator, which synthesize enhanced images in a generative adversarial manner.

7. A system for training a fake face detection model, characterized in that It comprises a processor and a memory; wherein the memory is coupled to the processor and is used to store a computer program, and when the computer program is executed by the processor, the processor implements the method described in any one of claims 1 to 6.

Images