Method, device, medium and equipment for tracing fake face based on region classification

Abstract

The application discloses a method and device for tracing a fake face based on region classification, a medium and equipment. The method comprises the following steps: acquiring a face picture, wherein the face picture comprises a regionally fake picture, a globally fake picture and a real picture; performing face analysis on the face picture to obtain a face region picture, a background region picture and a connection region picture; performing trace feature extraction on the face picture to obtain a global feature and a global feature decision vector, and performing trace feature extraction on the face region picture, the background region picture and the connection region picture to obtain a face region feature, a background region feature, a connection region feature, a face region feature decision vector, a background region feature decision vector and a connection region feature decision vector; performing feature-level fusion and decision-level fusion on the features and the feature decision vectors to obtain a coarse-grained trace decision vector; and inputting the coarse-grained trace decision vector into a classifier to predict a trace result. The application can improve the accuracy of face tracing.

Description

Technical Field

[0001] This application relates to the field of security technology, and in particular to a method, apparatus, storage medium and electronic device for tracing fake faces based on region classification. Background Technology

[0002] In recent years, with the continuous advancement of facial spoofing technology, anti-spoofing techniques have become increasingly crucial. Anti-spoofing techniques fall into two categories: detection and attribution. Forgery detection judges the authenticity of facial images, a binary classification task. Forgery attribution, on the other hand, identifies the specific forgery model of a forged facial image, a multi-classification task. Compared to forgery detection, forgery attribution provides more accurate forgery-related information for judicial investigations, which is of great significance for improving the credibility of forensic evidence identification.

[0003] With the development of deep learning in the field of computer vision, significant progress has been made in the research of forgery tracing algorithms, yielding remarkable results. For example, forgery tracing is based on the unique "fingerprints" left by Generative Adversarial Networks (GANs). Furthermore, since it has been observed that GAN structures leave globally consistent fingerprints during image generation, while the traces left by model weights differ in different regions, DNA-Det has been proposed to trace forged images back to the architecture of their source models. Additionally, a face manipulation detection method (TFMD) based on feature decomposition using an attention mechanism has been developed for three-class classification.

[0004] The aforementioned forgery tracing algorithms are not very accurate and lack sufficient resistance to complex forgery strategies. Summary of the Invention

[0005] This application provides a method, apparatus, storage medium, and electronic device for tracing fake faces based on region classification, which can improve the accuracy of face tracing.

[0006] This application provides a method for tracing the source of fake faces based on region classification, including:

[0007] Obtain facial images, including region-spoofed images, global-spoofed images, and real images;

[0008] A coarse-grained source tracing network model is constructed, and the face image is input into the trained coarse-grained source tracing network model to predict the source tracing result;

[0009] Specifically, the region-forged image, the global-forged image, and the real image are input into a trained coarse-grained source tracing network model to predict the source tracing result, including:

[0010] The face image is analyzed to obtain a face region image, a background region image, and a connection region image;

[0011] Source feature extraction is performed on the face image to obtain global features and global feature decision vector. Source feature extraction is performed on the face region image, the background region image and the connected region image respectively to obtain face region features, background region features, connected region features, face region feature decision vector, background region feature decision vector and connected region feature decision vector.

[0012] The global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector are fused at the feature level and at the decision level to obtain a coarse-grained source tracing decision vector.

[0013] The coarse-grained source tracing decision vector is input into the classifier to predict the source tracing result.

[0014] Furthermore, in the aforementioned method for tracing fake faces based on region classification, the coarse-grained tracing model includes a face parsing module, which embeds a BiSeNet network.

[0015] The step of performing face analysis on the face image to obtain a face region image, a background region image, and a connection region image includes:

[0016] The face image is input into the BiSeNet network to obtain a face contour mask;

[0017] An erosion operation is performed on the face contour mask to obtain a face region mask, and a face region image is obtained based on the face region mask and the face image;

[0018] The face contour mask is dilated and inverted to obtain a background region mask. A background region image is obtained based on the background region mask and the face image.

[0019] A logical AND operation is performed on the inverted face contour mask and the inverted background region mask to obtain a connection region mask. A connection region image is then obtained based on the connection region mask and the face image.

[0020] Furthermore, in the aforementioned method for tracing fake faces based on region classification, the coarse-grained tracing model includes a region classification fusion module, which includes a feature fusion module and a decision fusion module. The feature fusion module includes a channel attention fusion submodule and a spatial attention fusion submodule.

[0021] The step of performing feature-level fusion and decision-level fusion on the global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector to obtain a coarse-grained source tracing decision vector includes:

[0022] The face region features, the background region features, and the connection region features are input into the channel attention fusion submodule to perform channel attention feature fusion, resulting in three-region channel attention fusion features;

[0023] The three-region channel attention fusion features and the global features are input into the spatial attention fusion submodule to perform spatial attention feature fusion, and the first decision vector of feature-level fusion is obtained.

[0024] The global feature decision vector, the face region feature decision vector, the background region feature decision vector, the connection region feature decision vector, and the first decision vector are input into the decision fusion module for decision fusion to obtain a coarse-grained source tracing decision vector for decision-level fusion.

[0025] Furthermore, in the aforementioned method for tracing the source of fake faces based on region classification, the coarse-grained tracing network model includes a tracing feature extraction module. The tracing feature extraction module is used to extract tracing features from face images. The tracing feature extraction module includes a global tracing feature extractor, a face region tracing feature extractor, a background region tracing feature extractor, and a connection region tracing feature extractor.

[0026] The trained coarse-grained source tracing network model is trained based on a coarse-grained overall loss function, which is:

[0027]

[0028] in, For the legality loss function, It is the cross-entropy loss function of the background region source feature extractor. It is the cross-entropy loss function of the face region source feature extractor. It is the cross-entropy loss function of the region source extraction feature extractor. It is the cross-entropy loss function of the global source feature extractor.

[0029] Furthermore, in the above-mentioned method for tracing fake faces based on region classification, the legitimacy loss function is:

[0030]

[0031] in, , and These represent the probability values ​​of the true labels for the background region source feature extractor, the face region source feature extractor, and the connection region source feature extractor, respectively. , and These represent the probability values ​​of false labels for the background region source feature extractor, the face region source feature extractor, and the connection region source feature extractor, respectively. This represents the probability value of the mixed true and false labels in the region source feature extractor.

[0032] Furthermore, the aforementioned method for tracing fake faces based on region classification, after the step of inputting the coarse-grained tracing decision vector into the classifier to predict the tracing result, includes:

[0033] If the source tracing result is a regional forgery or a global forgery, then a judgment is made on whether the region is forged or the whole map is forged.

[0034] The determination of region and full-map forgery includes:

[0035] A fine-grained source tracing network model is constructed, which includes a coarse-fine grained source tracing feature fusion module, a region forgery fine-grained source tracing module, and a full-image forgery fine-grained source tracing module. The coarse-fine grained source tracing feature fusion module includes a coarse-grained fusion submodule and a coarse-fine weight fusion submodule.

[0036] The global features, the face region features, the background region features, and the connection region features are input into the trained coarse-grained fusion module for feature fusion to obtain coarse-grained fused features.

[0037] The global features are input into the region forgery fine-grained tracing module or the full-image forgery fine-grained tracing module for tracing feature extraction to obtain fine-grained features;

[0038] The coarse-grained fusion feature and the fine-grained feature are input into the coarse-fine weight fusion submodule for weight fusion to obtain the weight fusion feature;

[0039] The weighted fusion features are input into the classifier to obtain specific source tracing results.

[0040] Furthermore, in the aforementioned method for tracing the source of forged faces based on region classification, the fine-grained tracing network model includes a region forgery fine-grained tracing network model and a full-image forgery fine-grained tracing network model, and the fine-grained tracing network model is trained based on a two-stage loss function;

[0041] The first-stage loss function is:

[0042]

[0043] in, For consistency loss, The weighted cross-entropy loss function for a fine-grained source tracing network model for regional forgery. Weighted cross-entropy loss function for a fine-grained source tracing network model for full-graph forgery;

[0044] The two-stage loss function is:

[0045]

[0046] in, To predict the category to which a sample belongs in a coarse-grained source tracing network model. The probability, To predict the category of a sample in a fine-grained source tracing network model for full-image forgery. The probability, For regional forgery fine-grained source tracing network model to predict the category of samples The probability, , , These are the values ​​of the corresponding real labels in their respective real label vectors. This is the two-stage loss function.

[0047] This application also provides a device for tracing fake faces based on region classification, including:

[0048] The acquisition module is used to acquire face images, which include region-forged images, global-forged images, and real images;

[0049] The face analysis module is used to perform face analysis on the face image to obtain a face region image, a background region image, and a connection region image;

[0050] The source feature extraction module is used to extract source features from the face image to obtain global features and global feature decision vectors. It also extracts source features from the face region image, the background region image, and the connected region image to obtain face region features, background region features, connected region features, face region feature decision vectors, background region feature decision vectors, and connected region feature decision vectors.

[0051] The region classification fusion module is used to perform feature-level fusion and decision-level fusion on the global features, the face region features, the background region features, the connected region features, the global feature decision vector, the face region feature decision vector, the background region feature decision vector, and the connected region feature decision vector to obtain a coarse-grained source tracing decision vector;

[0052] The prediction module is used to input the coarse-grained source tracing decision vector into the classifier to predict the source tracing result.

[0053] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute any of the above-described methods for tracing fake faces based on region classification.

[0054] This application also provides an electronic device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used in the steps of the region classification-based fake face tracing method described above.

[0055] The method, apparatus, storage medium, and electronic device for tracing fake faces based on region classification provided in this application extract features from the face image and features from three regions respectively, and fuse the two at the feature level and decision level to obtain a coarse-grained tracing decision vector. The tracing result is predicted by the coarse-grained tracing decision vector. This application makes full use of complementary information at different levels such as region and whole image, decision level and feature level, thereby realizing coarse-grained tracing of fake faces, improving the accuracy of face tracing, and having a strong resistance to complex forgery strategies. Attached Figure Description

[0056] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0057] Figure 1 A flowchart illustrating the method for tracing fake faces based on region classification provided in this application embodiment.

[0058] Figure 2 Another flowchart of the method for tracing fake faces based on region classification provided in this application embodiment.

[0059] Figure 3 A flowchart illustrating the face analysis process provided in this application embodiment.

[0060] Figure 4 This is a flowchart illustrating the generation of channel attention fusion features provided in an embodiment of this application.

[0061] Figure 5A flowchart illustrating the generation of the first decision vector provided in this application embodiment.

[0062] Figure 6 This is a flowchart illustrating the generation of coarse-grained source tracing decision vectors provided in an embodiment of this application.

[0063] Figure 7 This is a flowchart for judging regional forgery and full-map forgery provided in an embodiment of this application.

[0064] Figure 8 A flowchart illustrating the generation of coarse-grained fusion features provided in this application embodiment.

[0065] Figure 9 A flowchart illustrating the generation of fine-grained features provided in this application embodiment.

[0066] Figure 10 This is a schematic diagram of the structure of the fake face tracing device based on region classification provided in the embodiments of this application.

[0067] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0068] Figure 12 Another structural schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] This application provides a method, apparatus, storage medium, and electronic device for tracing fake faces based on region classification. The region classification-based fake face tracing apparatus provided in this application can be integrated into an electronic device, which can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0071] Current forgery tracing algorithms mostly focus on either full-image forgery or region forgery, neglecting the situation where both types of forgery coexist. They fail to consider the differences in the technical principles of these two basic forgery types, making them unsuitable for real-world applications where both types of forgery exist simultaneously. Furthermore, current forgery tracing algorithms suffer from poor accuracy and insufficient resistance to complex forgery strategies.

[0072] To address the aforementioned technical problems, this application provides a method, apparatus, storage medium, and electronic device for tracing the origin of forged faces based on region classification. Please refer to [link / reference]. Figure 1 and Figure 2 , Figure 1 A flowchart illustrating the method for tracing fake faces based on region classification provided in this application embodiment. Figure 2 Another flowchart of the method for tracing fake faces based on region classification provided in this application embodiment, which is applied in an electronic device, includes the following steps:

[0073] S1, Obtain face images, including region-spoofed images, global-spoofed images, and real images.

[0074] Specifically, some of the facial images were obtained from video files. Several frames were randomly sampled from the video files as image data, and all images were standardized to a 224×224 scale and saved in JPG format.

[0075] To minimize the impact of external factors on the algorithm, all images were cropped to only include the face. All data was processed using a random compression factor between 50 and 90. This random compression factor simulates common compression scenarios in real-world network propagation, making the training and test sets more closely resemble real-world environments, thus improving the model's generalization ability and practicality.

[0076] S2. Construct a coarse-grained source tracing network model. Input the face image into the trained coarse-grained source tracing network model to predict the source tracing result.

[0077] Specifically, the coarse-grained source tracing network model includes a face parsing module, a source tracing feature extraction module, and a region classification fusion module. First, the face parsing module obtains three region images: a face region image, a background region image, and a connection region image. Second, the source tracing feature extraction module extracts features from the three region images. Finally, the region classification fusion module fuses the three region features from the global branch and the region branch to obtain a coarse-grained source tracing decision vector, which is input into the classifier to predict the source tracing result.

[0078] Step S2 includes the following steps:

[0079] S21, perform face analysis on the face image to obtain the face region image, background region image, and connection region image.

[0080] In one embodiment, step S2 is implemented through a face analysis module, which embeds a BiSeNet network. Figure 3 A flowchart of the face analysis process provided in the embodiments of this application is shown below. Figure 3 As shown, step S21 includes the following steps:

[0081] Face analysis of a face image yields a face region image, a background region image, and a connectivity region image, including:

[0082] S211, input the face image into the BiSeNet (Bilateral Segmentation Network) to obtain the face contour mask.

[0083] Specifically, it can be expressed by the following formula:

[0084]

[0085] in, It's a face image. It is a facial contour mask.

[0086] S212, perform an erosion operation on the face contour mask to reduce the boundary and obtain a face region mask. Ensure that the face region mask does not include the edge part. Obtain a face region image based on the face region mask and the face image.

[0087] Specifically, it can be expressed by the following formula:

[0088]

[0089]

[0090] in, It is a corrosion operation. Masking for the face region. This is a pixel-by-pixel multiplication operation. Image of a face region.

[0091] S213, perform a dilation operation on the face contour mask and invert it to obtain the background region mask, and obtain the background region image based on the background region mask and the face image.

[0092] Specifically, it can be expressed by the following formula:

[0093]

[0094]

[0095] in, It is an expansion operation. It is a negation operation. Mask the background area. This is the background image.

[0096] S214: Perform a logical AND operation on the inverted face contour mask and the inverted background region mask to obtain a connected region mask. Based on the connected region mask and the face image, obtain a connected region image.

[0097] Specifically, it can be expressed by the following formula:

[0098]

[0099]

[0100] in, It is logic and operations. For connecting region mask, This is a connected region image.

[0101] S22, perform source feature extraction on the face image to obtain global features and global feature decision vectors. Perform source feature extraction on the face region image, background region image, and connected region image respectively to obtain face region features, background region features, connected region features, face region feature decision vectors, background region feature decision vectors, and connected region feature decision vectors.

[0102] Specifically, step S22 is implemented through the source feature extraction module, which includes 12 convolutional layers, 5 pooling layers, and 3 fully connected layers. Each convolutional block contains a convolutional layer, batch normalization, and RULU activation function. After passing through the conv12 layers of the network, a 14×14×512-dimensional feature map (including face region features, background region features, and connection region features) is obtained. This feature map is used as the intermediate feature of the source feature extraction module. After passing through the fully connected layer fc2, a 1×1×512-dimensional vector (including face region feature decision vector, background region feature decision vector, and connection region feature decision vector) is output. The structure of the source feature extraction module is shown in Table 1 below.

[0103] Table 1. Structure of the Source Tracing Feature Extraction Module

[0104] number of floors Layer name Operation type Input dimensions Output size Number of channels 1-2 conv1conv2 Conv+BN+ReLU 224×224 224×224 64 3 pool1 MaxPool 224×224 112×112 64 4-5 conv3conv4 Conv+BN+ReLU 112×112 112×112 128 6 pool2 MaxPool 112×112 56×56 128 7-8 conv5conv6 Conv+BN+ReLU 56×56 56×56 256 9 pool3 MaxPool 56×56 28×28 256 10-11 conv7conv8 Conv+BN+ReLU 28×28 28×28 512 12 pool4 MaxPool 28×28 14×14 512 13-14 conv9conv10 Conv+BN+ReLU 14×14 14×14 512 15-16 conv11conv12 Conv+BN+ReLU 14×14 14×14 512 17 pool5 MaxPool 14×14 7×7 512 18 fc1 FC+ReLU 7×7×512 4096 N / A 19 fc2 FC 4096 512 N / A 20 fc3 FC+Softmax 512 C N / A

[0105] The coarse-grained source tracing network model is a two-branch network model. The global branch includes a source tracing feature extraction module, which extracts source features from the face image using a global source tracing feature extractor to obtain global features, and outputs a global feature decision vector through a fully connected layer. The region branch includes a source tracing feature extraction module, which extracts source features from the face region, background region, and connection region features using face region source tracing feature extractors, background region source tracing feature extractors, and connection region source tracing feature extractors to obtain face region features, background region features, and connection region features, and outputs face region feature decision vectors, background region feature decision vectors, and connection region feature decision vectors.

[0106] S23, perform feature-level fusion and decision-level fusion on global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector to obtain coarse-grained source tracing decision vector;

[0107] Specifically, step S23 is implemented through a region classification fusion module, which includes a feature fusion module and a decision fusion module. The feature fusion module includes a channel attention fusion submodule and a spatial attention fusion submodule. In one embodiment, step S23 includes the following steps:

[0108] S231, input the face region features, background region features, and connection region features into the channel attention fusion submodule to perform channel attention feature fusion, and obtain the three-region channel attention fusion features.

[0109] Figure 4 The flowchart for generating channel attention fusion features provided in the embodiments of this application can be found in [reference]. Figure 4 Step S231 can be specifically represented by the following formula:

[0110]

[0111] in, Background region features, For facial region features, Connectivity features For channel attention fusion operations, This represents the attention fusion feature of the three-region channel.

[0112] S232, input the three-region channel attention fusion features and global features into the spatial attention fusion submodule to perform spatial attention feature fusion, and obtain the first decision vector of feature-level fusion.

[0113] Figure 5The flowchart for generating the first decision vector provided in the embodiments of this application can be found in [reference]. Figure 5 Step S232 can be specifically represented by the following formula:

[0114]

[0115] in, As a global feature, For spatial attention feature fusion operations, This is the first decision vector for feature fusion.

[0116] S233, the global feature decision vector, face region feature decision vector, background region feature decision vector, connection region feature decision vector and the first decision vector are input into the decision fusion module for decision fusion to obtain the coarse-grained source tracing decision vector of decision-level fusion.

[0117] Figure 6 The flowchart for generating coarse-grained source tracing decision vectors provided in the embodiments of this application can be found in [reference needed]. Figure 6 Step 233 can be specifically represented by the following formula:

[0118]

[0119] in, This is the global feature decision vector. For background region feature decision vectors, , is the decision vector for facial region features. For connecting the feature decision vectors of the region, The decision fusion network is used by the decision fusion module. This is the coarse-grained source tracing decision vector.

[0120] S24, input the coarse-grained source tracing decision vector into the classifier to predict the source tracing result.

[0121] Specifically, the classifier includes multiple fully connected layers, which determine whether a face image is a real image, a region forgery, or a full-image forgery.

[0122] The above describes the application process of the coarse-grained source tracing network model. The following describes the training process of the coarse-grained source tracing network model:

[0123] (1) Obtain a face image dataset, including region fake images, global fake images and real images, and label them with real labels;

[0124] (2) Input the face image dataset into the coarse-grained source tracing network model to obtain the predicted source tracing results;

[0125] (3) Iterative training is performed based on the source tracing results, the real labels and the coarse-grained overall loss function to obtain the trained coarse-grained source tracing network model.

[0126] In one embodiment, the coarse-grained global loss function is:

[0127] in, For the legality loss function, It is the cross-entropy loss function of the background region source feature extractor. It is the cross-entropy loss function of the face region source feature extractor. It is the cross-entropy loss function of the region source extraction feature extractor. It is the cross-entropy loss function of the global source feature extractor.

[0128] In one embodiment, the legality loss function is:

[0129]

[0130] in, , and These represent the probability values ​​of the true labels for the background region source feature extractor, the face region source feature extractor, and the connection region source feature extractor, respectively. , and These represent the probability values ​​of false labels for the background region source feature extractor, the face region source feature extractor, and the connection region source feature extractor, respectively. This represents the probability value of the mixed true and false labels in the region source feature extractor.

[0131] In addition to the validity losses of the three regional source feature extractors, each of these source feature extractors also requires an independent cross-entropy loss. Meanwhile, the global source feature extractor in the global branch does not interact with the three regional source feature extractors; it only needs its own cross-entropy loss function. These four cross-entropy loss functions are as follows:

[0132] The loss function for the background region source feature extractor is:

[0133]

[0134] The loss function of the face region source feature extractor is:

[0135]

[0136] The loss function of the connected region source feature extractor is:

[0137]

[0138] The loss function of the global source feature extractor is:

[0139]

[0140] in, , , , The background region source feature extractor predicts the category to which the sample belongs. The probability of a face region source feature extractor predicting the category to which a sample belongs. The probability of a sample belonging to a category is predicted by the connection region source feature extractor. The probability of a sample belonging to a category predicted by the global source feature extractor. The probability, For true labels, if the sample belongs to the category ,but =1, otherwise 0.

[0141] In one embodiment, after step S2, the following step is further included:

[0142] S3. If the source tracing result is regional forgery or global forgery, then perform a judgment on regional forgery and global forgery.

[0143] Figure 7 The flowchart for determining region forgery and full-map forgery provided in this application embodiment is as follows: Figure 7 As shown, the fine-grained source tracing network model is also a parallel network model, containing both region-forged and full-image-forged fine-grained source tracing network models. It should be noted that when the region-forged fine-grained source tracing network model is identified as genuine, the full-image-forged fine-grained source tracing network model should also be identified as genuine, and vice versa. Similarly, the region-forged fine-grained source tracing network model also includes a source tracing feature extraction module and a classifier, as does the full-image-forged fine-grained source tracing network model.

[0144] Specifically, For the input face image, This is the classification function for a coarse-grained source tracing network, which classifies images as Real, RegionalFake, or EntireFake. According to... The results determine whether to proceed with fine-grained source tracing. or .if Then the classification ends. If Then it enters the fine-grained classification of region forgery. If Then proceed to fine-grained classification of full-image forgery. Define the global source tracing function. for:

[0145]

[0146] The determination of regional and full-map forgery includes:

[0147] S31. Construct a fine-grained source tracing network model. The fine-grained source tracing network model includes a coarse-fine grained source tracing feature fusion module, a region forgery fine-grained source tracing module, and a full-image forgery fine-grained source tracing module. The coarse-fine grained source tracing feature fusion module includes a coarse-grained fusion sub-module and a coarse-fine weight fusion sub-module.

[0148] S32, input the global features, face region features, background region features and connection region features into the trained coarse-grained fusion module for feature fusion to obtain coarse-grained fused features.

[0149] Figure 8 A flowchart for generating coarse-grained fusion features provided in the embodiments of this application is shown below. Figure 8 As shown, step S32 can be expressed by the following formula:

[0150]

[0151] in, This indicates a coarse-grained fusion submodule. This indicates coarse-grained fusion characteristics.

[0152] S33. Input the global features into the region forgery fine-grained tracing module or the full-image forgery fine-grained tracing module to extract tracing features and obtain fine-grained features.

[0153] S34. Input the coarse-grained fusion features and fine-grained features into the coarse-fine weight fusion submodule to perform weight fusion and obtain the weighted fusion features.

[0154] Figure 9 A flowchart for generating fine-grained features provided in the embodiments of this application is shown below. Figure 9 As shown, step S34 can be expressed by the following formula:

[0155]

[0156] in, Fine-grained features generated for either the regional forgery fine-grained tracing module or the full-map forgery fine-grained tracing module. This is a submodule for merging coarse and fine weights. These are weighted fusion features.

[0157] S35, input the weighted fusion features into the classifier to obtain the specific source tracing results.

[0158] The above describes the application process of the fine-grained source attribution network model. The fake face source attribution network model includes both coarse-grained and fine-grained models. The training process of the fake face source attribution network model is described below:

[0159] (1) Obtain a face image dataset, including region fake images, global fake images and real images, and label them with real labels;

[0160] (2) Input the face image dataset into the fake face tracing network model to obtain the predicted tracing results;

[0161] (3) Iterative training is performed based on the source tracing results, real labels and overall loss function to obtain a trained fake face source tracing network model.

[0162] In one embodiment, the overall loss function comprises a two-stage loss function.

[0163] The first-stage loss function is:

[0164]

[0165] in, For consistency loss, The weighted cross-entropy loss function for a fine-grained source tracing network model for regional forgery. The weighted cross-entropy loss function for a fine-grained source tracing network model for full-graph forgery.

[0166] The consistency loss is:

[0167]

[0168] in, For indicator functions, it means when Belongs to set Returns 1 if the condition is met, otherwise returns 0. and These are the softmax output vectors of the full-image forgery attribution network model and the region forgery attribution network model, respectively, which contain the predicted probabilities for each category. , and These represent the set of full-image forgery models, the set of regional forgery models, and the set of true results, respectively. It represents the probability that the regional forgery tracing network prediction model is a full-map forgery. It represents the probability that the full-map forgery tracing network prediction model identifies a region as forged. and It represents the probability that the region-based fine-grained source tracing network model and the full-graph-based fine-grained source tracing network model predict the true result.

[0169] Weighted cross-entropy loss emphasizes the importance of fine-grained source tracing network models for both genuine and coarse-grained forgery types. By assigning higher weights to these two categories, it improves the accuracy of backpropagation. The weighted cross-entropy of the two fine-grained source tracing networks are:

[0170]

[0171] in, It is the weighted cross-entropy loss function of the region forgery fine-grained source tracing network model. It is the number of output categories of the network. It is a category In the fine-grained source tracing network model for regional forgery, the weights for full-image forgery and authenticity are set to 2, and the weights for the other categories are 1. The regional forgery fine-grained source tracing network model predicts that the sample belongs to the category. The probability of. It is the corresponding category in the real label vector. The value if the sample belongs to the category ,but Otherwise, it is 0. It is the weighted cross-entropy loss function of the fine-grained source tracing network for full-graph forgery. It is the weighted cross-entropy loss function of the fake fine-grained source tracing network model. It is the number of output categories of the network. It is a category The weight, The full-image forgery fine-grained source tracing network model predicts the category to which the sample belongs. The probability, It is the corresponding category in the real label vector. The value of .

[0172] The two-stage loss function is:

[0173]

[0174] in, To predict the category to which a sample belongs in a coarse-grained source tracing network model. The probability, To predict the category of a sample in a fine-grained source tracing network model for full-image forgery. The probability, For regional forgery fine-grained source tracing network model to predict the category of samples The probability, , , Each of these represents its corresponding real label. This is the two-stage loss function.

[0175] The region-based forgery tracing method provided in this application extracts facial image features and features from three regions respectively, and then fuses these features at the feature level and decision level to obtain a coarse-grained tracing decision vector. This coarse-grained tracing decision vector is used to predict the tracing result. This application fully utilizes complementary information at different levels, such as region and whole image, decision level and feature level, thereby achieving coarse-grained forgery tracing and improving the accuracy of face tracing. Furthermore, this application also utilizes the distribution differences of forgery traces of two forgery types in image regions and considers the guiding role of the two basic forgery type classifications on the classification of specific forgery models. It adopts a combined coarse-fine tracing strategy to adapt to practical application scenarios where two forgery types coexist, enabling accurate fine-grained forgery tracing.

[0176] Based on the method described in the above embodiments, this embodiment will further describe it from the perspective of a fake face tracing device based on region classification. The fake face tracing device based on region classification can be implemented as an independent entity or integrated into an electronic device. The electronic device can be a terminal, server, or other devices. The terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessor box, or other devices.

[0177] Please see Figure 10 , Figure 10 This application provides a detailed description of a region-based face spoofing tracing device, which is applied in electronic devices. The region-based face spoofing tracing device may include:

[0178] The acquisition module is used to acquire face images, which include region-spoofed images, global-spoofed images, and real images.

[0179] The face analysis module is used to analyze face images to obtain face region images, background region images, and connection region images;

[0180] The source feature extraction module is used to extract source features from face images to obtain global features and global feature decision vectors. It extracts source features from face region images, background region images, and connected region images respectively to obtain face region features, background region features, connected region features, face region feature decision vectors, background region feature decision vectors, and connected region feature decision vectors.

[0181] The region classification and fusion module is used to perform feature-level fusion and decision-level fusion on global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector to obtain a coarse-grained source tracing decision vector;

[0182] The prediction module is used to input the coarse-grained source tracing decision vector into the classifier to predict the source tracing result.

[0183] In specific implementation, the above modules and / or units can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of the above modules and / or units, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0184] In addition, embodiments of this application also provide an electronic device, which may be a computer, tablet computer, or other similar device. Figure 11 As shown, the electronic device 400 includes a processor 401 and a memory 402. The processor 401 and the memory 402 are electrically connected.

[0185] The processor 401 is the control center of the electronic device 400. It connects various parts of the electronic device through various interfaces and lines. By running or loading the application program stored in the memory 402 and calling the data stored in the memory 402, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole.

[0186] In this embodiment, the processor 401 in the electronic device 400 loads the instructions corresponding to the processes of one or more application programs into the memory 402 according to the following steps, and the processor 401 runs the application programs stored in the memory 402 to realize various functions:

[0187] Obtain facial images, including region-spoofed images, global-spoofed images, and real images;

[0188] A coarse-grained source tracing network model is constructed, and the face image is input into the trained coarse-grained source tracing network model to predict the source tracing result;

[0189] Specifically, the region-forged image, the global-forged image, and the real image are input into a trained coarse-grained source tracing network model to predict the source tracing result, including:

[0190] The face image is analyzed to obtain a face region image, a background region image, and a connection region image;

[0191] Source feature extraction is performed on the face image to obtain global features and global feature decision vector. Source feature extraction is performed on the face region image, the background region image and the connected region image respectively to obtain face region features, background region features, connected region features, face region feature decision vector, background region feature decision vector and connected region feature decision vector.

[0192] The global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector are fused at the feature level and at the decision level to obtain a coarse-grained source tracing decision vector.

[0193] The coarse-grained source tracing decision vector is input into the classifier to predict the source tracing result.

[0194] This electronic device can implement the steps in any embodiment of the fake face tracing method based on region classification provided in the embodiments of this application. Therefore, it can achieve the beneficial effects that any fake face tracing method based on region classification provided in the embodiments of this invention can achieve. For details, please refer to the previous embodiments, which will not be repeated here.

[0195] Figure 12 A specific structural block diagram of an electronic device provided in an embodiment of the present invention is shown. This electronic device can be used to implement the region classification-based method for tracing fake faces provided in the above embodiments. The electronic device 500 can be a terminal, server, or other device. The terminal can include a tablet computer, laptop computer, personal computer (PC), microprocessor box, or other devices.

[0196] RF circuit 510 is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals and vice versa, thereby enabling communication with communication networks or other devices. RF circuit 510 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity modules (SIM cards), memory, etc. RF circuit 510 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). The aforementioned wireless networks may use various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, including those that have not yet been developed.

[0197] The memory 520 can be used to store software programs and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, such as taking pictures with the front-facing camera, processing the captured images, and switching the display colors of the content displayed on the screen. The memory 520 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 520 may further include memory remotely located relative to the processor 580, and these remote memories can be connected to the electronic device 500 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0198] The input unit 530 can be used to receive input numeric or character information, and to generate a keyboard and mouse related to user settings and function control.

[0199] Display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, which can be composed of graphics, text, icons, video, and any combination thereof. Display unit 540 may include display panel 541, which may optionally be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms.

[0200] Audio circuitry 560, speaker 561, and microphone 562 provide an audio interface between the user and electronic device 500. Audio circuitry 560 converts received audio data into electrical signals and transmits them to speaker 561, where speaker 561 converts them into sound signals for output. Conversely, microphone 562 converts collected sound signals into electrical signals, which are then received by audio circuitry 560, converted back into audio data, and processed by processor 580. The audio data is then transmitted via RF circuitry 510 to, for example, another terminal, or output to memory 520 for further processing. Audio circuitry 560 may also include an earphone jack to facilitate communication between external headphones and electronic device 500.

[0201] Electronic device 500, through transmission module 570 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 570 is shown in the figure, it is understood that it is not an essential component of electronic device 500 and can be omitted as needed without changing the essence of the invention.

[0202] The processor 580 is the control center of the electronic device 500. It connects to various parts of the phone via various interfaces and lines, and performs various functions and processes data of the electronic device 500 by running or executing software programs and / or modules stored in the memory 520, and by calling data stored in the memory 520, thereby providing overall monitoring of the electronic device. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 580.

[0203] Electronic device 500 also includes a power supply 590 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to processor 580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 590 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0204] Although not shown, the electronic device 500 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory and one or more programs, wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations:

[0205] Obtain facial images, including region-spoofed images, global-spoofed images, and real images;

[0206] A coarse-grained source tracing network model is constructed, and the face image is input into the trained coarse-grained source tracing network model to predict the source tracing result;

[0207] Specifically, the region-forged image, the global-forged image, and the real image are input into a trained coarse-grained source tracing network model to predict the source tracing result, including:

[0208] The face image is analyzed to obtain a face region image, a background region image, and a connection region image;

[0209] Source feature extraction is performed on the face image to obtain global features and global feature decision vector. Source feature extraction is performed on the face region image, the background region image and the connected region image respectively to obtain face region features, background region features, connected region features, face region feature decision vector, background region feature decision vector and connected region feature decision vector.

[0210] The global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector are fused at the feature level and at the decision level to obtain a coarse-grained source tracing decision vector.

[0211] The coarse-grained source tracing decision vector is input into the classifier to predict the source tracing result.

[0212] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0213] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any embodiment of the region-based face spoofing tracing method provided by the present invention.

[0214] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0215] Since the instructions stored in the storage medium can execute the steps in any embodiment of the forged face tracing method based on region classification provided in the embodiments of the present invention, the beneficial effects that any forged face tracing method based on region classification provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0216] The foregoing has provided a detailed description of a method, apparatus, storage medium, and electronic device for tracing fake faces based on region classification, as provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for tracing the source of forged faces based on region classification, characterized in that, The method includes: Obtain facial images, including region-spoofed images, global-spoofed images, and real images; A coarse-grained source tracing network model is constructed. The face image is input into the trained coarse-grained source tracing network model to predict the source tracing result. The coarse-grained source tracing model includes a region classification fusion module, which includes a feature fusion module and a decision fusion module. The feature fusion module includes a channel attention fusion submodule and a spatial attention fusion submodule. Specifically, the region-forged image, the global-forged image, and the real image are input into a trained coarse-grained source tracing network model to predict the source tracing result, including: The face image is analyzed to obtain a face region image, a background region image, and a connection region image; Source feature extraction is performed on the face image to obtain global features and global feature decision vector. Source feature extraction is performed on the face region image, the background region image and the connected region image respectively to obtain face region features, background region features, connected region features, face region feature decision vector, background region feature decision vector and connected region feature decision vector. The global features, face region features, background region features, connected region features, global feature decision vector, face region feature decision vector, background region feature decision vector, and connected region feature decision vector are fused at the feature level and at the decision level to obtain a coarse-grained source tracing decision vector, including: The face region features, background region features, and connectivity region features are input into the channel attention fusion submodule for channel attention feature fusion to obtain three-region channel attention fusion features; the three-region channel attention fusion features and the global features are input into the spatial attention fusion submodule for spatial attention feature fusion to obtain a first decision vector for feature-level fusion; the global feature decision vector, the face region feature decision vector, the background region feature decision vector, the connectivity region feature decision vector, and the first decision vector are input into the decision fusion module for decision fusion to obtain a coarse-grained source tracing decision vector for decision-level fusion. The coarse-grained source tracing decision vector is input into the classifier to predict the source tracing result.

2. The method for tracing fake faces based on region classification according to claim 1, characterized in that, The coarse-grained source tracing model includes a face parsing module, which embeds a BiSeNet network. The step of performing face analysis on the face image to obtain a face region image, a background region image, and a connection region image includes: The face image is input into the BiSeNet network to obtain a face contour mask; An erosion operation is performed on the face contour mask to obtain a face region mask, and a face region image is obtained based on the face region mask and the face image; The face contour mask is dilated and inverted to obtain a background region mask. A background region image is obtained based on the background region mask and the face image. A logical AND operation is performed on the inverted face contour mask and the inverted background region mask to obtain a connection region mask. A connection region image is then obtained based on the connection region mask and the face image.

3. The method for tracing fake faces based on region classification according to claim 1, characterized in that, The coarse-grained source tracing network model includes a source tracing feature extraction module, which is used to extract source tracing features from face images. The source tracing feature extraction module includes a global source tracing feature extractor, a face region source tracing feature extractor, a background region source tracing feature extractor, and a connection region source tracing feature extractor. The trained coarse-grained source tracing network model is trained based on a coarse-grained overall loss function, which is: in, For the legality loss function, It is the cross-entropy loss function of the background region source feature extractor. It is the cross-entropy loss function of the face region source feature extractor. It is the cross-entropy loss function of the region source extraction feature extractor. It is the cross-entropy loss function of the global source feature extractor.

4. The method for tracing fake faces based on region classification according to claim 3, characterized in that, The legality loss function is: in, , and These represent the probability values ​​of the true labels for the background region source feature extractor, the face region source feature extractor, and the connection region source feature extractor, respectively. , and These represent the probability values ​​of false labels for the background region source feature extractor, the face region source feature extractor, and the connection region source feature extractor, respectively. This represents the probability value of the mixed true and false labels in the region source feature extractor.

5. The method for tracing the source of forged faces based on region classification according to claim 1, characterized in that, After the step of inputting the coarse-grained source tracing decision vector into the classifier to predict the source tracing result, the following is included: If the source tracing result is a regional forgery or a global forgery, then a judgment is made on regional forgery and global forgery. The determination of region and full-map forgery includes: A fine-grained source tracing network model is constructed, which includes a coarse-fine grained source tracing feature fusion module, a region forgery fine-grained source tracing module, and a full-image forgery fine-grained source tracing module. The coarse-fine grained source tracing feature fusion module includes a coarse-grained fusion submodule and a coarse-fine weight fusion submodule. The global features, the face region features, the background region features, and the connection region features are input into the trained coarse-grained fusion module for feature fusion to obtain coarse-grained fused features. The global features are input into the region forgery fine-grained tracing module or the full-image forgery fine-grained tracing module for tracing feature extraction to obtain fine-grained features; The coarse-grained fusion feature and the fine-grained feature are input into the coarse-fine weight fusion submodule for weight fusion to obtain the weight fusion feature; The weighted fusion features are input into the classifier to obtain specific source tracing results.

6. The method for tracing fake faces based on region classification according to claim 5, characterized in that, The fine-grained source tracing network model includes a region forgery fine-grained source tracing network model and a full-graph forgery fine-grained source tracing network model, and the fine-grained source tracing network model is trained based on a two-stage loss function; The first-stage loss function is: in, For consistency loss, The weighted cross-entropy loss function for a fine-grained source tracing network model for regional forgery. Weighted cross-entropy loss function for a fine-grained source tracing network model for full-graph forgery; The two-stage loss function is: in, To predict the category to which a sample belongs in a coarse-grained source tracing network model. The probability, To predict the category of a sample in a fine-grained source tracing network model for full-image forgery. The probability, For regional forgery fine-grained source tracing network model to predict the category of samples The probability, , , These are the values ​​of the corresponding real labels in their respective real label vectors. This is the two-stage loss function.

7. A device for tracing fake faces based on region classification, wherein the device is used to implement the method for tracing fake faces based on region classification as described in claim 1, characterized in that, include: The acquisition module is used to acquire face images, which include region-forged images, global-forged images, and real images; The face analysis module is used to perform face analysis on the face image to obtain a face region image, a background region image, and a connection region image; The source feature extraction module is used to extract source features from the face image to obtain global features and global feature decision vectors. It also extracts source features from the face region image, the background region image, and the connected region image to obtain face region features, background region features, connected region features, face region feature decision vectors, background region feature decision vectors, and connected region feature decision vectors. The region classification fusion module is used to perform feature-level fusion and decision-level fusion on the global features, the face region features, the background region features, the connected region features, the global feature decision vector, the face region feature decision vector, the background region feature decision vector, and the connected region feature decision vector to obtain a coarse-grained source tracing decision vector; The prediction module is used to input the coarse-grained source tracing decision vector into the classifier to predict the source tracing result.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the region classification-based method for tracing fake faces as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, The method includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to execute the steps in the method for tracing fake faces based on region classification as described in any one of claims 1 to 6.

Images