A method, device, equipment and storage medium for detecting forged faces

Abstract

The present application discloses a method, device, equipment and storage medium for detecting face forgery, the method comprising: constructing a backbone network of a face image to be detected, determining a plurality of backbone network layers according to the backbone network, grouping the plurality of backbone network layers to obtain a plurality of network layer groups, inputting the plurality of network layer groups into a priori attention mechanism to obtain a plurality of priori attention results, and obtaining a detection result of the face image to be detected according to the plurality of priori attention results. In this way, by using a priori attention mechanism to process the network layer group and enhancing the modeling of local forgery traces, the underlying information can be captured to avoid the loss of details of the underlying information, and the accuracy of the face forgery detection results can be improved. At the same time, the detection of the face image to be detected can be realized through the obtained priori attention results, which can further guide the area that the detection model needs to pay attention to, provide forgery clues, and improve the efficiency of face forgery detection.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, equipment and storage medium for detecting forged faces. Background Art

[0002] With the development of science and technology, face forgery detection technology can identify forged faces from images or videos to reduce or avoid the negative impact of forged face images or videos.

[0003] In the prior art, the "RGB+image decomposition information" detection method is generally used to detect forged facial images. However, with the development of science and technology, facial images or videos forged by various forgery methods have become increasingly realistic, and can achieve a fake-realistic effect. The existing "RGB+image decomposition information" detection method is difficult to directly find forgery clues from the perspective of RGB images. At the same time, since the frequency domain information and noise information obtained by decomposition contain less shape, color and other content than the original image, the underlying information of such images cannot be captured during detection, which may cause the details of such information to be lost, resulting in low accuracy of the detection results of face forgery and low detection efficiency.

[0004] Based on this, how to solve the problem of low accuracy and low detection efficiency of face forgery detection results is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] Based on the above problems, the present application provides a face forgery detection method, device, equipment and storage medium to improve the accuracy of face forgery detection results and improve detection efficiency.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for detecting face forgery, the method comprising:

[0008] Construct a backbone network for the face image to be detected;

[0009] Determine a plurality of backbone network layers according to the backbone network;

[0010] Grouping the multiple backbone network layers to obtain multiple network layer groups;

[0011] Inputting the plurality of network layer groups into a priori attention mechanism to obtain a plurality of priori attention results;

[0012] The detection result of the face image to be detected is obtained according to the multiple prior attention results.

[0013] Optionally, the grouping of the multiple backbone network layers to obtain multiple network layer groups includes:

[0014] When the number of the backbone network layers is N, two adjacent backbone network layers are taken as a network layer group to obtain N-1 network layer groups.

[0015] Optionally, determining a plurality of backbone network layers according to the backbone network includes:

[0016] A first backbone network layer, a second backbone network layer, a third backbone network layer and a fourth backbone network layer are determined according to the backbone network.

[0017] Optionally, the grouping of the multiple backbone network layers to obtain multiple network layer groups includes:

[0018] The first backbone network layer and the second backbone network layer are used as a first network layer group;

[0019] The second backbone network layer and the third backbone network layer are used as a second network layer group;

[0020] The third backbone network layer and the fourth backbone network layer are used as a third network layer group.

[0021] Optionally, after grouping the multiple backbone network layers to obtain multiple network layer groups, the method further includes:

[0022] Processing each backbone network layer in each network layer group to obtain multiple target network layer groups;

[0023] Correspondingly, the plurality of network layer groups are input into the prior attention mechanism to obtain a plurality of prior attention results, including:

[0024] The multiple target network layer groups are input into the prior attention mechanism to obtain multiple prior attention results.

[0025] Optionally, after grouping the multiple backbone network layers to obtain multiple network layer groups, the method further includes:

[0026] Determine the target feature sequence of each network layer group to obtain multiple target feature sequences;

[0027] Correspondingly, the multiple network layer groups are input into the prior attention mechanism to obtain multiple prior attention results, including:

[0028] The multiple target feature sequences are respectively input into the prior attention mechanism to obtain multiple prior attention results.

[0029] Optionally, the inputting the multiple target feature sequences into the prior attention mechanism respectively to obtain multiple prior attention results includes:

[0030] Perform layer normalization on each target feature sequence;

[0031] Determine a noise information map and an edge information map corresponding to the face image to be detected;

[0032] Determine a first information map according to the face image to be detected and the noise information map;

[0033] Determine a second information map according to the face image to be detected and the edge information map;

[0034] A target attention map is determined according to each processed target feature sequence, the first information map and the second information map as a priori attention result.

[0035] In a second aspect, an embodiment of the present application provides a face forgery detection device, the device comprising: an acquisition module, a backbone network layer determination module, a network layer group determination module, a priori attention result determination module and a detection module;

[0036] The acquisition module is used to construct a backbone network of the face image to be detected;

[0037] The backbone network layer determination module is used to determine multiple backbone network layers according to the backbone network;

[0038] The network layer group determination module is used to group the multiple backbone network layers to obtain multiple network layer groups;

[0039] The prior attention result determination module is used to input the multiple network layer groups into the prior attention mechanism to obtain multiple prior attention results;

[0040] The detection module is used to obtain the detection result of the face image to be detected based on the multiple prior attention results.

[0041] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the face forgery detection method as described in the first aspect is implemented.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a terminal device, the terminal device executes the face forgery detection method as described in the first aspect.

[0043] Compared with the prior art, this application has the following beneficial effects:

[0044] The present application constructs a backbone network of a face image to be detected, determines multiple backbone network layers based on the backbone network, groups the multiple backbone network layers to obtain multiple network layer groups, inputs the multiple network layer groups into a priori attention mechanism to obtain multiple priori attention results, and obtains the detection result of the face image to be detected based on the multiple priori attention results. Among them, by using the priori attention mechanism to process the network layer group and enhance the modeling of local forgery traces, the underlying information can be captured to avoid the loss of details of the underlying information, and the accuracy of the face forgery detection results can be improved. At the same time, the detection of the face image to be detected is realized through the obtained priori attention results, which can further guide the area that the detection model needs to pay attention to, provide forgery clues, and improve the efficiency of face forgery detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0046] Figure 1 A flowchart of a face forgery method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of a face forgery detection process provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of the structure of a face counterfeiting device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] As described above, in the research on face forgery detection, it was found that in the prior art, the "RGB+image decomposition information" detection method is generally used to detect forged human images. However, with the development of science and technology, face images or videos forged by various forgery means have gradually become more realistic, and can achieve an effect of being indistinguishable from the real thing. The existing "RGB+image decomposition information" detection method is difficult to directly find forgery clues from the perspective of RGB images. At the same time, since the frequency domain information and noise information obtained by decomposition contain less shape, color and other content than the original image, the underlying information of such images cannot be captured during detection, which may cause the details of such information to be lost, resulting in low accuracy of face forgery detection results and low detection efficiency.

[0050] In order to solve the above problems, the embodiments of the present application provide a method, device, equipment and storage medium for detecting face forgery. The method includes: constructing a backbone network of a face image to be detected, determining multiple backbone network layers according to the backbone network, grouping the multiple backbone network layers to obtain multiple network layer groups, inputting the multiple network layer groups into a priori attention mechanism to obtain multiple priori attention results, and obtaining the detection result of the face image to be detected according to the multiple priori attention results.

[0051] In this way, by adopting the prior attention mechanism to process the network layer group and enhancing the modeling of local forgery traces, the underlying information can be captured, the loss of details of the underlying information can be avoided, and the accuracy of the face forgery detection results can be improved. At the same time, the detection of the face image to be detected can be realized through the obtained prior attention results, which can further guide the area that the detection model needs to pay attention to, provide forgery clues, and improve the efficiency of face forgery detection.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] See also Figure 1 , which is a flow chart of a face forging method provided in an embodiment of the present application, combined with Figure 1 As shown, the face forging method provided in the embodiment of the present application may include:

[0054] S101: Construct a backbone network for the face image to be detected.

[0055] Among them, construction means selecting a backbone network model, such as ResNet34 or FcaNet34, and then building each layer of the model structure for the selected backbone network model.

[0056] The face image to be detected refers to an image containing relevant features of a person's face, and the image is generally a received image.

[0057] The backbone network refers to the basic convolutional neural network. The backbone network used in this application can be ResNet34 and FcaNet34.

[0058] It should be noted that by constructing a backbone network of the face image to be detected, deeper features can be further extracted based on the backbone network, thereby further finding clues of forgery.

[0059] S102: Determine a plurality of backbone network layers according to the backbone network.

[0060] It should be noted that the backbone network includes multiple backbone network layers. Each layer of the backbone network is each layer of the convolutional neural network. Whether each layer is the same depends on the specific structure of the backbone network used.

[0061] S103: Group the multiple backbone network layers to obtain multiple network layer groups.

[0062] A network layer group refers to a set of backbone network layers including a preset number. In the present application, the preset number may be 2, that is, one network layer group includes two backbone network layers.

[0063] In an achievable implementation, step S103 may specifically include:

[0064] When the number of the backbone network layers is N, two adjacent backbone network layers are taken as a network layer group to obtain N-1 network layer groups.

[0065] It should be noted that N is the number of backbone network layers, and N-1 is the number of backbone network layer groups.

[0066] Further, in another achievable implementation, when the number of the backbone network layers is 4, the step S103 specifically includes:

[0067] Determine a first backbone network layer, a second backbone network layer, a third backbone network layer and a fourth backbone network layer according to the backbone network;

[0068] The first backbone network layer and the second backbone network layer are used as a first network layer group;

[0069] The second backbone network layer and the third backbone network layer are used as a second network layer group;

[0070] The third backbone network layer and the fourth backbone network layer are used as a third network layer group.

[0071] It should be noted that in order to better distinguish different backbone network layers, "first" and "second" are used to distinguish them. However, whether two backbone network layers are the same depends on the specific structure of the backbone network used. The serial number is only used to distinguish multiple backbone network layers.

[0072] It should be noted that, in this embodiment, the number of backbone networks included in the backbone network layer group is 2. Correspondingly, in this embodiment, two adjacent backbone networks are taken as a group and a network layer group is generated.

[0073] S104: Input the multiple network layer groups into the prior attention mechanism to obtain multiple prior attention results.

[0074] It should be noted that the prior attention mechanism means analyzing each network layer group according to the prior attention map to determine the part of the image to be verified first, so that clues of face forgery can be quickly found.

[0075] It should be noted that the prior attention mechanism refers to a special structure embedded in the machine learning model. Its essence is a mechanism that autonomously learns a set of weight coefficients through the network and emphasizes the areas of interest while suppressing irrelevant background areas in a "dynamically weighted" manner. The "key-query-value" theory is generally used to describe the mechanism of the prior attention mechanism.

[0076] S105: Obtaining a detection result of the face image to be detected according to the multiple prior attention results.

[0077] The face forgery detection method provided in the embodiment of the present application is to construct a backbone network of the face image to be detected, determine multiple backbone network layers based on the backbone network, group the multiple backbone network layers to obtain multiple network layer groups, input the multiple network layer groups into the prior attention mechanism to obtain multiple prior attention results, and obtain the detection result of the face image to be detected based on the multiple prior attention results. Among them, by using the prior attention mechanism to process the network layer group and enhance the modeling of local forgery traces, the underlying information can be captured to avoid the loss of details of the underlying information, and the accuracy of the face forgery detection results can be improved. At the same time, the detection of the face image to be detected is realized through the obtained prior attention results, which can further guide the area that the detection model needs to pay attention to, provide forgery clues, and improve the efficiency of face forgery detection.

[0078] Based on the face forgery detection method provided in the above embodiment, in order to further improve the face forgery detection efficiency, after step S103, the method may further include:

[0079] Processing each backbone network layer in each network layer group to obtain multiple target network layer groups;

[0080] Correspondingly, the plurality of network layer groups are input into the prior attention mechanism to obtain a plurality of prior attention results, including:

[0081] The multiple target network layer groups are input into the prior attention mechanism to obtain multiple prior attention results.

[0082] To give a specific example, in an implementable example, assuming that the number of backbone network layers is 4, the corresponding number of network layer groups is 3, and taking the first network layer group composed of the first backbone network layer and the second backbone network layer as an example, the features corresponding to the first backbone network layer are subjected to maximum pooling processing and linear layer processing to obtain class labels, and the features corresponding to the second backbone network layer are subjected to convolution processing and maximum normalization processing to obtain an image block sequence.

[0083] It should be noted that the accuracy of the prior attention results can be further improved by processing each backbone network layer in each network layer group before inputting the prior attention mechanism.

[0084] Based on the face forgery detection method provided in the above embodiment, after S103, the method may further include:

[0085] Determine the target feature sequence of each network layer group to obtain multiple target feature sequences;

[0086] Correspondingly, the multiple network layer groups are input into the prior attention mechanism to obtain multiple prior attention results, including:

[0087] The multiple target feature sequences are respectively input into the prior attention mechanism to obtain multiple prior attention results.

[0088] It should be noted that, based on the class labels and image block sequences obtained in the above examples, a feature sequence corresponding to the first network layer group is further obtained according to the class labels and image block sequences as a target feature sequence.

[0089] Based on the target feature sequences obtained in the above embodiments, in an achievable implementation, the multiple target feature sequences are respectively input into the prior attention mechanism to obtain multiple prior attention results, which may include:

[0090] Step 11: Perform layer normalization on each target feature sequence;

[0091] Step 12: Determine a noise information map and an edge information map corresponding to the face image to be detected;

[0092] Step 13: determining a first information map according to the face image to be detected and the noise information map;

[0093] Step 14: determining a second information map according to the face image to be detected and the edge information map;

[0094] Step 15: Determine a target attention map based on each processed target feature sequence, the first information map and the second information map as a priori attention result.

[0095] The noise information map means the image noise information obtained by using the filter.

[0096] The edge information map refers to the image edge information obtained using an edge detector.

[0097] It should be noted that by explicitly modeling the attention map using the noise information map and edge information map corresponding to the face image to be detected obtained in advance, which is used to guide the model which part of the face image to be detected is more worthy of attention, providing an interpretability of the forged area, that is, through this example, the attention map response of the forged area can be made stronger. Since ordinary attention maps converge to a concentrated area in the face area, they rarely show obvious differences for images of different categories, but by explicitly modeling the attention map, more differences between real images and fake images can be determined; in addition, the loss of detail information caused by the pooling operation of the convolutional neural network is avoided.

[0098] It should be noted that in this embodiment, by adopting layer normalization processing, data standardization is added to ensure the stability of data feature distribution, so that a larger learning rate can be used to accelerate the convergence speed of the model.

[0099] Based on the face forgery detection method provided in the above embodiment, the present application embodiment further provides a face forgery detection process. Figure 2 , which is a schematic diagram of a face forgery detection process provided by an embodiment of the present application, and the result Figure 2 As shown, the face forgery detection process provided in the embodiment of the present application may include:

[0100] Step 1: Extract the backbone network of the target image to be detected and obtain the target backbone network.

[0101] It should be noted that the backbone network refers to the basic convolutional neural network. The backbone network used in this embodiment can be ResNet34 and FcaNet34, where the ResNet34 refers to a 34-layer CNN (convolutional neural network), and the FcaNet34 refers to a 34-layer convolutional neural network based on the channel attention mechanism.

[0102] Step 2: determine the first backbone network layer according to the target backbone network; obtain the second backbone network layer based on the first backbone network layer; and obtain the third backbone network layer and the fourth backbone network layer in sequence according to the method for determining the second backbone network layer.

[0103] It should be noted that whether the various backbone network layers are the same depends on the network structure of the backbone network, which is not specifically limited or explained here.

[0104] Step three: perform maximum pooling and linear layer processing on the first backbone network layer to obtain a class label; perform convolution and normalization processing on the second backbone network layer to obtain an image block sequence; obtain a first target feature sequence based on the class label and the image block sequence; determine the second target feature sequence and the third target feature sequence respectively according to the determination process of the first target feature sequence, wherein the second target feature sequence is obtained by the second backbone network layer and the third backbone network layer, and the third target feature sequence is obtained by the third backbone network layer and the third backbone network layer.

[0105] It should be noted that the process of performing maximum pooling and linear layer processing on the first backbone network layer to obtain class labels is the process of extracting representative features of the previous layer, generating a 1*N dimensional feature vector, and the class label is a learnable embedding vector Class Token (vector 0) proposed in the ViT model. The process of obtaining image blocks by the second layer of convolution + normalization is the process of extracting representative features of the current layer, generating an M*N dimensional feature vector, where M represents the number of image blocks and N represents the dimension of each image block.

[0106] It should be noted that the ViT model (Visual Transformer) refers to a machine vision paradigm that divides an image into a series of position-embedded patches, which are processed by the Transformer Encoder to understand the local and global features present in the image. ViT has higher accuracy on a continuous large dataset and reduces training time. Among them, Transformer is a feature extractor based on the attention mechanism, which can be used to replace CNN and RNN to extract sequence features.

[0107] Among them, CNN means Convolutional Neural Networks (CNN), which is a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning.

[0108] RNN refers to a recurrent neural network (RNN), which is a type of recursive neural network that takes sequence data as input, performs recursion in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain.

[0109] In an achievable implementation, in step three, the target feature sequence can be obtained by:

[0110] Step 21: Obtain image decomposition information.

[0111] The SRM filter is used to obtain the noise information map, the Canny edge detector is used to obtain the edge information map, and the noise information map and the edge information map are used as image decomposition information.

[0112] Step 22: The feature map extracted from the backbone network layer through CNN is represented as f l ∈R H×W×C (H, W, C represent height, width, and channel respectively), divide the feature map into N non-overlapping image blocks (patch tokens),

[0113] Among them, P 2 Represents the resolution of each patch token, Represents the total number of reorganized patch tokens, each patch token is a length of (P 2 C) is the picture encoding row vector.

[0114] Step 23: Insert a specially designed class token (X) at the beginning of the patch tokens sequence cls ), and obtain the target feature sequence S = [X CLS ,X1,X2,...,XN]∈R (N+1)×D , where D = P 2 ·C.

[0115] It should be noted that the above examples are applicable to the first target feature sequence, the second target feature sequence, the third target feature sequence, and other target feature sequences, and are not specifically limited here.

[0116] Step 4: Perform layer normalization on the first target feature sequence, and input the processed first target sequence into the prior attention mechanism to obtain the prior attention result.

[0117] It should be noted that layer normalization of class labels and image block sequences is one of the basic steps of the ViT model, among which layer normalization (LayerNorm) is one of the most commonly used data normalization (normalization) methods. When using the gradient descent method for optimization, as the network depth increases, the feature distribution of the input data will continue to change. In order to ensure the stability of the data feature distribution, data normalization will be added. As a result, a larger learning rate can be used to accelerate the convergence of the model. At the same time, data standardization also has a certain anti-overfitting effect, making the training process smoother. Specifically, the main function of data standardization is to normalize the features of each layer before inputting them into the activation function, so that they are converted into data with a mean of 1 and a variance of 0, thereby avoiding the data falling in the saturation area of ​​the activation function to reduce the problem of gradient disappearance.

[0118] It should be noted that the first target feature sequence is essentially a class label and an image block sequence, and correspondingly, layer normalization processing is actually performed on the class label and the image block sequence.

[0119] It should be noted that the process of obtaining the prior attention result based on the first target feature sequence can be specifically implemented in the following ways:

[0120] Step 31: According to the attention mechanism, map the target feature sequence S to query: Q∈R (N+1)×(D / h) ,key:K∈R (N+1)×(D / h) ,value:V∈R (N+1)×(D / h) .

[0121] Among them, h represents the number of layers of the network structure, query, key and value are three intrinsic terms in the attention mechanism, representing three different feature sequences in the attention mechanism. The similarity between the query and each key is calculated to obtain the weight, including the class token. The obtained attention map is a (N+1)×(N+1) matrix, expressed as Among them, class token is a learnable embedding vector proposed in the ViT algorithm, and its main function is to assist in classification prediction.

[0122] Step 32: In order to maintain the autocorrelation between features, the prior attention template needs to be added to the classtoken. To ensure the uniformity of the dimension, the prior image decomposition information P needs to be downsampled and flattened to obtain AttTemp = Norm(Flatten(Downsample(P))).

[0123] Among them, Downsample(*) means downsampling operation; Flatten(*) means flattening operation; Norm(*) means normalization operation.

[0124] Among them, the prior attention template means a template that superimposes external prior information and adds different weight ratios to the features of different areas in the image.

[0125] Step 33: Update the prior attention map. We use AttMap0 to represent the attention map of the class token. Add the prior information to the attention map and update it to obtain AttMap0=AttTemp+Softmax(AttMap0); where Softmax(*) means the classification operation.

[0126] Step 34: After superimposing the prior attention map, the final attention map is obtained as the prior attention result by multiplying it with the value PriAtt(S)=Softmax(AttMap)V.

[0127] It should be noted that the process of obtaining the prior attention results from the second target feature sequence and the third target feature sequence is the same as the process of obtaining the prior attention results from the first target feature sequence mentioned above, so it will not be repeated here.

[0128] Step 5: Perform layer normalization and forward propagation network processing on the three attention maps obtained according to the first target feature sequence, the second target feature sequence and the third target feature sequence, respectively, to obtain three processed attention maps.

[0129] Among them, the layer normalization processing of the attention map is also one of the basic steps of the ViT model. Through layer normalization processing, the data can be prevented from falling into the saturation area of ​​the activation function to reduce the problem of gradient disappearance.

[0130] Among them, the forward propagation network is a series of fully connected layers + activation functions. Each neuron calculates the value z based on the weight w and bias b. After being stimulated by the activation function, an output y is generated, and this y is the input of the next layer. Then the next layer generates an output as the input of the next layer, and so on until the last layer. The output of each layer is used as the input of the next layer, and the transmission layer by layer is called forward propagation.

[0131] Step 6: Determine the detection result of the target image to be detected based on the three processed attention maps.

[0132] It should be noted that the definition of true or false for the target image to be detected is determined by the probability value of the output result. By setting a preset threshold x, if the output result y>x, the image is judged to be false; y≤x, the image is judged to be true.

[0133] It should be noted that, in this embodiment, steps three, four and five can be encapsulated into a feature abstraction module, there can be one feature abstraction module, or a corresponding number of feature abstraction modules can be set according to the number of target feature sequences, and there is no specific limit on the number.

[0134] It should be noted that, on the basis of encapsulating the feature abstraction module, the prior attention mechanism in step 4 can be encapsulated into a prior attention mechanism module, which can be a submodule of the feature abstraction module in terms of logical relationship.

[0135] In an achievable implementation, the implementation process of face forgery detection based on the feature abstraction module of the prior attention mechanism can be:

[0136] Step 41: Referring to the traditional ViT (Visual Transformer) model, considering that too many channels in the convolutional neural network may cause information redundancy, 1*1 convolution and normalization operations are used to compress the channel dimension as the inner-layer feature abstraction, expressed as PatchTokens = Reshape (IN (Conv 1×1 (f l ))).

[0137] Among them, Conv 1×1 Represents the convolution operation; IN (Instance Normalization) represents the normalization operation; Reshape is a reshaping function, which is used to ensure the uniformity of the front and back dimensions.

[0138] Step 42: In order to better realize feature interaction, different from the traditional ViT model, in this embodiment, the previous layer feature is abstracted into a one-dimensional embedding as an inter-layer feature abstraction, which is represented as the class token of the current layer, represented as ClsToken = Linear (Flatten (MaxPool (f l-1 ))).

[0139] Among them, MaxPool(*) represents the pooling operation; Flatten(*) represents the flattening operation; Linear(*) represents the linear transformation operation.

[0140] It should be noted that the traditional ViT model uses random numbers to represent Class Tokens, which do not have any characteristic meaning. Therefore, in this embodiment, the representative features of the previous layer are extracted as the Class Token of the current layer, which contains more underlying information of the image, so that the image information is retained more completely.

[0141] Step 43: Thus, the target feature sequence S = Concat(ClsToken, PatchTokens) is obtained, and the target feature sequence is input into the prior attention mechanism module to obtain the output prior attention result z′ = PriAtt(LN(S)) + S. The prior attention result is further combined with the ViT model algorithm to obtain the final output z = FFT(LN(z′)) + z′.

[0142] Among them, LN (Layer Normalization) means normalization operation; FFN means feedforward neural network.

[0143] It should be noted that z is the final output of each feature abstraction module. After the multiple output z are reorganized and passed through the classifier, the probability value of judging the authenticity of the target face image can be obtained.

[0144] It should be noted that the prior attention result refers to the attention map on the one hand, and the output result corresponding to the attention map on the other hand. The two are just different ways of expression, but both represent the prior attention results.

[0145] It should be noted that the output z value is further compared with the preset threshold x. If the output result y>x, the image is judged to be false; if y≤x, the image is judged to be true.

[0146] Based on the face forging method provided in the above embodiment, the present application also provides a face forging device, see Figure 3 , which is a schematic diagram of the structure of a face forgery device provided in an embodiment of the present application, combined with Figure 3 As shown, the apparatus 300 may include: an acquisition module 301, a backbone network layer determination module 302, a network layer group determination module 303, a priori attention result determination module 304 and a detection module 305;

[0147] The acquisition module 301 is used to construct a backbone network of a face image to be detected;

[0148] The backbone network layer determination module 302 is used to determine multiple backbone network layers according to the backbone network;

[0149] The network layer group determination module 303 is used to group the multiple backbone network layers to obtain multiple network layer groups;

[0150] The prior attention result determination module 304 is used to input the multiple network layer groups into the prior attention mechanism to obtain multiple prior attention results;

[0151] The detection module 305 is used to obtain the detection result of the face image to be detected based on the multiple prior attention results.

[0152] As an example, the network layer group determination module 303 is specifically used to:

[0153] When the number of the backbone network layers is N, two adjacent backbone network layers are taken as a network layer group to obtain N-1 network layer groups.

[0154] As an example, the backbone network layer determination module 302 includes:

[0155] A first backbone network layer, a second backbone network layer, a third backbone network layer and a fourth backbone network layer are determined according to the backbone network.

[0156] As an example, the network layer group determination module 303 is specifically used to:

[0157] The first backbone network layer and the second backbone network layer are used as a first network layer group;

[0158] The second backbone network layer and the third backbone network layer are used as a second network layer group;

[0159] The third backbone network layer and the fourth backbone network layer are used as a third network layer group.

[0160] As an example, after the network layer group determination module 303, the device further includes:

[0161] A processing module, used for processing each backbone network layer in each network layer group to obtain multiple target network layer groups;

[0162] Correspondingly, the prior attention result determination module 304 is specifically used for:

[0163] The multiple target network layer groups are input into the prior attention mechanism to obtain multiple prior attention results.

[0164] As an example, after the network layer group determination module 303, the device further includes:

[0165] A target feature sequence determination module is used to determine the target feature sequence of each network layer group to obtain multiple target feature sequences;

[0166] Correspondingly, the prior attention result determination module 304 includes:

[0167] The prior attention result determination submodule is used to input the multiple target feature sequences into the prior attention mechanism respectively to obtain multiple prior attention results.

[0168] As an example, the prior attention result determination submodule is specifically used for:

[0169] Perform layer normalization on each target feature sequence;

[0170] Determine a noise information map and an edge information map corresponding to the face image to be detected;

[0171] Determine a first information map according to the face image to be detected and the noise information map;

[0172] Determine a second information map according to the face image to be detected and the edge information map;

[0173] A target attention map is determined according to each processed target feature sequence, the first information map and the second information map as a priori attention result.

[0174] The face forging device provided in the embodiment of the present application has the same beneficial effects as the face forging method provided in the above embodiment, so it will not be described in detail.

[0175] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the solutions provided by the embodiments of the present application.

[0176] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the face forgery detection method described in any embodiment of the present application.

[0177] The computer storage medium stores codes, and when the codes are executed, the device executing the codes implements the face forgery detection method described in any embodiment of the present application.

[0178] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and equipment embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and equipment embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0179] The "first" and "second" in the names such as "first" and "second" (if any) mentioned in the embodiments of the present application are only used as name identifiers and do not represent the first or second in order.

[0180] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0181] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting forged faces, characterized in that: The method comprises: Construct a backbone network for the face image to be detected; Determine N backbone network layers according to the backbone network; N is an integer greater than or equal to 2; Take two adjacent backbone network layers as one network layer group, and obtain N-1 network layer groups; The features corresponding to the first backbone network in each network layer group are processed by maximum pooling and linear layer to obtain class labels, and the features corresponding to the second backbone network in each network layer group are processed by convolution and maximum normalization to obtain image sequences; According to the class labels and image sequences corresponding to each network layer group, the corresponding feature sequences are determined as the target feature sequences; Perform layer normalization on each target feature sequence; Determine a noise information map and an edge information map corresponding to the face image to be detected; Determine a first information map according to the face image to be detected and the noise information map; Determine a second information map according to the face image to be detected and the edge information map; Determine a target attention map according to each processed target feature sequence, the first information map, and the second information map as a priori attention result; A detection result of the face image to be detected is obtained according to multiple prior attention results.

2. The face forgery detection method according to claim 1, characterized in that: Determining a plurality of backbone network layers according to the backbone network includes: A first backbone network layer, a second backbone network layer, a third backbone network layer and a fourth backbone network layer are determined according to the backbone network.

3. The face forgery detection method according to claim 2, characterized in that: The multiple backbone network layers are grouped to obtain multiple network layer groups, including: The first backbone network layer and the second backbone network layer are used as a first network layer group; The second backbone network layer and the third backbone network layer are used as a second network layer group; The third backbone network layer and the fourth backbone network layer are used as a third network layer group.

4. A face forgery detection device, characterized in that: The device comprises: an acquisition module, a backbone network layer determination module, a network layer group determination module, a priori attention result determination module and a detection module; The acquisition module is used to construct a backbone network of the face image to be detected; The backbone network layer determination module is used to determine N backbone network layers according to the backbone network; N is an integer greater than or equal to 2; The network layer group determination module is used to take two adjacent backbone network layers as one network layer group to obtain N-1 network layer groups; A target feature sequence determination module is used to perform maximum pooling and linear layer processing on the features corresponding to the first backbone network in each network layer group to obtain a class label, and to perform convolution processing and maximum normalization processing on the features corresponding to the second backbone network in each network layer group to obtain an image sequence; and to determine the corresponding feature sequence of each network layer group according to the class label and image sequence, as the target feature sequence; The prior attention result determination module is used to perform layer normalization processing on each target feature sequence; determine the noise information map and edge information map corresponding to the face image to be detected; determine the first information map according to the face image to be detected and the noise information map; determine the second information map according to the face image to be detected and the edge information map; determine the target attention map according to each processed target feature sequence, the first information map and the second information map as the prior attention result; The detection module is used to obtain the detection result of the face image to be detected based on multiple prior attention results.

5. A computer device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting face forgery according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the face forgery detection method according to any one of claims 1 to 3.

Images