Face forgery detection model construction method, face forgery detection method and device

Abstract

The present invention provides a method for constructing a face forgery detection model, a face forgery detection method and a device, wherein the method for constructing a face forgery detection model comprises: determining a first forgery detection model, the first forgery detection model is used to extract multiple candidate features of a first sample face image, and based on the multiple candidate features and their weights, performing forgery detection on the first sample face image; based on the weights of the multiple candidate features, determining a target feature from the multiple candidate features; constructing a second forgery detection model for forgery detection based on the target feature; based on the second sample face image and its authenticity label, training the second forgery detection model to obtain the face forgery detection model, which not only ensures the excellent performance of the model, but also reduces the model specifications, overcomes the defects of the traditional solution that it is time-consuming and labor-intensive to distinguish forged information, and cannot know the role of the collaborative analysis of different candidate features, and improves the subsequent face forgery detection process based on the face forgery detection model.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for constructing a face forgery detection model, a face forgery detection method and a device. Background Art

[0002] In recent years, face forgery detection has gradually become a hot research topic in the field of computer vision, especially in vision-based pattern recognition applications, where face forgery detection occupies a pivotal position. It aims to distinguish tampered face images from various images and videos in order to curb the negative impact of forged face images or videos.

[0003] At present, the forged images obtained through various forgery methods are already very realistic, and can even be mistaken for the real thing. This makes it difficult for researchers to find forgery clues directly from the RGB image perspective. In this case, researchers shift their attention from the original RGB image to other levels to extract forgery clues from this level. However, the variety of forgery methods has caused researchers to spend a lot of time and energy to determine whether each candidate feature contains forgery clues, and it is difficult to find out whether the collaborative analysis of different candidate features has a promoting effect on the face forgery detection task. All these have led to the stagnation of face forgery detection technology. Summary of the invention

[0004] The present invention provides a method for constructing a face forgery detection model, a face forgery detection method and a device, so as to solve the defects in the prior art that it is time-consuming and laborious to distinguish forged information and the role of collaborative analysis of different candidate features cannot be known.

[0005] The present invention provides a method for constructing a face forgery detection model, comprising:

[0006] Determining a first forgery detection model, where the first forgery detection model is used to extract multiple candidate features of the first sample face image, and perform forgery detection on the first sample face image based on the multiple candidate features and their weights;

[0007] Determining a target feature from the multiple candidate features based on the weights of the multiple candidate features;

[0008] constructing a second forgery detection model for forgery detection based on the target feature;

[0009] Based on the second sample face image and its authenticity label, the second forgery detection model is trained to obtain a face forgery detection model.

[0010] According to a method for constructing a face forgery detection model provided by the present invention, the multiple candidate features are determined based on the following steps:

[0011] Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain multiple candidate features of the first sample face image;

[0012] Based on the encoding layer in the first forgery detection model, encoding the candidate features to obtain a candidate feature map of the first sample face image;

[0013] Based on the decoding layer in the first forgery detection model, the candidate feature map is decoded to obtain a candidate feature vector of the first sample face image, wherein the candidate features, the candidate feature map and the candidate feature vector are features of different feature levels;

[0014] Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features at the same feature level are fused to obtain candidate forgery detection features.

[0015] According to a method for constructing a face forgery detection model provided by the present invention, the method comprises: based on the fusion layer in the first forgery detection model and the candidate elements contained in the features of each feature level, fusing the features of the same feature level to obtain the candidate forgery detection features, including:

[0016] Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features of the same feature level are fused to obtain first fused features of different feature levels, wherein the first fused features include first candidate fused features, first candidate fused feature maps, and first candidate fused feature vectors;

[0017] Based on the fusion layer in the first forgery detection model, the candidate elements included in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels, wherein the second fused features include a second candidate fused feature map and a second candidate fused feature vector;

[0018] Based on the fusion layer in the first forgery detection model and the weight corresponding to the second candidate fused feature vector, the second candidate fused feature vector is fused to obtain a candidate forgery detection feature.

[0019] According to a method for constructing a face forgery detection model provided by the present invention, the method is based on the fusion layer in the first forgery detection model and the candidate elements contained in the features of each feature level, fusing the features of the same feature level to obtain the first fused features of different feature levels, including:

[0020] If the candidate elements included in the multiple features at any feature level are different, the multiple features at any feature level are fused based on the fusion layer in the first forgery detection model to obtain a first fused feature at any feature level.

[0021] According to a method for constructing a face forgery detection model provided by the present invention, based on the fusion layer in the first forgery detection model, the candidate elements contained in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels, including:

[0022] If the candidate elements contained in multiple first fused features of any feature level are the same, the multiple first fused features of any feature level are fused based on the fusion layer in the first forgery detection model and the weights corresponding to the multiple first fused features of any feature level to obtain the second fused feature of any feature level.

[0023] According to a method for constructing a face forgery detection model provided by the present invention, the feature extraction layer in the first forgery detection model is used to extract features from the first sample face image to obtain multiple candidate features of the first sample face image, including:

[0024] Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain initial candidate features of the first sample face image;

[0025] Based on the feature extraction layer in the first forgery detection model, initial candidate features having spatial correspondence are fused to obtain initial candidate fused features;

[0026] Based on the initial candidate fusion features and the initial candidate features of the first sample face image, multiple candidate features of the first sample face image are determined.

[0027] The present invention also provides a method for detecting forged faces, comprising:

[0028] Determine a face image to be detected;

[0029] Inputting the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model;

[0030] The face forgery detection model is determined based on any of the above-mentioned face forgery detection model construction methods.

[0031] The present invention also provides a device for constructing a face forgery detection model, comprising:

[0032] a first forgery detection model determining unit, configured to determine a first forgery detection model, wherein the first forgery detection model is configured to extract a plurality of candidate features of a first sample face image, and perform forgery detection on the first sample face image based on the plurality of candidate features and their weights;

[0033] a target feature determination unit, configured to determine a target feature from the plurality of candidate features based on weights of the plurality of candidate features;

[0034] a second forgery detection model building unit, configured to build a second forgery detection model for forgery detection based on the target feature;

[0035] The face forgery detection model determination unit is used to train the second forgery detection model based on the second sample face image and its authenticity label to obtain the face forgery detection model.

[0036] The present invention also provides a face forgery detection device, comprising:

[0037] A face image determination unit, used to determine a face image to be detected;

[0038] A face forgery detection unit is used to input the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method as described in any one of the above items.

[0039] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for constructing a face forgery detection model as described above or the method for detecting face forgery as described above is implemented.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for constructing a face forgery detection model as described above or the method for detecting face forgery as described above is implemented.

[0041] The face forgery detection model construction method, face forgery detection method and device provided by the present invention use the weights of multiple candidate features obtained by a first forgery detection model to screen out target features from multiple candidate features, and construct a second forgery detection model based on the target feature for forgery detection, and train the second forgery detection model to obtain the face forgery detection model, which not only ensures the excellent performance of the model, but also reduces the model specifications, reduces resource usage, and realizes the lightweight of the model, overcomes the defects of traditional solutions that it is time-consuming and labor-intensive to distinguish forged information and it is impossible to know the role of collaborative analysis of different candidate features, and improves the subsequent face forgery detection process based on the face forgery detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flow chart of the method for constructing a face forgery detection model provided by the present invention;

[0044] Figure 2 is a structural schematic diagram of a first forgery detection model provided by the present invention;

[0045] Figure 3 It is a flow chart of the face forgery detection method provided by the present invention;

[0046] Figure 4 It is a structural schematic diagram of a face forgery detection model building device provided by the present invention;

[0047] Figure 5 It is a structural schematic diagram of the face forgery detection device provided by the present invention;

[0048] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] The present invention provides a method for constructing a face forgery detection model, which aims to use a network structure search technology to determine multiple candidate features containing forgery information from a first sample face image, and determine the optimal combination mode, that is, the target feature, according to the weight, so as to construct a face forgery detection model capable of forgery detection based on the target feature, thereby reducing the model specifications, reducing resource occupation, achieving lightweight of the model, and optimizing performance, and providing assistance for the subsequent face forgery detection process based on the face forgery detection model. Figure 1 FIG. 1 is a flow chart of a method for constructing a face forgery detection model provided by the present invention. Figure 1 As shown, the method includes:

[0051] Step 110, determining a first forgery detection model, the first forgery detection model is used to extract multiple candidate features of the first sample face image, and perform forgery detection on the first sample face image based on the multiple candidate features and their weights;

[0052] Specifically, before constructing a face forgery detection model, it is first necessary to determine an initial face forgery detection model, i.e., a first forgery detection model, through which a plurality of candidate features of a first sample face image can be determined, i.e., on the basis of the first forgery detection model, the first sample face image can be deconstructed and combined by using computer graphics, frequency domain decomposition and other methods, so as to obtain a plurality of candidate features of the first sample face image, and each candidate feature contains one or more candidate elements, where the candidate elements can be understood as forgery information carried in the corresponding candidate feature;

[0053] Subsequently, the first forgery detection model may perform forgery detection on the first sample face image based on the candidate features and the weights corresponding to the candidate features to determine the authenticity of the first sample face image, thereby obtaining a first forgery detection result corresponding to the first sample face image;

[0054] Afterwards, the first forgery detection model can be trained based on the first forgery detection result and the authenticity label of the first sample face image. This process is essentially to adjust the weights of multiple candidate features of the first sample face image in the first forgery detection model, so that the first forgery detection result obtained by the adjusted first forgery detection model for forgery detection can be consistent with the authenticity label of the first sample face image. When the model output is consistent with the sample label, the training is terminated to obtain the trained first forgery detection model.

[0055] Step 120, determining a target feature from the multiple candidate features based on the weights of the multiple candidate features;

[0056] Specifically, in step 110, after the first forgery detection model is trained, step 120 may be executed to determine the target feature from the multiple candidate features according to the weights of the multiple candidate features of the first sample face image. The specific process includes the following steps:

[0057] Since the training process of the first forgery detection model is actually a process of making the model output continuously approach the sample label, the parameters (weights) of the trained first forgery detection model can reflect the contribution of the corresponding candidate features to forgery detection, that is, the larger the weight, the more forgery clues (candidate elements) contained in the corresponding candidate features, and / or the higher the recognition of the combination of candidate elements, and the greater the role played in the forgery detection task; conversely, the smaller the weight, the fewer forgery clues (candidate elements) contained in the corresponding candidate features, and / or the lower the recognition of the combination of candidate elements, and the smaller the contribution made to the forgery detection task.

[0058] In view of this, in an embodiment of the present invention, the weights of multiple candidate features of the first sample facial image can be determined through the trained first forgery detection model, and based on the weights, the target feature can be selected from the multiple candidate features. The target feature here is the candidate feature corresponding to the first preset number of weights when the weights of the multiple candidate features are arranged in order from large to small. That is, the weights of the multiple candidate features can be sorted in order from large to small, and the first preset number of weights can be selected from them, and the candidate features corresponding to them are used as target features.

[0059] It is worth noting that the preset number here can be set according to actual needs, for example, it can be 1, 2, 3, etc., and preferably, in the embodiment of the present invention, the preset number is determined to be 1, that is, the maximum weight is determined from the weights of multiple candidate features, and the candidate feature corresponding to the maximum weight is determined, and this candidate feature is used as the target feature.

[0060] In the embodiment of the present invention, the process of selecting target features with the help of weights is actually a process of selecting the essence and removing the dross. It can also be understood as selecting candidate features that contribute greatly to the forgery detection task and screening out candidate features that have little effect in the forgery detection task. This not only ensures the performance of the subsequent model built based on the target features, but also reduces the specifications of the model and the computing power required for operation.

[0061] Step 130, constructing a second forgery detection model for forgery detection based on the target feature;

[0062] Specifically, after step 120, after the target feature is determined from multiple candidate features, step 130 can be executed to construct a second forgery detection model for forgery detection based on the target feature. The specific process can be to use network structure search technology to extract the information flow for forgery detection based on the target feature from the first forgery detection model, and use this information flow as the architecture of the second forgery detection model. In other words, the entire information flow from the candidate feature as the target feature to the candidate forgery detection feature in the first forgery detection model is extracted as the architecture of the second forgery detection model.

[0063] From candidate features to candidate forgery detection features, encoding, decoding, fusion, and weighted fusion are required. After the candidate features are encoded by the encoding layer, the candidate feature map can be obtained, and after the candidate feature map is decoded by the decoding layer, the candidate feature vector can be obtained. Among them, the candidate features, candidate feature maps, and candidate feature vectors belong to features of different feature levels, and the features of each feature level contain one or more candidate elements.

[0064] For features of each feature level, fusion and weighted fusion can be performed based on the feature level of the feature and the candidate elements contained in the feature, and finally the candidate forgery detection features for the forgery detection task are obtained. The essence of the fusion operation is the superposition of the candidate elements contained in the feature, and the superposition of the candidate elements will lead to an increase in the number of features, making the model difficult to converge or too large. Therefore, in the embodiment of the present invention, two pruning principles are used to limit the scope of the fusion operation. The pruning principles are based on the feature level of the feature and the candidate elements contained. In other words, the fusion operation is aimed at features that are in the same feature and contain different candidate elements, that is, for features that are in the same feature level and do not contain overlapping candidate elements, the candidate elements can be superimposed to obtain the first fused feature.

[0065] Furthermore, considering that after encoding, decoding and fusion operations, multiple features at the same feature level and containing the same candidate elements can be obtained, but they are generated by different paths, at this time, they can be weighted fused. In other words, the weighted fusion operation is for the first fused features at the same feature level and containing the same candidate elements, that is, for the first fused features at the same feature level and containing completely overlapping candidate elements, they can be combined with the weights corresponding to the features, and weighted fused to obtain the weighted fused features, that is, the second fused features. The weights corresponding to the features here are the parameters of the first forgery detection model, which can be determined by the first forgery detection model.

[0066] The essence of information flow extraction is that, for each weighted fusion operation, the first fusion feature corresponding to the maximum weight is determined from the first fusion features that are at the same feature level and contain the same candidate elements. Then, the entire information flow from the target feature through each first fusion feature corresponding to the maximum weight to the candidate forgery detection feature is extracted from the first forgery detection model to obtain the architecture of the second forgery detection model.

[0067] After that, the architecture of the second forgery detection model can also be verified by reverse search. Specifically, the network structure search technology can be used to start the reverse search from the output of the first forgery detection model. For each node involved in the weighted fusion operation in the model, the information flow where the first fusion feature corresponding to the maximum weight is located is selected, and other information flows are deleted. Then, the reverse search is continued until the image-level feature in the first forgery detection model is terminated, that is, the reverse search is ended at the candidate feature in the first forgery detection model, and the result of the reverse search, that is, the retained information flow is extracted as the second forgery detection model.

[0068] It should be noted that, in the process of constructing the second forgery detection model based on the target features, a penalty term for the parameter amount of the second forgery detection model can also be added, that is, a penalty function corresponding to each parameter in the second forgery detection model is added. The larger the coefficient of the penalty term, the smaller the parameter amount of the second forgery detection model searched, and the coefficient of the penalty term is determined according to the number of encoding layers and / or decoding layers in the second forgery detection model. The addition of the penalty term can make the construction process of the second forgery detection model more flexible. In other words, the use of the penalty function can achieve lightweight model with better performance.

[0069] In an embodiment of the present invention, the process of constructing the second forgery detection model with the help of target features can dynamically adjust the coefficient of the penalty term according to the computing power level to adjust the richness of the first forgery detection model, thereby expanding the search range when the computing power level is high, in order to search for a better model structure; when the computing power level is low, not only can the excellent performance of the model be guaranteed, but also a lighter model can be selected.

[0070] Step 140: training a second forgery detection model based on the second sample face image and its authenticity label to obtain a face forgery detection model.

[0071] Specifically, in step 130, after the second forgery detection model capable of forgery detection is constructed by means of the target features, step 140 can be executed to train the second forgery detection model to obtain a trained second forgery detection model, i.e., a face forgery detection model. The specific process includes the following steps:

[0072] First, a large number of sample face images are collected. To distinguish them from the first sample face images mentioned above, the sample face images here are called second sample face images, and the authenticity of each second sample face image is marked to form an authenticity label;

[0073] Then, the second sample face image and its authenticity label are used to train the second forgery detection model. Specifically, the second sample face image is input into the second forgery detection model. The second forgery detection model can first perform face detection on the second sample face image to determine the face area therein; then the image is cropped to retain the face area and remove the non-face area (blank area and background area); thereafter, the second forgery detection model can perform forgery detection based on the clear first sample face image obtained by cropping to obtain a second forgery detection result corresponding to the second sample face image; finally, the second forgery detection model can be iterated on parameters according to the authenticity label of the second sample face image and the second forgery detection result, so as to obtain a face forgery detection model.

[0074] The method for constructing a face forgery detection model provided by the present invention uses the weights of multiple candidate features obtained through a first forgery detection model to screen out target features from multiple candidate features, and constructs a second forgery detection model for forgery detection based on the target features, and trains the second forgery detection model to obtain a face forgery detection model, which not only ensures the excellent performance of the model, but also reduces the model specifications, reduces resource usage, and achieves lightweight model, overcomes the defects of traditional solutions that it is time-consuming and labor-intensive to distinguish forged information and cannot know the role of collaborative analysis of different candidate features, and improves the subsequent face forgery detection process based on the face forgery detection model.

[0075] Based on the above embodiment, multiple candidate features are determined based on the following steps:

[0076] Based on the feature extraction layer in the first forgery detection model, feature extraction is performed on the first sample face image to obtain multiple candidate features of the first sample face image;

[0077] Based on the encoding layer in the first forgery detection model, encoding the candidate features to obtain a candidate feature map of the first sample face image;

[0078] Based on the decoding layer in the first forgery detection model, the candidate feature map is decoded to obtain a candidate feature vector of the first sample face image, wherein the candidate features, the candidate feature map and the candidate feature vector are features of different feature levels;

[0079] Based on the fusion layer in the first forgery detection model and the candidate elements contained in the features of each feature level, the features at the same feature level are fused to obtain candidate forgery detection features.

[0080] Specifically, in the above process, the process of extracting multiple candidate features of the first sample face image by the first forgery detection model specifically includes the following steps:

[0081] Figure 2 is a schematic diagram of the structure of the first forgery detection model provided by the present invention, such as Figure 2 As shown, the first forgery detection model includes a feature extraction layer and a network structure search layer, and the network structure search layer includes a coding layer, a decoding layer, a fusion layer and a classification layer. Among them, the feature extraction layer is used to extract features of the first sample face image to obtain multiple candidate features of the first sample face image; the feature extraction layer includes two modules, namely, an image deconstruction module and an element combination module. The image deconstruction module is used to deconstruct the first sample face image using computer graphics, frequency domain decomposition and other methods to obtain multiple candidate features containing a single candidate element; the element combination module uses the reverse recombination algorithm during deconstruction to fuse the candidate features with spatial correspondence in the candidate features obtained in the previous step, and add the fused features to the candidate features obtained in the previous step, forming multiple candidate features containing different candidate elements.

[0082] For example, Figure 2 The first sample face image a in the feature extraction layer is deconstructed by the image deconstruction module to obtain multiple candidate features containing a single candidate element, which are I {a} ,I {b} ,I {c} ,I {d} ,I {e} and I {f} , where I {c} ,I {d} ,I {e} and I {f} There is a spatial correspondence between the elements, so the element combination module can be used to {c} ,I {d} ,I {e} and I {f} The different combinations of candidate elements in the fusion process are {g: (c, d)}, {h: (c, e)}, {i: (c, f)}, {j: (e, f)}, {k: (d, e)}, {l: (d, f)}, {m: (c, d, e)}, {n: (c, d, f)}, {o: (c, e, f)}, {p: (d, e, f)}, {q: (c, d, e, f)}, and the fusion feature is: I {g} ,I {h} ,I {i} ,I {j} ,I {k} ,I {l} ,I{m} ,I {n} ,I {o} ,I {p} and I {q} , therefore, the multiple candidate features of the first sample face image are I {a} to I {q} All candidate features.

[0083] It should be noted that the fusion here is actually the combination (superposition) of candidate elements in the candidate features, and different combinations of candidate elements can obtain different candidate features. It can also be understood that the fusion operation greatly enriches the forged information contained in the candidate features.

[0084] The encoding layer in the first forgery detection model is used to encode the various candidate features of the first sample face image respectively to obtain various candidate feature maps of the first sample face image; correspondingly, the decoding layer is used to decode the various candidate feature maps of the first sample face image respectively to obtain various candidate feature vectors of the first sample face image.

[0085] Here, the encoding layer and the decoding layer are equivalent to the first half and the second half of CNN (Convolutional Neural Networks). The first half of CNN can encode the candidate features to obtain a two-dimensional candidate feature map; correspondingly, the second half of CNN can decode the two-dimensional candidate feature map to obtain a candidate feature vector.

[0086] The candidate features, candidate feature maps, and candidate feature vectors here are features at different feature levels, respectively. The candidate features are image-level features, the candidate feature maps are feature map-level features, and the candidate feature vectors are feature vector-level features.

[0087] Fusion layers are inserted before, after, and in the middle of the encoding layer and the decoding layer. The fusion layers are used to fuse features at the same feature level. The fusion basis is the feature level of the features and the candidate elements contained in the features. That is, the candidate elements contained in the features of each feature level can be used as a benchmark, and the fusion layer in the first forgery detection model can be applied to fuse features at the same feature level to obtain candidate forgery detection features.

[0088] The process of fusing features at the same feature level is divided into fusion and weighted fusion; the fusion operation is for features that are at the same feature and contain different candidate elements, that is, for features that are at the same feature level and do not contain overlapping candidate elements, they can be fused to obtain the first fused feature; the weighted fusion operation is for the first fused feature that is at the same feature level and contains the same candidate elements, that is, for the first fused feature that is at the same feature level and contains all overlapping candidate elements, it can be weighted fused based on the corresponding weights of the features to obtain the second fused feature.

[0089] After the second fused feature is obtained, a candidate forgery detection feature for the forgery detection task can be determined according to the second fused feature and its weight.

[0090] The classification layer is used to perform forgery detection on the first sample face image according to the candidate forgery detection features, and obtain a first forgery detection result that can characterize the authenticity of the first sample face image.

[0091] Based on the above embodiment, the Xception structure can be used as the skeleton of CNN, the part before the seventh block can be used as the encoding layer, and the part after the seventh block can be used as the decoding layer. The encoding and decoding process can be expressed as follows:

[0092]

[0093] Among them, I is the candidate feature with a size of 3*299*299; ε is the encoding layer, the input of the encoding layer is I, and the output is the candidate feature map m of 19*19*728; It is the decoding layer. The input of the decoding layer is 19*19*728*C, and the output is a 2048-dimensional candidate feature vector v.

[0094] In the model construction stage, in order to make the model lighter, the fusion operation uses sum operation at both the feature map level and the feature vector level. At this time, C is 1; in the model training stage, the fusion operation uses cat (concatenate) at the feature map level and sum at the feature vector level. At this time, C is the number of cat operations before entering the decoding layer.

[0095] It should be noted that any candidate feature can only be encoded by the encoding layer and then decoded by the decoding layer, that is, it can only be encoded first and then decoded.

[0096] Based on the above embodiment, based on the fusion layer in the first forgery detection model and the candidate elements contained in the features of each feature level, the features of the same feature level are fused to obtain candidate forgery detection features, including:

[0097] Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features of the same feature level are fused to obtain first fused features of different feature levels, the first fused features including first candidate fused features, first candidate fused feature maps and first candidate fused feature vectors;

[0098] Based on the fusion layer in the first forgery detection model, the candidate elements included in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels, the second fused features including a second candidate fused feature map and a second candidate fused feature vector;

[0099] Based on the fusion layer in the first forgery detection model and the weight corresponding to the second candidate fused feature vector, the second candidate fused feature vector is fused to obtain a candidate forgery detection feature.

[0100] Specifically, the process of fusing the features at the same feature level according to the fusion layer in the first forgery detection model and the candidate elements contained in the features at each feature level to obtain the candidate forgery detection features may specifically include the following steps:

[0101] First, based on the fusion layer in the first forgery detection model, the features at the same feature level can be fused according to the candidate elements contained in the features at each feature level, so as to obtain first fused features at different feature levels, that is, for two or more features whose candidate elements contained in the same feature level do not overlap, the candidate elements are combined (superimposed) to obtain a combined first fused feature, where the first fused feature includes a first candidate fused feature obtained by fusion at the image level, a first candidate fused feature map obtained by fusion at the feature map level, and a first candidate fused feature vector obtained by fusion at the feature vector level;

[0102] For example, I {c，d} =I {c} ⊙I {d} , where I is the image-level feature, i.e., the candidate feature, {c}, {d}, and {c, d} are the candidate elements contained in the candidate feature, ⊙ represents fusion, I {c，d} is the fused image-level feature, that is, the first candidate fused feature in the first fused feature;

[0103] m {c，d} =m {c} ⊙m {d} , where m is the feature at the feature map level, i.e., the candidate feature map, {c}, {d} and {c, d} are the candidate elements contained in the candidate feature map, ⊙ represents fusion, m {c，d}is the feature at the feature map level after fusion, that is, the first candidate fusion feature map in the first fusion feature;

[0104] v {c，d} =v {c} ⊙v {d} , where v is the feature vector level feature, i.e., the candidate feature vector, {c}, {d} and {c, d} are the candidate elements contained in the candidate feature vector, ⊙ represents fusion, v {c，d} It is the feature at the feature vector level after fusion, that is, the first candidate fused feature vector in the first fused feature.

[0105] Accordingly, for those at different feature levels, or those containing candidate elements that are partially or completely the same, that is, the candidate elements contain overlapping features, they cannot be directly fused and need to be encoded or decoded before fusion.

[0106] For example, m {c，d} and m {d，e} There is overlap in the candidate elements contained in m {c，d} and m {d，e} Direct fusion is not possible.

[0107] For example, m {c，d} and v {c，d} are features of different feature levels, so m {c，d} and v {c，d} Direct fusion is not possible.

[0108] Then, considering that after the encoding layer, decoding layer and fusion layer, multiple features at the same feature level and containing the same candidate elements can be obtained, but they are obtained in different ways, for example, the candidate feature vector v containing the candidate elements {c, d} {c,d} It can be obtained in the following ways:

[0109]

[0110] in, and are candidate feature vectors obtained in different ways, ε is the encoding layer, The decoding layer.

[0111] Therefore, in the embodiment of the present invention, the first fused features of the same feature level can be fused in the fusion layer in the first forgery detection model by combining the candidate elements included in the first fused features of each feature level and the weights corresponding to the first fused features of each feature level, so as to obtain the second fused features of different feature levels. That is, the weights corresponding to the first fused features of each feature level can be used as a basis to apply the fusion layer in the first forgery detection model to perform weighted fusion on the first fused features with the same candidate elements included in the same feature level, so as to obtain the second fused features.

[0112] For example, you can and For weighted fusion, the calculation formula is:

[0113]

[0114] Among them, α 1 for The corresponding weight, α 2 for The corresponding weight, α 3 Then The corresponding weight, and α 1 +α 2 +α 3 =1.

[0115] It is worth noting that the weighted fusion process is only for the first candidate fusion feature map and the first candidate fusion feature vector in the first fusion feature. Therefore, the second fusion feature obtained by weighted fusion also only includes the second candidate fusion feature map and the second candidate fusion feature vector.

[0116] Thereafter, the second candidate fused feature vector can be fused according to the fusion layer in the first forgery detection model and the weight corresponding to the second candidate fused feature vector to obtain the candidate forgery detection feature. That is, based on the weight corresponding to the second candidate fused feature vector, the fusion layer in the first forgery detection model is applied to fuse the second candidate fused feature vector to obtain the candidate forgery detection feature.

[0117] Based on the above embodiment, the weighted fusion process can be expressed as the following formula:

[0118]

[0119] in, represents the candidate element, To include The second candidate fusion feature map or the second candidate fusion feature vector, For inclusion generated by different means The first candidate fusion feature map set or the first candidate fusion feature vector set, for The corresponding weight, The value range of is 0 to 1, and, in the first candidate fusion feature map set or the first candidate fusion feature vector set, the sum of the weights corresponding to each first candidate fusion feature map or each first candidate fusion feature vector is 1.

[0120] The weighted fusion process of the second candidate fusion feature vector can be expressed as:

[0121]

[0122] Among them, v out is a candidate forgery detection feature, is the second candidate fusion feature vector, for The corresponding weight, For all candidate element combinations, yes A subset of .

[0123] Based on the above embodiment, based on the fusion layer in the first forgery detection model and the candidate elements contained in the features of each feature level, the features of the same feature level are fused to obtain first fused features of different feature levels, including:

[0124] If the candidate elements included in multiple features at any feature level are different, the multiple features at the feature level are fused based on the fusion layer in the first forgery detection model to obtain a first fused feature at the feature level.

[0125] Specifically, in the above process, according to the fusion layer in the first forgery detection model and the candidate elements contained in the features of each feature level, the features of the same feature level are fused to obtain the first fused features of different feature levels. Specifically, it can be that when the candidate elements contained in multiple features of any feature level are different, the fusion layer in the first forgery detection model can be applied to fuse the multiple features of the feature level to obtain the first fused feature of the feature level.

[0126] Accordingly, when the candidate elements contained in two candidate features at any feature level overlap, the two cannot be directly fused and need to be encoded by the encoding layer or decoded by the decoding layer before they can be fused.

[0127] Based on the above embodiment, based on the fusion layer in the first forgery detection model, the candidate elements contained in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels, including:

[0128] If the candidate elements included in multiple first fused features of any feature level are the same, the multiple first fused features of the feature level are fused based on the fusion layer in the first forgery detection model and the weights corresponding to the multiple first fused features of the feature level to obtain the second fused feature of the feature level.

[0129] Specifically, in the above process, according to the fusion layer in the first forgery detection model, the candidate elements contained in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels. Specifically, in the case where the candidate elements contained in multiple first fused features of any feature level are the same, the weights corresponding to the multiple first fused features of the feature level can be used as a reference, and the fusion layer in the first forgery detection model can be applied to fuse the multiple first fused features of the feature level to obtain the second fused features of the feature level.

[0130] Based on the above embodiment, based on the feature extraction layer in the first forgery detection model, feature extraction is performed on the first sample face image to obtain multiple candidate features of the first sample face image, including:

[0131] Based on the feature extraction layer in the first forgery detection model, extract features of the first sample face image to obtain initial candidate features of the first sample face image;

[0132] Based on the feature extraction layer in the first forgery detection model, initial candidate features having spatial correspondence are fused to obtain initial candidate fused features;

[0133] Based on the initial candidate fusion features and the initial candidate features of the first sample face image, multiple candidate features of the first sample face image are determined.

[0134] Specifically, the process of extracting features from the first sample face image according to the feature extraction layer in the first forgery detection model to obtain multiple candidate features of the first sample face image specifically includes the following steps:

[0135] First, the first sample face image is deconstructed by an image deconstruction module in a feature extraction layer of the first forgery detection model. Specifically, the first sample face image is deconstructed by using computer graphics, frequency domain decomposition and other methods, so as to obtain multiple candidate features containing a single candidate element, namely, initial candidate features;

[0136] Then, it is necessary to determine the initial candidate features that have a spatial correspondence among the initial candidate features, and fuse these initial candidate features through the element combination module in the feature extraction layer of the first forgery detection model. Specifically, the initial candidate features that have a spatial correspondence are fused using the reverse reconstruction algorithm during deconstruction. The fusion here is essentially a combination of candidate elements. There are many different combinations between the candidate elements, so it is possible to obtain a variety of initial candidate fused features containing different combinations of candidate elements.

[0137] Here, the initial candidate features with spatial correspondence are the initial candidate features corresponding to the candidate elements deconstructed by computer imaging methods. The reason why only the initial candidate features with spatial correspondence are fused at the image level is that only the initial candidate features with spatial correspondence still have physical meaning after being fused by the inverse reconstruction algorithm during deconstruction. For example, the initial candidate features containing the candidate element "ambient light" and the initial candidate features containing the candidate element "direct light" can be fused by the inverse reconstruction algorithm to obtain the initial candidate fused features containing "face illumination".

[0138] However, the initial candidate features corresponding to the candidate elements that are not deconstructed by computer imaging methods cannot be superimposed on the candidate elements from a physical perspective, and therefore cannot be fused at the image level.

[0139] Thereafter, multiple candidate features of the first sample face image can be determined based on the initial candidate fusion features and the initial candidate features of the first sample face image, that is, the initial candidate fusion features and the initial candidate features are both used as candidate features of the first sample face image.

[0140] Based on the above embodiment, in step 130, the process of extracting the target feature through the information flow from the feature corresponding to each maximum weight to the candidate forgery detection feature as the architecture of the second forgery detection model can be expressed as:

[0141] Each second fusion feature obtained through the weighted fusion operation is regarded as the weighted sum of multiple candidate features, which can be expressed by the following formula:

[0142]

[0143] in, For inclusion candidate features generated by different methods The first candidate fusion feature map set or the first candidate fusion feature vector set, for The corresponding weight is used to indicate whether the feature is selected. It is the second fusion feature.

[0144] Then, the first fusion feature corresponding to the maximum weight is used to replace the corresponding second fusion feature, and a discrete system structure can be obtained; and in the second forgery detection model, for each weighted fusion operation, only the first fusion feature corresponding to the maximum weight is retained, and other features are discarded. This process can be expressed as:

[0145]

[0146] in, is the first fusion feature corresponding to the maximum weight, It is the second fusion feature.

[0147] In step 130, the loss function of the process of constructing a second forgery detection model for forgery detection according to the target feature is:

[0148]

[0149] in, is the cross entropy loss function, is the network complexity loss function, λ cost is a penalty term. The larger the coefficient of the penalty term is, the stricter the penalty for network complexity is, and the smaller the number of parameters of the second forgery detection model is.

[0150] The calculation formula of the network complexity loss function is:

[0151]

[0152] in, is the weight of each first fusion feature for weighted fusion operation, It represents the network complexity brought by each first fusion feature, and N is the number of first fusion features.

[0153] The network complexity brought by the first fusion feature can be expressed as:

[0154]

[0155]

[0156] Among them, α * (t) is the second forgery detection model obtained by searching, is the first candidate fusion feature map or the first candidate fusion feature vector in the first fusion feature, Represents the transition from target feature to The set that needs to be encoded, decoded and fused is ε is the coding layer, is the decoding layer, ⊙ is the layer from target feature to After the fusion operation, p(o) is the weight corresponding to each operation in the set.

[0157] It should be noted that α * The update frequency of (t) can be set according to the actual situation. As a preferred embodiment, in the embodiment of the present invention, in order to improve the construction process of the model, α * (t) is updated every 50 batches.

[0158] The present invention also provides a method for detecting forged faces. Figure 3 FIG. 1 is a flow chart of the face forgery detection method provided by the present invention, such as Figure 3 As shown, the method includes:

[0159] Step 310, determining a face image to be detected;

[0160] Specifically, before performing face forgery detection, it is first necessary to determine the face image to be detected. The face image can be a real image or a forged image. For example, it can be a real image containing a face area acquired by an image acquisition device, or it can be a forged image spliced ​​by an image editing method.

[0161] The number of face images may be one or more. In the case of multiple face images, multiple face images need to be detected for forgery one by one to determine the authenticity of each face image to be detected.

[0162] Step 320, inputting the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method as described in any one of the above items.

[0163] Specifically, after step 310, after obtaining the face image to be detected, step 320 can be executed, and the face forgery detection model is applied to perform forgery detection on the face image to be detected, so as to obtain a forgery detection result. This process can be specifically as follows: first, the face image to be detected is input into the face forgery detection model, and then, the preprocessing layer in the face forgery detection model preprocesses the face image to be detected, that is, performs face detection on the face image, determines the face area therein, and then performs image cropping to retain the face area and remove the non-face area (blank area and background area) to obtain a clear face image. Thereafter, the face forgery detection model can perform forgery detection based on the clear face image to obtain a forgery detection result corresponding to the face image to be detected.

[0164] Before inputting the face image to be detected into the face forgery detection model, the face forgery detection model can also be pre-trained. The face forgery detection model is trained based on the second forgery detection model by applying the second sample face image and its authenticity label. The second forgery detection model is determined based on the following steps:

[0165] First, a first forgery detection model is determined, where the first forgery detection model is used to extract multiple candidate features of the first sample face image, and perform forgery detection on the first sample face image based on the multiple candidate features and their weights;

[0166] Then, according to the weights of the multiple candidate features, a target feature is determined from the multiple candidate features;

[0167] Subsequently, a second forgery detection model is constructed for forgery detection based on the target features.

[0168] The face forgery detection method provided by the present invention determines a face image to be detected; inputs the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model, and applies the face forgery detection model trained on the basis of a second forgery detection model to perform forgery detection, thereby reducing the computing power requirement while ensuring the precision and accuracy of forgery detection; in addition, by performing forgery detection through a simplified model, the efficiency of forgery detection can be greatly improved.

[0169] The face forgery detection model construction device provided by the present invention is described below. The face forgery detection model construction device described below and the face forgery detection model construction method described above can be referenced to each other.

[0170] Figure 4 : is a schematic diagram of the structure of the face forgery detection model building device provided by the present invention, such as Figure 4 As shown, the device comprises:

[0171] a first forgery detection model determining unit 410, configured to determine a first forgery detection model, wherein the first forgery detection model is configured to extract multiple candidate features of the first sample face image, and perform forgery detection on the first sample face image based on the multiple candidate features and their weights;

[0172] A target feature determination unit 420, configured to determine a target feature from the plurality of candidate features based on weights of the plurality of candidate features;

[0173] A second forgery detection model building unit 430, configured to build a second forgery detection model for forgery detection based on the target feature;

[0174] The face forgery detection model determination unit 440 is used to train the second forgery detection model based on the second sample face image and its authenticity label to obtain the face forgery detection model.

[0175] The face forgery detection model construction device provided by the present invention uses the weights of multiple candidate features obtained through a first forgery detection model to screen out target features from multiple candidate features, and constructs a second forgery detection model based on the target features for forgery detection, and trains the second forgery detection model to obtain a face forgery detection model, which not only ensures the excellent performance of the model, but also reduces the model specifications, reduces resource usage, and achieves lightweight model, overcomes the defects of traditional solutions that it is time-consuming and labor-intensive to distinguish forged information and cannot know the role of collaborative analysis of different candidate features, and improves the subsequent face forgery detection process based on the face forgery detection model.

[0176] Based on the above embodiment, the device further includes a candidate feature determination unit, which is used to:

[0177] Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain multiple candidate features of the first sample face image;

[0178] Based on the encoding layer in the first forgery detection model, encoding the candidate features to obtain a candidate feature map of the first sample face image;

[0179] Based on the decoding layer in the first forgery detection model, the candidate feature map is decoded to obtain a candidate feature vector of the first sample face image, wherein the candidate features, the candidate feature map and the candidate feature vector are features of different feature levels;

[0180] Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features at the same feature level are fused to obtain candidate forgery detection features.

[0181] Based on the above embodiment, the device further includes a feature fusion unit, which is used to:

[0182] Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features of the same feature level are fused to obtain first fused features of different feature levels, wherein the first fused features include first candidate fused features, first candidate fused feature maps, and first candidate fused feature vectors;

[0183] Based on the fusion layer in the first forgery detection model, the candidate elements included in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels, wherein the second fused features include a second candidate fused feature map and a second candidate fused feature vector;

[0184] Based on the fusion layer in the first forgery detection model and the weight corresponding to the second candidate fused feature vector, the second candidate fused feature vector is fused to obtain a candidate forgery detection feature.

[0185] Based on the above embodiment, the feature fusion unit is used for:

[0186] If the candidate elements included in multiple features at any feature level are different, the multiple features at the feature level are fused based on the fusion layer in the first forgery detection model to obtain a first fused feature at the feature level.

[0187] Based on the above embodiment, the feature fusion unit is used for:

[0188] If the candidate elements included in multiple first fused features of any feature level are the same, the multiple first fused features of the feature level are fused based on the fusion layer in the first forgery detection model and the weights corresponding to the multiple first fused features of the feature level to obtain the second fused feature of the feature level.

[0189] Based on the above embodiment, the device further includes a feature extraction unit, which is used to:

[0190] Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain initial candidate features of the first sample face image;

[0191] Based on the feature extraction layer in the first forgery detection model, initial candidate features having spatial correspondence are fused to obtain initial candidate fused features;

[0192] Based on the initial candidate fusion features and the initial candidate features of the first sample face image, multiple candidate features of the first sample face image are determined.

[0193] The face forgery detection device provided by the present invention is described below. The face forgery detection device described below and the face forgery detection method described above can be referenced to each other.

[0194] Figure 5 : is a schematic diagram of the structure of the face forgery detection model building device provided by the present invention, such as Figure 5 As shown, the device comprises:

[0195] A face image determination unit 510, used to determine a face image to be detected;

[0196] The face forgery detection unit 520 is used to input the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method as described in any one of the above items.

[0197] The face forgery detection device provided by the present invention determines a face image to be detected; inputs the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model, and applies the face forgery detection model trained on the basis of a second forgery detection model to perform forgery detection, thereby reducing the computing power requirement while ensuring the precision and accuracy of forgery detection; in addition, by performing forgery detection through a simplified model, the efficiency of forgery detection can be greatly improved.

[0198] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the face forgery detection model construction method or the face forgery detection method, wherein the face forgery detection model construction method includes: determining a first forgery detection model, the first forgery detection model is used to extract multiple candidate features of a first sample face image, and based on the multiple candidate features and their weights, perform forgery detection on the first sample face image; based on the weights of the multiple candidate features, determine the target feature from the multiple candidate features; construct a second forgery detection model based on the target feature for forgery detection; based on the second sample face image and its authenticity label, train the second forgery detection model to obtain a face forgery detection model. The face forgery detection method includes: determining a face image to be detected; inputting the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method as described in any one of the above items.

[0199] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0200] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer-readable storage medium, the computer program includes program instructions, when the program instructions are executed by the computer, the computer can execute the face forgery detection model construction method or face forgery detection method provided by the above methods, wherein the face forgery detection model construction method includes: determining a first forgery detection model, the first forgery detection model is used to extract multiple candidate features of a first sample face image, and based on the multiple candidate features and their weights, the first sample face image is forged for detection; based on the weights of the multiple candidate features, a target feature is determined from the multiple candidate features; a second forgery detection model is constructed based on the target feature for forgery detection; based on the second sample face image and its authenticity label, the second forgery detection model is trained to obtain a face forgery detection model. The face forgery detection method includes: determining a face image to be detected; inputting the face image into the face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method as described in any one of the above items.

[0201] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the face forgery detection model construction method or face forgery detection method provided by the above methods when the computer program is executed, wherein the face forgery detection model construction method includes: determining a first forgery detection model, the first forgery detection model is used to extract multiple candidate features of a first sample face image, and based on the multiple candidate features and their weights, the first sample face image is forged for detection; based on the weights of the multiple candidate features, a target feature is determined from the multiple candidate features; a second forgery detection model is constructed for forgery detection based on the target feature; based on the second sample face image and its authenticity label, the second forgery detection model is trained to obtain a face forgery detection model. The face forgery detection method includes: determining a face image to be detected; inputting the face image into the face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method described in any one of the above items.

[0202] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0203] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a face forgery detection model, It is characterized in that include: Determining a first forgery detection model, where the first forgery detection model is used to extract multiple candidate features of the first sample face image, and perform forgery detection on the first sample face image based on the multiple candidate features and their weights; Determining a target feature from the multiple candidate features based on the weights of the multiple candidate features; constructing a second forgery detection model for forgery detection based on the target feature; Based on the second sample face image and its authenticity label, training the second forgery detection model to obtain a face forgery detection model; The multiple candidate features are determined based on the following steps: Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain multiple candidate features of the first sample face image; Based on the encoding layer in the first forgery detection model, encoding the candidate features to obtain a candidate feature map of the first sample face image; Based on the decoding layer in the first forgery detection model, the candidate feature map is decoded to obtain a candidate feature vector of the first sample face image, wherein the candidate features, the candidate feature map and the candidate feature vector are features of different feature levels; Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features at the same feature level are fused to obtain candidate forgery detection features.

2. The method for constructing a face forgery detection model according to claim 1, It is characterized in that The step of fusing the features at the same feature level based on the fusion layer in the first forgery detection model and the candidate elements contained in the features at each feature level to obtain the candidate forgery detection features includes: Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features of the same feature level are fused to obtain first fused features of different feature levels, wherein the first fused features include first candidate fused features, first candidate fused feature maps, and first candidate fused feature vectors; Based on the fusion layer in the first forgery detection model, the candidate elements included in the first fused features of each feature level, and the weights corresponding to the first fused features of each feature level, the first fused features of the same feature level are fused to obtain second fused features of different feature levels, wherein the second fused features include a second candidate fused feature map and a second candidate fused feature vector; Based on the fusion layer in the first forgery detection model and the weight corresponding to the second candidate fused feature vector, the second candidate fused feature vector is fused to obtain a candidate forgery detection feature.

3. The method for constructing a face forgery detection model according to claim 2, It is characterized in that The step of fusing the features at the same feature level based on the fusion layer in the first forgery detection model and the candidate elements contained in the features at each feature level to obtain first fused features at different feature levels includes: If the candidate elements included in the multiple features at any feature level are different, the multiple features at any feature level are fused based on the fusion layer in the first forgery detection model to obtain a first fused feature at any feature level.

4. The method for constructing a face forgery detection model according to claim 2, It is characterized in that The method of fusing the first fused features at the same feature level based on the fusion layer in the first forgery detection model, the candidate elements included in the first fused features at each feature level, and the weights corresponding to the first fused features at each feature level to obtain second fused features at different feature levels includes: If the candidate elements contained in multiple first fused features of any feature level are the same, the multiple first fused features of any feature level are fused based on the fusion layer in the first forgery detection model and the weights corresponding to the multiple first fused features of any feature level to obtain the second fused feature of any feature level.

5. The method for constructing a face forgery detection model according to any one of claims 1 to 4, It is characterized in that The feature extraction layer based on the first forgery detection model performs feature extraction on the first sample face image to obtain multiple candidate features of the first sample face image, including: Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain initial candidate features of the first sample face image; Based on the feature extraction layer in the first forgery detection model, initial candidate features having spatial correspondence are fused to obtain initial candidate fused features; Based on the initial candidate fusion features and the initial candidate features of the first sample face image, multiple candidate features of the first sample face image are determined.

6. A method for detecting forged faces, It is characterized in that include: Determine a face image to be detected; Inputting the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model; The face forgery detection model is determined based on the face forgery detection model construction method according to any one of claims 1 to 5.

7. A device for constructing a face forgery detection model, It is characterized in that include: a first forgery detection model determining unit, configured to determine a first forgery detection model, wherein the first forgery detection model is used to extract a plurality of candidate features of the first sample face image, and perform forgery detection on the first sample face image based on the plurality of candidate features and their weights; a target feature determination unit, configured to determine a target feature from the plurality of candidate features based on weights of the plurality of candidate features; a second forgery detection model building unit, configured to build a second forgery detection model for forgery detection based on the target feature; A face forgery detection model determination unit, configured to train the second forgery detection model based on the second sample face image and its authenticity label to obtain a face forgery detection model; The multiple candidate features are determined based on the following steps: Based on the feature extraction layer in the first forgery detection model, extract features from the first sample face image to obtain multiple candidate features of the first sample face image; Based on the encoding layer in the first forgery detection model, encoding the candidate features to obtain a candidate feature map of the first sample face image; Based on the decoding layer in the first forgery detection model, the candidate feature map is decoded to obtain a candidate feature vector of the first sample face image, wherein the candidate features, the candidate feature map and the candidate feature vector are features of different feature levels; Based on the fusion layer in the first forgery detection model and the candidate elements included in the features of each feature level, the features at the same feature level are fused to obtain candidate forgery detection features.

8. A face forgery detection device, It is characterized in that include: A face image determination unit, used to determine a face image to be detected; A face forgery detection unit, used to input the face image into a face forgery detection model to obtain a forgery detection result output by the face forgery detection model; the face forgery detection model is determined based on the face forgery detection model construction method according to any one of claims 1 to 5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the method for constructing a face forgery detection model according to any one of claims 1 to 5 or the method for detecting face forgery according to claim 6 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method for constructing a face forgery detection model according to any one of claims 1 to 5 or the method for detecting face forgery according to claim 6 is implemented.

Images